mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
* v0.36.5.0 feat: secure DATABASE_URL access for shell jobs (inherit: ["database_url"]) Replaces PR #1137's plaintext-config / plaintext-env workarounds with code. Shell-job params gain `inherit: ["database_url"]`, validated pre-enqueue in both the CLI (`gbrain jobs submit`) and `submit_job` MCP op handler. Worker resolves the value from its own loadConfig() at child-spawn time; the persisted `minion_jobs.data` row stores only the name. Plain `env: { GBRAIN_DATABASE_URL: ... }` / `env: { DATABASE_URL: ... }` / `env: { GBRAIN_DIRECT_DATABASE_URL: ... }` are rejected pre-enqueue with a paste-ready hint pointing at `inherit:`. Codex pre-landing review caught two bypasses + one missing shadow name: - H1: cmd/argv inline-secret regex scan (cmd:"GBRAIN_DATABASE_URL=... gbrain sync" was a clean bypass — fixed) - H3: GBRAIN_DIRECT_DATABASE_URL added to shadowKeys - H2: honest docs about output-side leakage (stdout_tail/stderr_tail can still carry the value if the script prints it; that's the script author's responsibility, not gbrain's) Also: gbrain doctor learns home_dir_in_worktree (warns when ~/.gbrain lives inside a git worktree); ~/.gbrain/.gitignore retroactive via saveConfig + post-upgrade. New canonical guide: docs/guides/agent-to-gbrain.md (two-domain framing for downstream agent authors: MCP ops via OAuth vs localOnly admin ops via shell-job inherit:). Closes #1137. Tests: +53 new (21 validator + 12 inherit-record + 6 ensureGitignore + 5 doctor + 2 PGLite E2E + 7 codex-driven H1/H3 cases). Credit: @wintermute filed PR #1137 which made the env-stripping gap visible enough to fix in code. Thank you. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * v0.36.5.0 redesign: free-form inherit:, drop closed enum User feedback: "agent spawning minions should have agency to do what it wants with secrets and pass only the ones that it needs. don't be a security nazi please." Replaces the closed INHERITABLE enum (database_url only) with three small helpers in shell-inherit.ts: - INHERIT_NAME_RE: snake_case shape guard. Rejects __proto__, leading underscore, uppercase, path-traversal. Prototype-pollution defense. - deriveEnvKey(name): config-key → child-env-key. Uppercase by default with one override: database_url → GBRAIN_DATABASE_URL. - resolveInheritValue(cfg, name): value lookup with Object.hasOwn. inherit: now accepts any snake_case config-key the worker has. Agent picks what it needs per-job (database_url, anthropic_api_key, voyage_api_key, or any custom field). Validator does NOT police WHICH keys — single-uid trust model treats agent as peer of worker. Drops the v0.36.5.0-RC rules that were paternalistic for the actual threat model: - closed-enum check - env-shadow rejection - cmd/argv inline-secret scan Keeps the parts that defend real problems: - pre-enqueue validation (closes the persistence-before-throw window) - snake_case regex (prototype-pollution + audit-log readability) - fail-fast on missing config value (UX guardrail, not security) Tests: shell-validate (existing rules + new free-form + prototype-pollution defense + T1 regression guard) and shell-inherit (regex matrix, deriveEnvKey per-name, resolveInheritValue with hasOwn defense). E2E case now exercises inherit:["anthropic_api_key"] to prove genuinely free-form. Docs and CHANGELOG rewritten to reflect the open design + the design-arc story (closed → cut → free-form). Migration file too. 7653 unit tests green. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * v0.36.5.0 add: redact_secrets opt-in for stdout/stderr scrubbing Honest defense for the documented output-side leakage. When a script prints an inherited secret, the value lands plaintext in result.stdout_tail / result.stderr_tail / error_text. v0.36.5.0 adds: - `redact_secrets: true` ShellJobParams field - `--redact-secrets` CLI convenience flag on `gbrain jobs submit shell` - shell-redact.ts: pure `redactSecretsInText(text, secrets)` helper (string-mode replaceAll; regex metachars in values stay literal) - Handler post-processes both tails before throw/return, so the persisted row carries `<REDACTED:name>` tokens instead of values Only inherit-resolved values are scrubbed. env: values are not (those are the agent's "fine in the row" channel by design). Heuristic — defeats accidental `echo "$GBRAIN_DATABASE_URL"`, not adversarial encode-then-print. Default false for back-compat. Tests: - test/minions-shell-redact.test.ts (9 cases): pure-function behavior, regex-metachar safety, multi-secret independent redaction, substring overlap, empty-input/map edge cases - test/minions-shell-validate.test.ts: +4 cases for redact_secrets shape - test/e2e/minions-shell-pglite.test.ts: +2 cases proving redact_secrets: true scrubs persisted row AND redact_secrets:false preserves plaintext (back-compat regression guard) Docs + CHANGELOG + migration file + CLAUDE.md updated. 7667 unit tests green. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
157 lines
6.2 KiB
TypeScript
157 lines
6.2 KiB
TypeScript
/**
|
|
* Tests for the `home_dir_in_worktree` doctor check (v0.35.8.0).
|
|
*
|
|
* Hermetic — drives the file system + GBRAIN_HOME + HOME envs directly via
|
|
* `withEnv`, then invokes `runDoctor(null, ['--fast', '--json'])` and parses
|
|
* the resulting JSON `checks` array. Skips the DB phase (engine=null + --fast).
|
|
*
|
|
* Covers F4 edge cases nailed in plan-eng-review:
|
|
* - .git as DIRECTORY (main repo) — warns
|
|
* - .git as FILE (linked worktree) — warns
|
|
* - walk terminates at $HOME — no false positive past it
|
|
* - GBRAIN_HOME override outside any worktree — ok
|
|
*/
|
|
|
|
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
|
import { mkdirSync, writeFileSync, rmSync } from 'fs';
|
|
import { join } from 'path';
|
|
import { tmpdir } from 'os';
|
|
import { withEnv } from './helpers/with-env.ts';
|
|
import { runDoctor } from '../src/commands/doctor.ts';
|
|
|
|
let scratch: string;
|
|
|
|
beforeEach(() => {
|
|
scratch = join(tmpdir(), `gbrain-doctor-hw-${Date.now()}-${Math.random().toString(36).slice(2)}`);
|
|
mkdirSync(scratch, { recursive: true });
|
|
});
|
|
|
|
afterEach(() => {
|
|
try { rmSync(scratch, { recursive: true, force: true }); } catch { /* best-effort */ }
|
|
});
|
|
|
|
/** Run the local doctor (no DB; null engine + --fast) under a stubbed HOME +
|
|
* GBRAIN_HOME, capture stdout AND prevent runDoctor's `process.exit(N)` from
|
|
* killing the test runner. Returns the check matching `name`. */
|
|
async function getCheck(name: string, env: Record<string, string | undefined>) {
|
|
const captured: string[] = [];
|
|
// Patch console.log directly — Bun's console.log doesn't route through the
|
|
// current process.stdout.write reference (it appears to cache the binding
|
|
// at module load), so monkey-patching write() doesn't catch it. console.log
|
|
// is the canonical doctor JSON-output channel.
|
|
const origLog = console.log;
|
|
console.log = (...args: unknown[]) => {
|
|
captured.push(args.map(a => typeof a === 'string' ? a : JSON.stringify(a)).join(' ') + '\n');
|
|
};
|
|
const origExit = process.exit;
|
|
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
|
(process as any).exit = (code?: number) => {
|
|
// Throw a tagged error so the test's try-block sees it; runDoctor's
|
|
// own try/catch doesn't catch this because it's outside its scope.
|
|
throw new Error(`__doctor_exit__:${code ?? 0}`);
|
|
};
|
|
try {
|
|
await withEnv(env, async () => {
|
|
try {
|
|
await runDoctor(null, ['--fast', '--json']);
|
|
} catch (e) {
|
|
// Swallow the synthetic __doctor_exit__ sentinel; rethrow other errors.
|
|
if (!(e instanceof Error) || !e.message.startsWith('__doctor_exit__:')) throw e;
|
|
}
|
|
});
|
|
} finally {
|
|
console.log = origLog;
|
|
process.exit = origExit;
|
|
}
|
|
const text = captured.join('');
|
|
// The doctor's JSON envelope is the last `{` block containing `"checks":`.
|
|
const idx = text.lastIndexOf('"checks"');
|
|
const objStart = idx >= 0 ? text.lastIndexOf('{', idx) : text.lastIndexOf('{');
|
|
const jsonStr = objStart >= 0 ? text.slice(objStart) : text;
|
|
let parsed: { checks: { name: string; status: string; message: string }[] };
|
|
try {
|
|
parsed = JSON.parse(jsonStr);
|
|
} catch {
|
|
throw new Error(`Could not parse doctor JSON; saw: ${text.slice(-500)}`);
|
|
}
|
|
return parsed.checks.find(c => c.name === name);
|
|
}
|
|
|
|
describe('home_dir_in_worktree doctor check', () => {
|
|
test('gbrain home outside any worktree → ok', async () => {
|
|
// scratch/.gbrain — no parent has a .git, scratch IS our fake $HOME
|
|
const home = scratch;
|
|
const gbrainParent = home;
|
|
const check = await getCheck('home_dir_in_worktree', {
|
|
HOME: home,
|
|
GBRAIN_HOME: gbrainParent,
|
|
});
|
|
expect(check).toBeDefined();
|
|
expect(check!.status).toBe('ok');
|
|
});
|
|
|
|
test('gbrain home inside dir-style .git worktree → warn', async () => {
|
|
// scratch/home/myrepo/.git/ (directory)
|
|
// scratch/home/myrepo/.gbrain/ ← gbrain home is inside the worktree
|
|
const home = join(scratch, 'home');
|
|
const repo = join(home, 'myrepo');
|
|
mkdirSync(join(repo, '.git'), { recursive: true });
|
|
mkdirSync(repo, { recursive: true });
|
|
const check = await getCheck('home_dir_in_worktree', {
|
|
HOME: home,
|
|
GBRAIN_HOME: repo,
|
|
});
|
|
expect(check).toBeDefined();
|
|
expect(check!.status).toBe('warn');
|
|
expect(check!.message).toContain('myrepo');
|
|
});
|
|
|
|
test('gbrain home inside .git-AS-FILE linked worktree → warn (F4)', async () => {
|
|
// Linked worktrees use a `.git` FILE (not a directory) containing
|
|
// `gitdir: /path/to/main/.git/worktrees/<name>`. Doctor MUST recognize
|
|
// both shapes — this is the Conductor + git-worktrees topology our
|
|
// dev environment runs in.
|
|
const home = join(scratch, 'home');
|
|
const repo = join(home, 'linked-wt');
|
|
mkdirSync(repo, { recursive: true });
|
|
writeFileSync(join(repo, '.git'), 'gitdir: /some/other/path/.git/worktrees/linked-wt\n');
|
|
const check = await getCheck('home_dir_in_worktree', {
|
|
HOME: home,
|
|
GBRAIN_HOME: repo,
|
|
});
|
|
expect(check).toBeDefined();
|
|
expect(check!.status).toBe('warn');
|
|
expect(check!.message).toContain('linked-wt');
|
|
});
|
|
|
|
test('walk terminates at $HOME — .git ABOVE $HOME does NOT trigger warn (F4)', async () => {
|
|
// scratch/.git/ (ABOVE the fake $HOME — should be ignored)
|
|
// scratch/home/ (fake $HOME)
|
|
// scratch/home/.gbrain/ (no worktree below $HOME)
|
|
mkdirSync(join(scratch, '.git'), { recursive: true });
|
|
const home = join(scratch, 'home');
|
|
mkdirSync(home, { recursive: true });
|
|
const check = await getCheck('home_dir_in_worktree', {
|
|
HOME: home,
|
|
GBRAIN_HOME: home,
|
|
});
|
|
expect(check).toBeDefined();
|
|
// OK because the .git is above $HOME, outside our walk scope.
|
|
expect(check!.status).toBe('ok');
|
|
});
|
|
|
|
test('GBRAIN_HOME override pointing outside any worktree → ok', async () => {
|
|
// Real $HOME might be inside a worktree, but the user pointed
|
|
// GBRAIN_HOME at a clean location. Doctor should report ok.
|
|
const home = scratch;
|
|
const safe = join(scratch, 'safe-elsewhere');
|
|
mkdirSync(safe, { recursive: true });
|
|
const check = await getCheck('home_dir_in_worktree', {
|
|
HOME: home,
|
|
GBRAIN_HOME: safe,
|
|
});
|
|
expect(check).toBeDefined();
|
|
expect(check!.status).toBe('ok');
|
|
});
|
|
});
|