Files
gbrain/test/minions-shell-validate.test.ts
T
e227965024 v0.36.5.0 feat: secure DATABASE_URL access for shell jobs (inherit: ["database_url"]) (#1192)
* v0.36.5.0 feat: secure DATABASE_URL access for shell jobs (inherit: ["database_url"])

Replaces PR #1137's plaintext-config / plaintext-env workarounds with code.
Shell-job params gain `inherit: ["database_url"]`, validated pre-enqueue in
both the CLI (`gbrain jobs submit`) and `submit_job` MCP op handler. Worker
resolves the value from its own loadConfig() at child-spawn time; the
persisted `minion_jobs.data` row stores only the name. Plain
`env: { GBRAIN_DATABASE_URL: ... }` / `env: { DATABASE_URL: ... }` /
`env: { GBRAIN_DIRECT_DATABASE_URL: ... }` are rejected pre-enqueue with a
paste-ready hint pointing at `inherit:`.

Codex pre-landing review caught two bypasses + one missing shadow name:
- H1: cmd/argv inline-secret regex scan (cmd:"GBRAIN_DATABASE_URL=... gbrain
  sync" was a clean bypass — fixed)
- H3: GBRAIN_DIRECT_DATABASE_URL added to shadowKeys
- H2: honest docs about output-side leakage (stdout_tail/stderr_tail can still
  carry the value if the script prints it; that's the script author's
  responsibility, not gbrain's)

Also: gbrain doctor learns home_dir_in_worktree (warns when ~/.gbrain lives
inside a git worktree); ~/.gbrain/.gitignore retroactive via saveConfig +
post-upgrade.

New canonical guide: docs/guides/agent-to-gbrain.md (two-domain framing for
downstream agent authors: MCP ops via OAuth vs localOnly admin ops via
shell-job inherit:).

Closes #1137. Tests: +53 new (21 validator + 12 inherit-record + 6
ensureGitignore + 5 doctor + 2 PGLite E2E + 7 codex-driven H1/H3 cases).

Credit: @wintermute filed PR #1137 which made the env-stripping gap visible
enough to fix in code. Thank you.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* v0.36.5.0 redesign: free-form inherit:, drop closed enum

User feedback: "agent spawning minions should have agency to do what it wants
with secrets and pass only the ones that it needs. don't be a security nazi
please."

Replaces the closed INHERITABLE enum (database_url only) with three small
helpers in shell-inherit.ts:

- INHERIT_NAME_RE: snake_case shape guard. Rejects __proto__, leading
  underscore, uppercase, path-traversal. Prototype-pollution defense.
- deriveEnvKey(name): config-key → child-env-key. Uppercase by default with
  one override: database_url → GBRAIN_DATABASE_URL.
- resolveInheritValue(cfg, name): value lookup with Object.hasOwn.

inherit: now accepts any snake_case config-key the worker has. Agent picks
what it needs per-job (database_url, anthropic_api_key, voyage_api_key, or
any custom field). Validator does NOT police WHICH keys — single-uid trust
model treats agent as peer of worker.

Drops the v0.36.5.0-RC rules that were paternalistic for the actual threat
model:
- closed-enum check
- env-shadow rejection
- cmd/argv inline-secret scan

Keeps the parts that defend real problems:
- pre-enqueue validation (closes the persistence-before-throw window)
- snake_case regex (prototype-pollution + audit-log readability)
- fail-fast on missing config value (UX guardrail, not security)

Tests: shell-validate (existing rules + new free-form + prototype-pollution
defense + T1 regression guard) and shell-inherit (regex matrix, deriveEnvKey
per-name, resolveInheritValue with hasOwn defense). E2E case now exercises
inherit:["anthropic_api_key"] to prove genuinely free-form.

Docs and CHANGELOG rewritten to reflect the open design + the design-arc
story (closed → cut → free-form). Migration file too.

7653 unit tests green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* v0.36.5.0 add: redact_secrets opt-in for stdout/stderr scrubbing

Honest defense for the documented output-side leakage. When a script prints
an inherited secret, the value lands plaintext in
result.stdout_tail / result.stderr_tail / error_text. v0.36.5.0 adds:

- `redact_secrets: true` ShellJobParams field
- `--redact-secrets` CLI convenience flag on `gbrain jobs submit shell`
- shell-redact.ts: pure `redactSecretsInText(text, secrets)` helper
  (string-mode replaceAll; regex metachars in values stay literal)
- Handler post-processes both tails before throw/return, so the persisted
  row carries `<REDACTED:name>` tokens instead of values

Only inherit-resolved values are scrubbed. env: values are not (those are
the agent's "fine in the row" channel by design). Heuristic — defeats
accidental `echo "$GBRAIN_DATABASE_URL"`, not adversarial encode-then-print.
Default false for back-compat.

Tests:
- test/minions-shell-redact.test.ts (9 cases): pure-function behavior,
  regex-metachar safety, multi-secret independent redaction, substring
  overlap, empty-input/map edge cases
- test/minions-shell-validate.test.ts: +4 cases for redact_secrets shape
- test/e2e/minions-shell-pglite.test.ts: +2 cases proving redact_secrets:
  true scrubs persisted row AND redact_secrets:false preserves plaintext
  (back-compat regression guard)

Docs + CHANGELOG + migration file + CLAUDE.md updated.

7667 unit tests green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-19 13:12:40 -07:00

248 lines
10 KiB
TypeScript

/**
* Tests for `src/core/minions/handlers/shell-validate.ts` — the pre-enqueue
* validator.
*
* v0.36.5.0 design: `inherit:` is free-form (any snake_case config-key name).
* No closed-enum check, no env-shadow rejection, no inline-cmd scan. The
* single-uid trust model treats the agent as a peer of the worker — it
* decides which secrets to pass.
*
* What the validator DOES check:
* - Shape (cmd XOR argv, cwd absolute, env is string→string)
* - `inherit` is array of snake_case strings (prototype-pollution defense)
* - Every `inherit` name resolves on `loadConfig()` (UX fail-fast)
*
* The T1 regression guard at the bottom pins the load-bearing invariant:
* validation throws BEFORE any persistence call.
*/
import { describe, test, expect } from 'bun:test';
import { validateShellJobParams } from '../src/core/minions/handlers/shell-validate.ts';
import { UnrecoverableError } from '../src/core/minions/types.ts';
import type { GBrainConfig } from '../src/core/config.ts';
const dbUrl = 'postgresql://test:test@localhost:5432/test';
const fakeCfg: GBrainConfig = {
engine: 'postgres',
database_url: dbUrl,
anthropic_api_key: 'sk-ant-test',
openai_api_key: 'sk-test',
};
describe('validateShellJobParams — existing param shape checks', () => {
test('cmd XOR argv: both → reject', () => {
expect(() => validateShellJobParams({ cmd: 'echo', argv: ['echo'], cwd: '/tmp' }, { config: fakeCfg }))
.toThrow(UnrecoverableError);
});
test('cmd XOR argv: neither → reject', () => {
expect(() => validateShellJobParams({ cwd: '/tmp' }, { config: fakeCfg }))
.toThrow(UnrecoverableError);
});
test('cwd must be absolute', () => {
expect(() => validateShellJobParams({ cmd: 'echo', cwd: 'relative/path' }, { config: fakeCfg }))
.toThrow(/absolute path/);
});
test('cwd required', () => {
expect(() => validateShellJobParams({ cmd: 'echo', cwd: '' }, { config: fakeCfg }))
.toThrow(/cwd/);
});
test('env must be object of string values', () => {
expect(() => validateShellJobParams({ cmd: 'echo', cwd: '/tmp', env: 'oops' as unknown as Record<string, string> }, { config: fakeCfg }))
.toThrow(/env/);
expect(() => validateShellJobParams({ cmd: 'echo', cwd: '/tmp', env: { K: 1 as unknown as string } }, { config: fakeCfg }))
.toThrow(/string/);
});
test('happy path: cmd + cwd accepted', () => {
const p = validateShellJobParams({ cmd: 'echo hi', cwd: '/tmp' }, { config: fakeCfg });
expect(p.cmd).toBe('echo hi');
expect(p.argv).toBeUndefined();
expect(p.cwd).toBe('/tmp');
});
});
describe('inherit — free-form config-key names (v0.36.5.0)', () => {
test('inherit:["database_url"] accepted', () => {
const p = validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: ['database_url'] },
{ config: fakeCfg },
);
expect(p.inherit).toEqual(['database_url']);
});
test('inherit:["anthropic_api_key"] accepted (was scope-creep in closed enum)', () => {
const p = validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: ['anthropic_api_key'] },
{ config: fakeCfg },
);
expect(p.inherit).toEqual(['anthropic_api_key']);
});
test('inherit multiple keys at once', () => {
const p = validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: ['database_url', 'anthropic_api_key', 'openai_api_key'] },
{ config: fakeCfg },
);
expect(p.inherit).toEqual(['database_url', 'anthropic_api_key', 'openai_api_key']);
});
test('inherit must be an array', () => {
expect(() => validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: 'database_url' as unknown as string[] },
{ config: fakeCfg },
)).toThrow(/inherit must be an array/);
});
test('inherit non-string element rejected', () => {
expect(() => validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: [1] as unknown as string[] },
{ config: fakeCfg },
)).toThrow(/non-empty strings/);
});
test('inherit empty string rejected', () => {
expect(() => validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: [''] },
{ config: fakeCfg },
)).toThrow(/non-empty strings/);
});
});
describe('inherit — snake_case shape guard (prototype-pollution defense)', () => {
test('"__proto__" rejected (not snake_case shape)', () => {
expect(() => validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: ['__proto__'] },
{ config: fakeCfg },
)).toThrow(/must match \[a-z\]/);
});
test('"constructor" rejected (uppercase letters)', () => {
expect(() => validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: ['constructor'] as unknown as string[] },
{ config: fakeCfg },
)).toThrow(/worker has no constructor configured/);
});
test('path-traversal-looking name rejected', () => {
expect(() => validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: ['../etc/passwd'] },
{ config: fakeCfg },
)).toThrow(/must match \[a-z\]/);
});
test('"FOO_BAR" (uppercase) rejected — config keys are snake_case', () => {
expect(() => validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: ['FOO_BAR'] },
{ config: fakeCfg },
)).toThrow(/must match \[a-z\]/);
});
test('digits-after-letter allowed (matches regex)', () => {
// Won't actually resolve since fakeCfg doesn't have field2 — fail-fast hits
expect(() => validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: ['field2'] },
{ config: fakeCfg },
)).toThrow(/worker has no field2 configured/);
});
});
describe('inherit — fail-fast on missing config value', () => {
test('inherit:["database_url"] + config without database_url → reject with set-hint', () => {
expect(() => validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: ['database_url'] },
{ config: { engine: 'postgres' } as GBrainConfig },
)).toThrow(/gbrain config set database_url/);
});
test('inherit:["database_url"] + null config → reject', () => {
expect(() => validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: ['database_url'] },
{ config: null },
)).toThrow(/worker has no database_url/);
});
test('inherit:["voyage_api_key"] when not set → reject (fakeCfg has only db_url + anthropic + openai)', () => {
expect(() => validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: ['voyage_api_key'] },
{ config: fakeCfg },
)).toThrow(/worker has no voyage_api_key configured/);
});
test('inherit:["database_url"] + empty-string database_url → reject', () => {
expect(() => validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: ['database_url'] },
{ config: { engine: 'postgres', database_url: '' } },
)).toThrow(/worker has no database_url/);
});
});
describe('what the validator deliberately does NOT do (agency for the agent)', () => {
test('caller can use env: for ANY key, including ones with secret-looking names', () => {
// No shadow rejection. Agent decides if they want to put a URL in env:
// directly (and accept that it lands in the row plaintext). v0.36.5.0
// honors the agent's call.
const p = validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', env: { GBRAIN_DATABASE_URL: dbUrl, DATABASE_URL: dbUrl } },
{ config: fakeCfg },
);
expect(p.env).toEqual({ GBRAIN_DATABASE_URL: dbUrl, DATABASE_URL: dbUrl });
});
test('caller can put inline secret-key=value in cmd', () => {
// No inline-cmd scan. The agent knows what it's writing.
const p = validateShellJobParams(
{ cmd: 'GBRAIN_DATABASE_URL=postgresql://... gbrain sync', cwd: '/tmp' },
{ config: fakeCfg },
);
expect(p.cmd).toContain('GBRAIN_DATABASE_URL');
});
test('inherit + env: with overlapping intent both work (last write wins per overlay order)', () => {
// Agent might want inherit for value-from-config AND env: for an
// additional non-secret. Both are honored.
const p = validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: ['database_url'], env: { MY_FLAG: '1' } },
{ config: fakeCfg },
);
expect(p.inherit).toEqual(['database_url']);
expect(p.env).toEqual({ MY_FLAG: '1' });
});
});
describe('redact_secrets shape check', () => {
test('redact_secrets: true accepted', () => {
const p = validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', inherit: ['database_url'], redact_secrets: true },
{ config: fakeCfg },
);
expect(p.redact_secrets).toBe(true);
});
test('redact_secrets: false accepted', () => {
const p = validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', redact_secrets: false },
{ config: fakeCfg },
);
expect(p.redact_secrets).toBe(false);
});
test('redact_secrets: undefined is fine (default)', () => {
const p = validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp' },
{ config: fakeCfg },
);
expect(p.redact_secrets).toBeUndefined();
});
test('redact_secrets: non-boolean rejected', () => {
expect(() => validateShellJobParams(
{ cmd: 'echo', cwd: '/tmp', redact_secrets: 'yes' as unknown as boolean },
{ config: fakeCfg },
)).toThrow(/redact_secrets must be a boolean/);
});
});
describe('T1 regression guard: validation runs BEFORE persistence', () => {
// This test pins the load-bearing invariant codex caught: validation must
// throw before any persistence call. If a future refactor moves the
// validation call back into the shell.ts handler, this test fails.
test('bad payload throws synchronously, no queue.add could have been called', () => {
let queueAddCalled = false;
const fakeQueueAdd = () => { queueAddCalled = true; };
// Bad shape: inherit name doesn't pass snake_case regex.
const data = { cmd: 'echo', cwd: '/tmp', inherit: ['NotSnake'] };
let validatorThrew = false;
try {
validateShellJobParams(data, { config: fakeCfg });
fakeQueueAdd();
} catch {
validatorThrew = true;
}
expect(validatorThrew).toBe(true);
expect(queueAddCalled).toBe(false);
});
});