* v0.36.5.0 feat: secure DATABASE_URL access for shell jobs (inherit: ["database_url"]) Replaces PR #1137's plaintext-config / plaintext-env workarounds with code. Shell-job params gain `inherit: ["database_url"]`, validated pre-enqueue in both the CLI (`gbrain jobs submit`) and `submit_job` MCP op handler. Worker resolves the value from its own loadConfig() at child-spawn time; the persisted `minion_jobs.data` row stores only the name. Plain `env: { GBRAIN_DATABASE_URL: ... }` / `env: { DATABASE_URL: ... }` / `env: { GBRAIN_DIRECT_DATABASE_URL: ... }` are rejected pre-enqueue with a paste-ready hint pointing at `inherit:`. Codex pre-landing review caught two bypasses + one missing shadow name: - H1: cmd/argv inline-secret regex scan (cmd:"GBRAIN_DATABASE_URL=... gbrain sync" was a clean bypass — fixed) - H3: GBRAIN_DIRECT_DATABASE_URL added to shadowKeys - H2: honest docs about output-side leakage (stdout_tail/stderr_tail can still carry the value if the script prints it; that's the script author's responsibility, not gbrain's) Also: gbrain doctor learns home_dir_in_worktree (warns when ~/.gbrain lives inside a git worktree); ~/.gbrain/.gitignore retroactive via saveConfig + post-upgrade. New canonical guide: docs/guides/agent-to-gbrain.md (two-domain framing for downstream agent authors: MCP ops via OAuth vs localOnly admin ops via shell-job inherit:). Closes #1137. Tests: +53 new (21 validator + 12 inherit-record + 6 ensureGitignore + 5 doctor + 2 PGLite E2E + 7 codex-driven H1/H3 cases). Credit: @wintermute filed PR #1137 which made the env-stripping gap visible enough to fix in code. Thank you. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * v0.36.5.0 redesign: free-form inherit:, drop closed enum User feedback: "agent spawning minions should have agency to do what it wants with secrets and pass only the ones that it needs. don't be a security nazi please." Replaces the closed INHERITABLE enum (database_url only) with three small helpers in shell-inherit.ts: - INHERIT_NAME_RE: snake_case shape guard. Rejects __proto__, leading underscore, uppercase, path-traversal. Prototype-pollution defense. - deriveEnvKey(name): config-key → child-env-key. Uppercase by default with one override: database_url → GBRAIN_DATABASE_URL. - resolveInheritValue(cfg, name): value lookup with Object.hasOwn. inherit: now accepts any snake_case config-key the worker has. Agent picks what it needs per-job (database_url, anthropic_api_key, voyage_api_key, or any custom field). Validator does NOT police WHICH keys — single-uid trust model treats agent as peer of worker. Drops the v0.36.5.0-RC rules that were paternalistic for the actual threat model: - closed-enum check - env-shadow rejection - cmd/argv inline-secret scan Keeps the parts that defend real problems: - pre-enqueue validation (closes the persistence-before-throw window) - snake_case regex (prototype-pollution + audit-log readability) - fail-fast on missing config value (UX guardrail, not security) Tests: shell-validate (existing rules + new free-form + prototype-pollution defense + T1 regression guard) and shell-inherit (regex matrix, deriveEnvKey per-name, resolveInheritValue with hasOwn defense). E2E case now exercises inherit:["anthropic_api_key"] to prove genuinely free-form. Docs and CHANGELOG rewritten to reflect the open design + the design-arc story (closed → cut → free-form). Migration file too. 7653 unit tests green. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * v0.36.5.0 add: redact_secrets opt-in for stdout/stderr scrubbing Honest defense for the documented output-side leakage. When a script prints an inherited secret, the value lands plaintext in result.stdout_tail / result.stderr_tail / error_text. v0.36.5.0 adds: - `redact_secrets: true` ShellJobParams field - `--redact-secrets` CLI convenience flag on `gbrain jobs submit shell` - shell-redact.ts: pure `redactSecretsInText(text, secrets)` helper (string-mode replaceAll; regex metachars in values stay literal) - Handler post-processes both tails before throw/return, so the persisted row carries `<REDACTED:name>` tokens instead of values Only inherit-resolved values are scrubbed. env: values are not (those are the agent's "fine in the row" channel by design). Heuristic — defeats accidental `echo "$GBRAIN_DATABASE_URL"`, not adversarial encode-then-print. Default false for back-compat. Tests: - test/minions-shell-redact.test.ts (9 cases): pure-function behavior, regex-metachar safety, multi-secret independent redaction, substring overlap, empty-input/map edge cases - test/minions-shell-validate.test.ts: +4 cases for redact_secrets shape - test/e2e/minions-shell-pglite.test.ts: +2 cases proving redact_secrets: true scrubs persisted row AND redact_secrets:false preserves plaintext (back-compat regression guard) Docs + CHANGELOG + migration file + CLAUDE.md updated. 7667 unit tests green. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
8.7 KiB
How a downstream agent should talk to gbrain
This guide is for authors of downstream agents (hermes, openclaw, future forks) that need to call gbrain operations from their own runtime. Reading this first will save you a debugging cycle: gbrain has two distinct surfaces, and which one you pick depends on the operation.
The two surfaces
┌─────────────────────────────────────────────┐
│ gbrain process │
│ │
Agent (hermes, │ ┌──────────────────┐ ┌────────────────┐ │
openclaw, fork) ────┼──▶ MCP ops surface │ │ localOnly │ │
│ │ (HTTP + OAuth) │ │ admin ops │ │
│ │ │ │ │ │
│ │ search, query, │ │ sync, embed, │ │
│ │ put_page, │ │ dream, doctor,│ │
│ │ get_page, │ │ autopilot, │ │
│ │ find_experts, │ │ init, secrets │ │
│ │ ... │ │ │ │
│ └──────────────────┘ └────────────────┘ │
│ ▲ ▲ │
│ │ │ │
│ │ │ │
│ thin-client OAuth shell-job `inherit:`│
│ (preferred for (only path for │
│ MCP-equivalent ops) localOnly ops) │
└─────────────────────────────────────────────┘
The two surfaces are not interchangeable. Pick by op, not by preference.
Surface 1 — MCP ops over HTTP (thin-client + OAuth)
Use for any operation that has an MCP equivalent: search, query,
put_page, get_page, find_experts, find_orphans, find_anomalies,
get_recent_salience, find_trajectory, and so on. The canonical list is
the set of ops in src/core/operations.ts whose localOnly flag is unset
(or false).
Setup
The host runs gbrain as a long-lived HTTP server:
GBRAIN_ALLOW_SHELL_JOBS=1 gbrain serve --http --port 3131
The agent registers as an OAuth client (one-time):
gbrain auth register-client hermes \
--grant-types client_credentials \
--scopes read,write
# Prints client_id + client_secret one-time. Store securely.
The agent's runtime calls /mcp with a bearer token from client_credentials
grant. Secrets stay in the gbrain serve process; the agent never sees
DATABASE_URL or API keys.
Thin-client mode (gbrain init --mcp-only) gives the agent the same
client-credentials wiring, plus the gbrain CLI itself routes MCP-eligible
commands through the configured remote MCP. The agent can call
gbrain search / gbrain query directly and the CLI does the OAuth dance.
Why this is preferred for MCP ops
- Secrets never leave the server process.
- OAuth scopes give you
read,write,adminseparation — agent only gets what it needs. - Source-scoped tokens (
--source dept-xonregister-client) confine the agent to a specific source within a federated brain. - One audit surface (
mcp_request_log) covers every op call uniformly.
Surface 2 — localOnly admin ops via shell-job inherit:
Some operations are flagged localOnly: true in src/core/operations.ts and
are refused in thin-client mode at src/cli.ts:isThinClient. The full
list (as of v0.36.5.0) includes:
sync(filesystem walks need local FS access)embed(orchestrates the embed pipeline)extract(walks markdown files)dream(synthesis cycle)doctor(filesystem hygiene checks)autopilot(background daemon orchestration)init(creates~/.gbrain/)secrets(config management)
For these, the agent cannot route through HTTP MCP. The only path is to run
gbrain as a CLI subprocess. The recommended pattern is to submit the
subprocess as a shell job to the gbrain Minions worker so retry / backoff /
DLQ / audit trail all come for free.
Setup
gbrain jobs submit shell --params '{
"cmd": "gbrain sync --skip-failed && gbrain embed --stale",
"cwd": "/data/gbrain",
"inherit": ["database_url"]
}'
The inherit: ["database_url"] field tells the worker to look up
database_url from its loadConfig() and inject the value into the child
env as GBRAIN_DATABASE_URL. The DB row in minion_jobs.data carries the
names only — inherit: ["database_url"] — never the value. See
minions-shell-jobs.md#secrets for the
full validation rules and error catalog.
Why this is preferred over writing secrets into env: per-job
- Pre-v0.36.5.0 callers passed
env: { GBRAIN_DATABASE_URL: "postgresql://..." }per job. The URL landed plaintext inminion_jobs.dataand the shell-audit JSONL. Anyone with brain-DB read access (or a brain dump, or a shared brain via mounts) saw the URL. As of v0.36.5.0, this is rejected at pre-enqueue validation. The error message namesinherit: ["database_url"]as the replacement.
Worker setup (one-time, per host)
The agent's host needs a worker that processes shell jobs:
# One-shot inline execution (PGLite or Postgres):
gbrain jobs submit shell --params '{...}' --follow
# Persistent worker (Postgres only — PGLite uses --follow inline):
GBRAIN_ALLOW_SHELL_JOBS=1 gbrain jobs work
GBRAIN_ALLOW_SHELL_JOBS=1 is the worker-side opt-in. Without it, shell jobs
sit in waiting indefinitely. Set it on the worker process env (or in your
deploy unit / launchd plist), not per-submission — submitter env is a weak
proxy for worker env.
Decision table
| Operation | Surface | Why |
|---|---|---|
search / query |
HTTP MCP via thin-client | Has MCP op; OAuth-scoped. |
get_page / list_pages |
HTTP MCP | Same. |
put_page |
HTTP MCP | Same; respects subagent allow-list when applicable. |
find_experts / find_orphans |
HTTP MCP | Same. |
sync / embed / extract |
Shell job + inherit: |
localOnly: true. |
dream |
Shell job + inherit: |
localOnly: true. |
doctor |
Shell job + inherit: (or no inherit if no DB) |
localOnly: true. |
autopilot |
Run as a daemon directly on the host | Long-lived, not job-shaped. |
init / secrets |
One-time host setup | Operator action, not agent action. |
Recommended patterns
- Prefer
inherit:for secrets you don't want in the row. Names land inminion_jobs.data; values resolve at child-spawn from the worker's config. If a brain DB ever traverses a trust boundary, secrets stay out. - Free-form names.
inherit:accepts any snake_case config-key on your worker —database_url,anthropic_api_key,openai_api_key,voyage_api_key,groq_api_key,zeroentropy_api_key, or any custom field you stuff into~/.gbrain/config.json. The agent picks what it needs. env:still works for non-secret values, or for cases where you WANT the value in the row (e.g. an opaque correlation token your audit flow needs to read back later). The validator doesn't second-guess you.- Never try to route a
localOnlyop through thin-client MCP. It will fail withlocalOnly op refused in thin-client mode. Use shell-job +inherit:(for secrets) orenv:(for non-secrets).
Migration: from pre-v0.36.5.0
If your agent submits shell jobs that pass secrets via env::
// Pre-v0.36.5.0: works but URL persists in minion_jobs.data plaintext.
{
"cmd": "gbrain sync --skip-failed",
"cwd": "/data/gbrain",
"env": { "GBRAIN_DATABASE_URL": "postgresql://..." }
}
Switch to (recommended):
// v0.36.5.0+: name in row, value resolved at child-spawn from worker config.
{
"cmd": "gbrain sync --skip-failed",
"cwd": "/data/gbrain",
"inherit": ["database_url"]
}
Make sure the worker host has database_url configured (either via
gbrain config set database_url <value> or via GBRAIN_DATABASE_URL /
DATABASE_URL env on the worker process). If the worker can't resolve the
key, the validator rejects the job at submit time with a paste-ready hint.