Files
gbrain/docs/guides/agent-to-gbrain.md
T
e227965024 v0.36.5.0 feat: secure DATABASE_URL access for shell jobs (inherit: ["database_url"]) (#1192)
* v0.36.5.0 feat: secure DATABASE_URL access for shell jobs (inherit: ["database_url"])

Replaces PR #1137's plaintext-config / plaintext-env workarounds with code.
Shell-job params gain `inherit: ["database_url"]`, validated pre-enqueue in
both the CLI (`gbrain jobs submit`) and `submit_job` MCP op handler. Worker
resolves the value from its own loadConfig() at child-spawn time; the
persisted `minion_jobs.data` row stores only the name. Plain
`env: { GBRAIN_DATABASE_URL: ... }` / `env: { DATABASE_URL: ... }` /
`env: { GBRAIN_DIRECT_DATABASE_URL: ... }` are rejected pre-enqueue with a
paste-ready hint pointing at `inherit:`.

Codex pre-landing review caught two bypasses + one missing shadow name:
- H1: cmd/argv inline-secret regex scan (cmd:"GBRAIN_DATABASE_URL=... gbrain
  sync" was a clean bypass — fixed)
- H3: GBRAIN_DIRECT_DATABASE_URL added to shadowKeys
- H2: honest docs about output-side leakage (stdout_tail/stderr_tail can still
  carry the value if the script prints it; that's the script author's
  responsibility, not gbrain's)

Also: gbrain doctor learns home_dir_in_worktree (warns when ~/.gbrain lives
inside a git worktree); ~/.gbrain/.gitignore retroactive via saveConfig +
post-upgrade.

New canonical guide: docs/guides/agent-to-gbrain.md (two-domain framing for
downstream agent authors: MCP ops via OAuth vs localOnly admin ops via
shell-job inherit:).

Closes #1137. Tests: +53 new (21 validator + 12 inherit-record + 6
ensureGitignore + 5 doctor + 2 PGLite E2E + 7 codex-driven H1/H3 cases).

Credit: @wintermute filed PR #1137 which made the env-stripping gap visible
enough to fix in code. Thank you.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* v0.36.5.0 redesign: free-form inherit:, drop closed enum

User feedback: "agent spawning minions should have agency to do what it wants
with secrets and pass only the ones that it needs. don't be a security nazi
please."

Replaces the closed INHERITABLE enum (database_url only) with three small
helpers in shell-inherit.ts:

- INHERIT_NAME_RE: snake_case shape guard. Rejects __proto__, leading
  underscore, uppercase, path-traversal. Prototype-pollution defense.
- deriveEnvKey(name): config-key → child-env-key. Uppercase by default with
  one override: database_url → GBRAIN_DATABASE_URL.
- resolveInheritValue(cfg, name): value lookup with Object.hasOwn.

inherit: now accepts any snake_case config-key the worker has. Agent picks
what it needs per-job (database_url, anthropic_api_key, voyage_api_key, or
any custom field). Validator does NOT police WHICH keys — single-uid trust
model treats agent as peer of worker.

Drops the v0.36.5.0-RC rules that were paternalistic for the actual threat
model:
- closed-enum check
- env-shadow rejection
- cmd/argv inline-secret scan

Keeps the parts that defend real problems:
- pre-enqueue validation (closes the persistence-before-throw window)
- snake_case regex (prototype-pollution + audit-log readability)
- fail-fast on missing config value (UX guardrail, not security)

Tests: shell-validate (existing rules + new free-form + prototype-pollution
defense + T1 regression guard) and shell-inherit (regex matrix, deriveEnvKey
per-name, resolveInheritValue with hasOwn defense). E2E case now exercises
inherit:["anthropic_api_key"] to prove genuinely free-form.

Docs and CHANGELOG rewritten to reflect the open design + the design-arc
story (closed → cut → free-form). Migration file too.

7653 unit tests green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* v0.36.5.0 add: redact_secrets opt-in for stdout/stderr scrubbing

Honest defense for the documented output-side leakage. When a script prints
an inherited secret, the value lands plaintext in
result.stdout_tail / result.stderr_tail / error_text. v0.36.5.0 adds:

- `redact_secrets: true` ShellJobParams field
- `--redact-secrets` CLI convenience flag on `gbrain jobs submit shell`
- shell-redact.ts: pure `redactSecretsInText(text, secrets)` helper
  (string-mode replaceAll; regex metachars in values stay literal)
- Handler post-processes both tails before throw/return, so the persisted
  row carries `<REDACTED:name>` tokens instead of values

Only inherit-resolved values are scrubbed. env: values are not (those are
the agent's "fine in the row" channel by design). Heuristic — defeats
accidental `echo "$GBRAIN_DATABASE_URL"`, not adversarial encode-then-print.
Default false for back-compat.

Tests:
- test/minions-shell-redact.test.ts (9 cases): pure-function behavior,
  regex-metachar safety, multi-secret independent redaction, substring
  overlap, empty-input/map edge cases
- test/minions-shell-validate.test.ts: +4 cases for redact_secrets shape
- test/e2e/minions-shell-pglite.test.ts: +2 cases proving redact_secrets:
  true scrubs persisted row AND redact_secrets:false preserves plaintext
  (back-compat regression guard)

Docs + CHANGELOG + migration file + CLAUDE.md updated.

7667 unit tests green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-19 13:12:40 -07:00

8.7 KiB

How a downstream agent should talk to gbrain

This guide is for authors of downstream agents (hermes, openclaw, future forks) that need to call gbrain operations from their own runtime. Reading this first will save you a debugging cycle: gbrain has two distinct surfaces, and which one you pick depends on the operation.

The two surfaces

                       ┌─────────────────────────────────────────────┐
                       │                gbrain process                │
                       │                                              │
   Agent (hermes,      │  ┌──────────────────┐    ┌────────────────┐ │
   openclaw, fork) ────┼──▶  MCP ops surface  │    │   localOnly    │ │
                       │  │ (HTTP + OAuth)    │    │   admin ops    │ │
                       │  │                   │    │                │ │
                       │  │  search, query,   │    │  sync, embed,  │ │
                       │  │  put_page,        │    │  dream, doctor,│ │
                       │  │  get_page,        │    │  autopilot,    │ │
                       │  │  find_experts,    │    │  init, secrets │ │
                       │  │  ...              │    │                │ │
                       │  └──────────────────┘    └────────────────┘ │
                       │           ▲                       ▲          │
                       │           │                       │          │
                       │           │                       │          │
                       │     thin-client OAuth      shell-job `inherit:`│
                       │     (preferred for          (only path for   │
                       │      MCP-equivalent ops)    localOnly ops)   │
                       └─────────────────────────────────────────────┘

The two surfaces are not interchangeable. Pick by op, not by preference.

Surface 1 — MCP ops over HTTP (thin-client + OAuth)

Use for any operation that has an MCP equivalent: search, query, put_page, get_page, find_experts, find_orphans, find_anomalies, get_recent_salience, find_trajectory, and so on. The canonical list is the set of ops in src/core/operations.ts whose localOnly flag is unset (or false).

Setup

The host runs gbrain as a long-lived HTTP server:

GBRAIN_ALLOW_SHELL_JOBS=1 gbrain serve --http --port 3131

The agent registers as an OAuth client (one-time):

gbrain auth register-client hermes \
  --grant-types client_credentials \
  --scopes read,write
# Prints client_id + client_secret one-time. Store securely.

The agent's runtime calls /mcp with a bearer token from client_credentials grant. Secrets stay in the gbrain serve process; the agent never sees DATABASE_URL or API keys.

Thin-client mode (gbrain init --mcp-only) gives the agent the same client-credentials wiring, plus the gbrain CLI itself routes MCP-eligible commands through the configured remote MCP. The agent can call gbrain search / gbrain query directly and the CLI does the OAuth dance.

Why this is preferred for MCP ops

  • Secrets never leave the server process.
  • OAuth scopes give you read, write, admin separation — agent only gets what it needs.
  • Source-scoped tokens (--source dept-x on register-client) confine the agent to a specific source within a federated brain.
  • One audit surface (mcp_request_log) covers every op call uniformly.

Surface 2 — localOnly admin ops via shell-job inherit:

Some operations are flagged localOnly: true in src/core/operations.ts and are refused in thin-client mode at src/cli.ts:isThinClient. The full list (as of v0.36.5.0) includes:

  • sync (filesystem walks need local FS access)
  • embed (orchestrates the embed pipeline)
  • extract (walks markdown files)
  • dream (synthesis cycle)
  • doctor (filesystem hygiene checks)
  • autopilot (background daemon orchestration)
  • init (creates ~/.gbrain/)
  • secrets (config management)

For these, the agent cannot route through HTTP MCP. The only path is to run gbrain as a CLI subprocess. The recommended pattern is to submit the subprocess as a shell job to the gbrain Minions worker so retry / backoff / DLQ / audit trail all come for free.

Setup

gbrain jobs submit shell --params '{
  "cmd": "gbrain sync --skip-failed && gbrain embed --stale",
  "cwd": "/data/gbrain",
  "inherit": ["database_url"]
}'

The inherit: ["database_url"] field tells the worker to look up database_url from its loadConfig() and inject the value into the child env as GBRAIN_DATABASE_URL. The DB row in minion_jobs.data carries the names only — inherit: ["database_url"] — never the value. See minions-shell-jobs.md#secrets for the full validation rules and error catalog.

Why this is preferred over writing secrets into env: per-job

  • Pre-v0.36.5.0 callers passed env: { GBRAIN_DATABASE_URL: "postgresql://..." } per job. The URL landed plaintext in minion_jobs.data and the shell-audit JSONL. Anyone with brain-DB read access (or a brain dump, or a shared brain via mounts) saw the URL. As of v0.36.5.0, this is rejected at pre-enqueue validation. The error message names inherit: ["database_url"] as the replacement.

Worker setup (one-time, per host)

The agent's host needs a worker that processes shell jobs:

# One-shot inline execution (PGLite or Postgres):
gbrain jobs submit shell --params '{...}' --follow

# Persistent worker (Postgres only — PGLite uses --follow inline):
GBRAIN_ALLOW_SHELL_JOBS=1 gbrain jobs work

GBRAIN_ALLOW_SHELL_JOBS=1 is the worker-side opt-in. Without it, shell jobs sit in waiting indefinitely. Set it on the worker process env (or in your deploy unit / launchd plist), not per-submission — submitter env is a weak proxy for worker env.

Decision table

Operation Surface Why
search / query HTTP MCP via thin-client Has MCP op; OAuth-scoped.
get_page / list_pages HTTP MCP Same.
put_page HTTP MCP Same; respects subagent allow-list when applicable.
find_experts / find_orphans HTTP MCP Same.
sync / embed / extract Shell job + inherit: localOnly: true.
dream Shell job + inherit: localOnly: true.
doctor Shell job + inherit: (or no inherit if no DB) localOnly: true.
autopilot Run as a daemon directly on the host Long-lived, not job-shaped.
init / secrets One-time host setup Operator action, not agent action.
  • Prefer inherit: for secrets you don't want in the row. Names land in minion_jobs.data; values resolve at child-spawn from the worker's config. If a brain DB ever traverses a trust boundary, secrets stay out.
  • Free-form names. inherit: accepts any snake_case config-key on your worker — database_url, anthropic_api_key, openai_api_key, voyage_api_key, groq_api_key, zeroentropy_api_key, or any custom field you stuff into ~/.gbrain/config.json. The agent picks what it needs.
  • env: still works for non-secret values, or for cases where you WANT the value in the row (e.g. an opaque correlation token your audit flow needs to read back later). The validator doesn't second-guess you.
  • Never try to route a localOnly op through thin-client MCP. It will fail with localOnly op refused in thin-client mode. Use shell-job + inherit: (for secrets) or env: (for non-secrets).

Migration: from pre-v0.36.5.0

If your agent submits shell jobs that pass secrets via env::

// Pre-v0.36.5.0: works but URL persists in minion_jobs.data plaintext.
{
  "cmd": "gbrain sync --skip-failed",
  "cwd": "/data/gbrain",
  "env": { "GBRAIN_DATABASE_URL": "postgresql://..." }
}

Switch to (recommended):

// v0.36.5.0+: name in row, value resolved at child-spawn from worker config.
{
  "cmd": "gbrain sync --skip-failed",
  "cwd": "/data/gbrain",
  "inherit": ["database_url"]
}

Make sure the worker host has database_url configured (either via gbrain config set database_url <value> or via GBRAIN_DATABASE_URL / DATABASE_URL env on the worker process). If the worker can't resolve the key, the validator rejects the job at submit time with a paste-ready hint.