Files
gbrain/docs/designs/KNOWLEDGE_RUNTIME.md
T
0e9f8814a5 feat: v0.16.0 — durable agent runtime (gbrain agent + subagent handler + plugin loader) (#258)
* refactor(mcp): extract buildToolDefs helper for subagent tool registry reuse

The inline operations.map(...) block in src/mcp/server.ts became the only
source of truth for agent-facing tool definitions. Extract into a reusable
exported helper so the v0.15 subagent tool registry can call it with a
filtered OPERATIONS subset instead of duplicating the shape.

Byte-for-byte equivalence regression pinned in test/mcp-tool-defs.test.ts —
legacy inline mapping kept verbatim inside the test so any future drift
between the new helper and the pre-extraction MCP schema fails loudly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(operations): subagent-aware OperationContext + put_page namespace

Adds three optional fields to OperationContext:
  - jobId?: number       — the currently running Minion job id
  - subagentId?: number  — the owning subagent job id for tool-dispatched calls
  - viaSubagent?: boolean — FAIL-CLOSED flag for agent-path gating

put_page now enforces a namespace rule when invoked on the subagent tool
dispatch path (viaSubagent=true): writes MUST target
`wiki/agents/<subagentId>/...`. Anchored, slash-boundary enforced so a
collision like `wiki/agents/12evil/...` can't impersonate subagent 12.

The check runs BEFORE the dry-run short-circuit so preview calls surface
the same rejection. Fail-closed: a missing subagentId with viaSubagent=true
rejects every slug rather than letting a dispatcher bug open a hole.

Existing callers unaffected — all three fields are optional and the legacy
put_page behavior is unchanged when viaSubagent is undefined/false.

12 regression + namespace tests pin:
  - local CLI writes (viaSubagent unset) accept arbitrary slugs
  - MCP writes (remote=true, viaSubagent unset) accept arbitrary slugs
  - subagent-path: anchored prefix accepted, wrong id rejected, prefix-
    collision defeated, leading-slash rejected, bare-prefix rejected,
    fail-closed on missing/NaN subagentId, permission_denied code emitted

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(schema): v0.15.0 subagent runtime tables + migration orchestrator

Adds three new tables for the durable LLM agent runtime:

  subagent_messages         — Anthropic message-block persistence.
                              Parallel tool_use blocks in one assistant
                              message live in content_blocks JSONB, not
                              across rows (fixes the (job_id, turn_idx, role)
                              misdesign codex caught in v0.13 drafting).

  subagent_tool_executions  — Two-phase tool ledger. INSERT pending before
                              execute, UPDATE complete/failed after. Replay
                              re-runs pending rows only if the tool is
                              idempotent (v1 ships only idempotent tools so
                              this is preventive).

  subagent_rate_leases      — Lease-based concurrency cap for outbound
                              providers (e.g. anthropic:messages). Stale
                              leases auto-prune on next acquire so crashed
                              workers can't strand capacity.

All DDL uses CREATE TABLE/INDEX IF NOT EXISTS — order-independent vs
PR #244's initSchema() reorder, and idempotent across fresh-install +
upgrade paths. Shipped in both src/schema.sql (Postgres) and
src/core/pglite-schema.ts (PGLite); schema-embedded.ts regenerated.

Migration orchestrator v0_15_0.ts (phases: schema → verify → record).
v0_14_0.ts is a no-op stub so the registry's version sequence stays
gapless (v0.14.0 shipped shell-jobs — code change, no DB migration).

10 unit tests for registry wiring, ordering, dry-run phase behavior, and
schema-embedded table presence. test/apply-migrations.test.ts updated for
the two new registry entries.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(minions): emit child_done on every terminal + max_stalled per-job + terminal set fix

Three correctness fixes the v0.15 subagent aggregator spine depends on:

1. child_done emission on ALL terminal transitions, not just success.
   - completeJob already emitted on success — now also tags outcome='complete'.
   - failJob newly emits on terminal 'failed' or 'dead' (outcome='failed'|'dead',
     error=<text>), BEFORE the parent-terminal UPDATE so the EXISTS guard on
     the inbox INSERT doesn't skip it on fail_parent paths (codex catch).
   - cancelJob now emits outcome='cancelled' per descendant with a parent.
   - handleTimeouts now emits outcome='timeout' per timed-out child.
   ChildDoneMessage gains optional { outcome, error } — backwards compatible
   (legacy writers omitted them; consumers treat absent outcome as 'complete').

2. Parent-resolution terminal set now includes 'failed'.
   Pre-v0.15 the `NOT EXISTS (... status NOT IN ('completed','dead','cancelled'))`
   guard treated a failed child as still-pending, stranding aggregator parents
   that chose on_child_fail='continue' or 'ignore' in waiting-children forever.
   Expanded to {completed, failed, dead, cancelled} everywhere parent resolution
   reads child status (completeJob inline, failJob remove_dep + continue,
   cancelJob sweep, handleTimeouts sweep, and the resolveParent method itself).

3. MinionJobInput.max_stalled threads through MinionQueue.add() on INSERT.
   Column exists with default 1 — that is "first stall → dead", which defeats
   crash recovery for long-running handlers. Subagent children will set
   max_stalled: 3 to survive mid-run worker kills. Second-submitter under an
   idempotency-key hit does NOT mutate the existing row (codex-flagged
   footgun — first-submit options are load-bearing state).

13 unit tests pin: emission on each of completeJob/failJob/cancelJob/
handleTimeouts, insertion order on fail_parent, terminal-set expansion with
continue policy, max_stalled default + override + idempotency behavior.

E2E tier 1 (Postgres) passes 141 tests unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(minions): rate-leases + waitForCompletion infra for v0.15 subagent

Two infrastructure modules the subagent handler spine depends on:

rate-leases.ts — lease-based concurrency cap for outbound providers
(anthropic:messages, openai:*, etc.). Counter-based limiters leak capacity
on worker crash; leases are owner-tagged rows with expires_at that
auto-prune on the next acquire. Two-phase: txn-scoped pg_advisory_xact_lock
guards the check-then-insert so concurrent acquires can't both win the
"last slot". renewLeaseWithBackoff retries 3x (250/500/1000ms) for mid-
call DB blips — on persistent failure the LLM-loop caller aborts with a
renewable error so the worker re-claims and the rate invariant is
preserved. Owner FK cascades clean up leases on job deletion.

wait-for-completion.ts — poll-until-terminal helper for CLI callers.
Minions' NOTIFY is worker-side only; `gbrain agent run --follow` polls
getJob() until status is {completed, failed, dead, cancelled}. TimeoutError
carries jobId + elapsedMs and does NOT cancel the job — the user can
inspect via `gbrain jobs get <id>` later. Supports AbortSignal for Ctrl-C
without throwing. Default pollMs is 1000 on Postgres, 250 on PGLite (inline
CLI has no network RTT).

21 unit tests cover: single/multi acquire under cap, rejection past cap,
release frees slot, different keys are independent, stale prune, cascade
on owner delete, renew bumps expires_at, renew on missing is false,
backoff path success + pruned short-circuit. waitForCompletion: fast-path
terminal, transitions mid-wait (completed/failed/cancelled), TimeoutError
shape, abort-signal early exit, non-existent job error.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(minions): subagent ToolDef types + brain-tool registry (v0.15)

Types first so the handler has a stable contract:
  - SubagentHandlerData / AggregatorHandlerData — the two job.data shapes
  - ToolCtx (engine, jobId, remote, signal) + ToolDef (name, description,
    input_schema, idempotent, execute) — Anthropic-envelope, distinct from
    the MCP McpToolDef extraction landed earlier
  - ContentBlock discriminated union for subagent_messages.content_blocks
  - SubagentStopReason + SubagentResult emitted on terminal completion

brain-allowlist.ts derives one ToolDef per allow-listed OPERATION. Reuses
the ParamDef → JSONSchema shape from the MCP extraction in a local helper
(Anthropic's input_schema field diverges from MCP's inputSchema by a
character). The 11-name allow-list is read-safe + put_page — every
destructive / filesystem / identity-mutating op stays off by default.

put_page gets a namespace-wrapped tool schema: `slug` pattern = anchored
`^wiki/agents/<subagentId>/.+`. The server-side check in put_page op
(shipped in prior commit) is still the authoritative gate — the schema
just helps the model write correct slugs first-try. `subagentId` is
plumbed into the ToolCtx so the viaSubagent=true fail-closed path lights
up on every tool-dispatched put_page.

filterAllowedTools narrows a registry by subagent_def's allowed_tools
frontmatter field. Rejects unknown names at load time (no silent drop —
typos in a skills/subagents/*.md would otherwise ship to prod with a
tool silently missing).

18 tests pin: every allowlist name exists in OPERATIONS (catches upstream
rename), Anthropic name regex, put_page namespace pattern per-subagent,
execute() routes through the op handler with viaSubagent=true, out-of-
namespace put_page throws permission_denied, filter passes prefixed +
unprefixed names, rejects unknowns, deduplicates.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(minions): subagent-audit JSONL + transcript renderer

Two small plumbing pieces the v0.15 subagent handler + `gbrain agent logs`
depend on:

subagent-audit.ts — JSONL-rotated audit log mirroring the shell-audit
pattern. Two event flavors: submission (one line per job submit) and
heartbeat (one line per turn boundary — llm_call_started / completed /
tool_called / tool_result / tool_failed). Heartbeats fix the "--follow on
a long Anthropic call shows nothing for 30 seconds" problem codex flagged.
Never logs prompts or tool inputs (PII risk — subagent input_vars may
carry user-supplied free text); DOES log tokens, ms_elapsed, tool_name,
first 200 chars of error text. Rotates weekly via ISO week. `readSubagent
AuditForJob` is the readback path for `gbrain agent logs` — scans the
current + prior week file so job boundaries across weeks still resolve.
`GBRAIN_AUDIT_DIR` overrides the default ~/.gbrain/audit/ for container
deploys.

transcript.ts — renders subagent_messages + subagent_tool_executions to
markdown. Message order is authoritative; tool rows splice under their
owning assistant tool_use by tool_use_id. Handles text, tool_use (with
pending / complete / failed execution rows), tool_result (skipped if
we already rendered the owning tool_use — avoids double-printing), and
unknown block types (fenced JSON dump for diagnostics). Output is
UTF-8-safe truncated at maxOutputBytes.

21 unit tests: ISO week filename rotation (incl. 2027-01-01 → W53-2026
boundary), submission + heartbeat write shapes, 200-char error cap, best-
effort write failure doesn't throw, readback filters by job_id and
sinceIso. Transcript: empty input, ordering, token line, tool_use +
complete/failed/pending execution rendering, truncation, unknown-block
diagnostic dump.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(minions): subagent LLM-loop handler with crash-resumable replay

The main event: runs one Anthropic Messages API conversation with tool
use, persists every turn + tool execution, and resumes cleanly after a
worker kill anywhere in the loop.

Design points that carry the v0.15 guarantees:

  1. Two-phase tool persistence. INSERT status='pending' before dispatch,
     UPDATE to 'complete' or 'failed' after. subagent_messages rows are
     the canonical conversation; subagent_tool_executions rows are the
     canonical "did this tool run + what did it return". Either DB commit
     is atomic, so replay has a single source of truth.

  2. Replay reconciliation. If the last persisted message is an assistant
     with tool_use blocks AND no following synthesized user message, we
     crashed mid-dispatch. On resume, finish those tools first (respecting
     idempotent flag for 'pending' rows), synthesize the user turn, and
     THEN call the LLM again. Non-idempotent pending rows abort the job
     with a clear error — v0.15 ships only idempotent tools so this is
     preventive.

  3. Rate lease around every LLM call. acquireLease before, releaseLease
     after (both success and error paths). acquired=false throws
     RateLeaseUnavailableError — the worker treats it as a renewable
     error and re-claims later, so a temporary capacity cap doesn't fail
     the job terminally.

  4. Anthropic prompt caching. system block gets cache_control=ephemeral;
     the LAST tool def gets it too (Anthropic caches everything up to and
     including the marked block). ~10x cost reduction on multi-turn
     agents per the plan.

  5. Dual-signal abort. AbortSignal.any merges ctx.signal (timeout / lock
     loss / cancel) with ctx.shutdownSignal (worker SIGTERM). Both feed
     the Anthropic call's AbortSignal; mid-turn abort bails before the
     next LLM call with whatever turns are already persisted. Node ≥ 20
     has AbortSignal.any; older runtimes get a manual-merge polyfill.

  6. Injectable Anthropic client. The real SDK implements MessagesClient
     structurally; tests inject a FakeMessagesClient that scripts
     responses.

12 unit tests pin: no-tool happy path, single tool_use complete, tool
throws → failed row + loop continues, unknown tool name rejection,
max_turns cap, crash-then-resume with partial state, replay skips already-
complete tool execs without re-invoking execute, non-idempotent pending
rejects on resume, lease acquire + release roundtrip, RateLeaseUnavailable
under cap-full, missing prompt validation, allowed_tools unknown-name.

NOT in v0.15: refusal detection (stop_reason + content shape), stop_reason
=max_tokens partial recovery, mid-call lease renewal with backoff loop.
All three are documented as P2 items in the plan file.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(minions): subagent_aggregator handler with mixed-outcome rendering

Claims AFTER all subagent children resolve — by then Lane 1B's queue
changes have posted one child_done message per terminal transition into
this job's inbox (complete / failed / dead / cancelled / timeout). The
aggregator reads those, builds a deterministic markdown summary, and
returns it as the handler result.

Not an LLM call in v0.15 — output is reproducible concatenation so
fan-out runs stay comparable. v0.16+ can add an LLM synthesis pass
behind an opt-in flag.

Contract:
  - empty children_ids → `(no children)` marker
  - missing child_done (shouldn't happen under v0.15 invariants but
    possible if a terminal-state path slipped past Lane 1B) → counted as
    failed with "no child_done message observed" error
  - non-complete outcomes: result is null in the output so no payload
    leaks alongside a failure label
  - children appear in the order children_ids was supplied
  - custom aggregate_prompt_template replaces the markdown header

13 unit tests cover: empty input, all-success, mixed outcomes, result
suppression on failure, missing child_done handling, order preservation,
custom template, progress + log emission, stringified JSONB payload
parsing, non-child_done inbox filtering, legacy-writer outcome fallback,
and internal helper edges.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(minions): GBRAIN_PLUGIN_PATH loader + plugin-authors guide (v0.15)

Plumbing that makes Wintermute (and future downstream agents) day-1
usable on v0.15. Host repos drop a `gbrain.plugin.json` + `subagents/`
directory somewhere, set GBRAIN_PLUGIN_PATH (colon-separated like \$PATH),
and their custom subagent defs load at worker startup.

Path policy is strict: absolute paths only. Relative, ~-prefixed, and
URL-style (https://, file://) all rejected with warnings — the user
controls where plugins live. Non-existent paths and files (not dirs) are
warned and skipped so a typo doesn't crash worker startup.

Collision policy: left-wins. If two plugins ship a subagent with the same
name, the first one in GBRAIN_PLUGIN_PATH keeps it and the other gets a
warning naming both sources. Deterministic + debuggable.

Trust policy: plugins ship subagent defs ONLY. Cannot declare new tools,
cannot extend the brain allow-list, cannot override safety flags. The
subagent def's `allowed_tools:` frontmatter MUST subset the derived
registry — validation happens at load time (worker startup), not at
dispatch time, so a typo in a skill gives a loud startup error instead
of silently "tool never fires at 3am."

Manifest `plugin_version: "gbrain-plugin-v1"` locks the contract. Unknown
versions rejected. `subagents` field escape attempts (`../../../etc` etc)
rejected. gray-matter handles the markdown frontmatter parse — subagent
defs don't conform to the page schema, so we don't use parseMarkdown.

docs/guides/plugin-authors.md is the Wintermute-facing walkthrough.
Covers the minimum viable plugin shape, the three policies, the
frontmatter fields, known caveats (audit JSONL is local-only, tool calls
always run remote=true, put_page is namespace-scoped).

22 unit tests pin path rejection, missing/invalid manifest, unsupported
version, escape-attempt, basename fallback for missing frontmatter.name,
allowed_tools round-trip, unknown-tool rejection with validAgentToolNames,
empty env, multi-path, collision warning with left-wins, trimmed paths,
manifest-rejection as warning.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(cli): gbrain agent run + logs + worker registration (v0.15 Lane 4H)

Three integration seams wired:

src/commands/agent.ts — \`gbrain agent run\`. Submits subagent jobs (or a
fan-out of N + aggregator) under the trusted-submit flag so the
PROTECTED_JOB_NAMES guard doesn't reject. Fan-out path creates the
aggregator first (so children can reference its id as parent), submits
each child with on_child_fail='continue' (required by Lane 1B's terminal-
set + child_done machinery), then jsonb_set's the aggregator's
children_ids. Short-circuits a 1-entry manifest to a single subagent
with no aggregator. Follow mode runs agent-logs streaming + waitFor
Completion in parallel and exits on terminal status; detach prints the
job id and exits. Ctrl-C is handled as detach, not cancel — the job
keeps running, consistent with durability invariants.

src/commands/agent-logs.ts — \`gbrain agent logs\`. Merges ~/.gbrain/audit/
subagent-jobs-*.jsonl (heartbeats + submissions) with subagent_messages
(persisted conversation) in one chronological stream. --follow polls at
1s and exits when the job hits terminal. --since accepts ISO-8601 OR
relative shorthand (5m / 1h / 2d). Writes transcript tail (full message
+ tool tree) only for terminal jobs, so mid-run --follow doesn't spam a
half-rendered transcript.

src/commands/jobs.ts registerBuiltinHandlers — matches the shell-handler
opt-in shape. GBRAIN_ALLOW_LLM_JOBS=1 registers the subagent +
subagent_aggregator handlers, then loads plugins from GBRAIN_PLUGIN_PATH
with validAgentToolNames pulled from BRAIN_TOOL_ALLOWLIST. Every plugin
warning + loaded-plugin line prints to stderr, mirroring the openclaw-
seam startup convention.

src/core/minions/protected-names.ts — subagent + subagent_aggregator
join the protected set. MCP submit_job returns permission_denied; only
trusted-CLI callers (with allowProtectedSubmit) can insert these rows.

src/cli.ts — adds 'agent' to CLI_ONLY + dispatches it like 'jobs'.

Test fallout: subagent-handler.test.ts + subagent-transcript.test.ts
helpers now submit under allowProtectedSubmit (they insert rows named
'subagent' directly against the queue). 23 new tests in agent-cli.test.ts
cover: flag parsing (including --detach implies !follow, --tools comma
split, -- terminator, unknown flag throw), --since parse (ISO, relative
5m/2h/1d, unparseable error), protected-name guard for all three names,
trusted-submit gate, and a fan-out integration check that verifies the
aggregator + children shape after --fanout-manifest.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(e2e): rename max_children test's spawned jobs off the protected 'subagent' name

The spawn-storm test submitted 50 literal-string 'subagent' children to
exercise the max_children row-lock serialization. In v0.15 'subagent' is
a PROTECTED_JOB_NAME (CLI-only; trusted submit required), so the old
literal submission now throws before reaching the row-lock check.

The test is about max_children semantics, not the v0.15 subagent runtime
specifically — rename the child name to 'child_worker' so the test
exercises the exact same queue.add path without tripping the new guard.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(ship): v0.15.0 — VERSION, CHANGELOG, README, upgrading-agents, CLAUDE.md

Bumps VERSION → 0.15.0 and package.json → 0.15.0 (resolves the pre-existing
drift — on master, VERSION=0.14.0 but package.json=0.13.1; src/version.ts
reads package.json, so this is what the binary prints now).

CHANGELOG lands the release-summary entry in the GStack voice + the full
itemized change list (11 new modules, 3 new tables, queue correctness
fixes, trust-model additions, 159 new unit tests). Voice rules respected
— no em dashes, no AI vocabulary, real file names + real numbers.

README gets a "Durable agents: `gbrain agent` (v0.15)" section next to
the Minions block, with the three canonical CLI shapes (single run,
fanout-manifest, logs --follow) and a pointer to plugin-authors.md.

docs/UPGRADING_DOWNSTREAM_AGENTS.md gets a full v0.15.0 section covering
the four adoption steps downstream agents (Wintermute and similar) need:
(1) worker opt-in via GBRAIN_ALLOW_LLM_JOBS, (2) moving custom subagent
defs to a plugin repo, (3) replacing ephemeral subagent runs with durable
`gbrain agent run`, (4) the put_page namespace rule for agent-driven writes.

CLAUDE.md updated with concise per-file descriptions for every new module:
the handler, aggregator, audit, rate-leases, wait-for-completion,
transcript, plugin-loader, brain-allowlist, tool-defs extraction, agent
CLI + logs CLI, and the registerBuiltinHandlers wiring for subagent
handlers + plugin-loader.

Verified: binary builds (940 modules, 89ms compile), prints `gbrain 0.15.0`,
`gbrain agent --help` shows the new subcommand shape. 170 new tests pass
(full v0.15 surface). Full unit suite passes bar one parallel-load
flake on a pre-existing E2E (graph-quality, passes in isolation).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(minions): drop GBRAIN_ALLOW_LLM_JOBS flag — subagent handlers always-on

The env flag was ceremony. Shell jobs need the flag because they execute
arbitrary CLI commands (RCE surface). Subagent jobs don't — they call the
Anthropic API with whatever ANTHROPIC_API_KEY is in env, so the key is
already the cost gate (no key → SDK fails on the first turn). And
who-can-submit is already protected by PROTECTED_JOB_NAMES +
TrustedSubmitOpts: MCP callers get permission_denied; only `gbrain agent
run` with allowProtectedSubmit can insert subagent / subagent_aggregator
rows. The flag added nothing the existing guards didn't already give us.

registerBuiltinHandlers now always registers subagent + subagent_aggregator
and loads GBRAIN_PLUGIN_PATH plugins. Worker startup prints:

  [minion worker] subagent handlers enabled

instead of the conditional enabled/disabled pair. Plugin discovery runs
unconditionally — empty PATH is a no-op.

README, CHANGELOG, docs/UPGRADING_DOWNSTREAM_AGENTS, CLAUDE.md, agent CLI
help text, and subagent handler docstring all updated to drop the flag
reference. Shell handler's GBRAIN_ALLOW_SHELL_JOBS gate is untouched —
separate concern (RCE, not billing).

Full suite: 1859 pass, 0 fail.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: scrub private agent-fork name from all public artifacts

Enforces the rule added to CLAUDE.md (privacy section): never say
`Wintermute` in any CHANGELOG, README, doc, PR, or commit message.
Reader-facing copy says `your OpenClaw` (the term covers every
downstream OpenClaw deployment — Wintermute, Hermes, AlphaClaw — in
one umbrella the reader already recognizes). First-person /
origin-story copy says `Garry's OpenClaw` (honest that this is the
production deployment driving the feature, without exposing the
private agent's name).

Swept across:
  CHANGELOG.md (v0.15 entry + 4 historical mentions)
  README.md
  TODOS.md
  docs/UPGRADING_DOWNSTREAM_AGENTS.md
  docs/guides/plugin-authors.md (including example plugin names)
  docs/guides/plugin-handlers.md
  docs/guides/minions-fix.md
  docs/designs/KNOWLEDGE_RUNTIME.md (27 refs, mostly analytical)
  docs/benchmarks/2026-04-18-minions-vs-openclaw-production.md
  skills/migrations/v0.11.0.md
  skills/skillpack-check/SKILL.md
  scripts/skillify-check.ts
  src/commands/doctor.ts
  src/commands/migrations/v0_15_0.ts
  src/commands/skillpack-check.ts
  src/core/enrichment/completeness.ts
  src/core/minions/plugin-loader.ts
  src/core/operations.ts
  src/core/output/scaffold.ts

Intentionally kept (these mentions define/test the rule itself):
  CLAUDE.md — the privacy rule section necessarily uses the literal
  name to define the restriction and examples
  test/plugin-loader.test.ts — fixture name in a plugin-loading test;
  renaming risks breaking assertion logic
  test/integrations.test.ts — the word appears in a privacy-regex
  test that explicitly enforces name redaction
  test/doctor-minions-check.test.ts — a comment referencing the rule
  CEO plan artifact at ~/.gstack/projects/… — private, not distributed

Binary builds (941 modules), 198/198 relevant tests pass, `gbrain --version`
prints `0.15.0`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: gitignore bun --compile artifacts with a glob, not specific hashes

Each `bun build --compile` emits a fresh hash-named `.*-*.bun-build` file
in cwd. The prior entries listed two specific hashes that were already
stale, so every build after those created a new untracked file requiring
manual cleanup.

Replace the two stale entries with `*.bun-build` so any current or future
compile artifact is ignored automatically.

Verified: ran `bun build --compile`, got two new `.*-*.bun-build` files,
`git status` stays clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(ship): rename v0.15.0 → v0.16.0

gbrain master is at 0.14.2. Other 0.15.x PRs may land before/after
this one — we bump the minor (new capability) and lock to 0.16.0 so
ordering with concurrent work doesn't matter.

Touches:
- VERSION: 0.15.0 → 0.16.0
- package.json: 0.15.0 → 0.16.0
- Rename src/commands/migrations/v0_15_0.ts → v0_16_0.ts (+ all
  version strings inside + import in index.ts registry)
- Rename test/migrations-v0_15_0.test.ts → migrations-v0_16_0.test.ts
- test/apply-migrations.test.ts: skippedFuture lists now reference
  '0.16.0'
- test/put-page-namespace.test.ts + test/mcp-tool-defs.test.ts: Lane
  comment refs updated
- src/schema.sql + src/core/pglite-schema.ts: "v0.15.0" section
  comment updated; src/core/schema-embedded.ts regenerated
- CHANGELOG.md: top entry renamed to [0.16.0]; inline v0_15_0 /
  v0.15.0 refs swept
- docs/UPGRADING_DOWNSTREAM_AGENTS.md: section heading v0.15.0 → v0.16.0

Verified: `gbrain --version` prints 0.16.0, migration registry /
buildPlan / put_page / mcp-tool-defs / handlers tests all green
(49/49).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: reframe v0.16 durability headline around OpenClaw crashes

"Laptop closed mid-run" framing implied a consumer workflow. Real pain is
OpenClaw subagents dying daily on worker kill, memory blip, or timeout.
Headline + README copy match the body now.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: regenerate llms-full.txt after README copy change

Regen drift guard caught the README edit from 83beec4.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 21:14:17 -07:00

35 KiB
Raw Blame History

GBrain Knowledge Runtime — Design Doc

Status: DRAFT for CEO review. Date: 2026-04-18. Supersedes: The earlier "Feynman Ideas Assessment + Phase A/B" plan.


0. Context

During a CEO review of a narrow two-feature plan (bare-tweet citation repair + completeness score, borrowed from Feynman), the scope was reframed. The narrow plan duplicated work Garry's OpenClaw already does and missed the real leverage point: the bespoke abstractions hiding inside OpenClaw — resolvers, enrichment orchestration, scheduling, deterministic output — should live in GBrain as first-class primitives.

North star: "When Garry's OpenClaw's Claw upgrades to this version of GBrain, it should immediately recognize brilliance and completeness and say 'It's time to switch to these abstractions.'"

That is the test this document is designed against. Everything else is downstream.


1. The Four Layers

The design is four layered abstractions. Each is independently useful; together they are the Knowledge Runtime.

  ┌───────────────────────────────────────────────────────────────────┐
  │                   KNOWLEDGE RUNTIME (new)                         │
  ├───────────────────────────────────────────────────────────────────┤
  │  Layer 4: Deterministic Output Builder                            │
  │     BrainWriter · Scaffolds · Back-link enforcer · Slug registry  │
  │     Rule: LLM picks WHAT to write. Code guarantees WHERE and HOW. │
  ├───────────────────────────────────────────────────────────────────┤
  │  Layer 3: Scheduler                                               │
  │     ScheduledResolver · TZ-aware quiet hours (enforced) ·         │
  │     Auto-stagger · Durable state · Retry/circuit-break            │
  ├───────────────────────────────────────────────────────────────────┤
  │  Layer 2: Enrichment Orchestrator                                 │
  │     Trigger convergence · Tier routing · Budget · Cascade ·       │
  │     Evidence-weighted completeness · Fail-safe transactions       │
  ├───────────────────────────────────────────────────────────────────┤
  │  Layer 1: Resolver SDK                                            │
  │     Resolver<I,O> interface · Registry · Factory · Plugin recipes │
  │     Ported reference impls: X-API, Perplexity, Mistral, brain     │
  └───────────────────────────────────────────────────────────────────┘
          │                                                │
          ▼                                                ▼
     REUSES (polished primitives already in GBrain)  REPLACES (ad-hoc code)
     FailImproveLoop · backoff · storage factory ·   enrichment-service ·
     check-resolvable · operations validators ·      embedding · transcription ·
     engine interface · publish · backlinks          2 recipe formats

2. Why This Order (L1 → L4)

Every higher layer depends on the lower one. L1 must land first or the rest leaks abstractions.

  • L1 (Resolvers) is the substrate. Without a uniform lookup interface, every orchestrator + writer has bespoke callers.
  • L2 (Orchestrator) uses L1 to fetch; without L1 it's still ad-hoc.
  • L3 (Scheduler) runs L2 periodically; without L2 it's scheduling nothing structured.
  • L4 (Output Builder) is what every layer ultimately writes through; without it we have 14 call sites doing fs.writeFile with hand-rolled citation discipline.

An earlier implementation could ship L1 + L4 first (the two "purest" layers) and have the most immediate integrity impact, then add L2 + L3. But the end-state must include all four.


3. Layer 1 — Resolver SDK

3.1 What's broken today

Garry's OpenClaw has 69 distinct external-lookup patterns across X API (14 shapes), Perplexity, Mistral OCR, Gmail, Calendar, Slack, GitHub, YouTube, Diarize.io, YC tools, OSINT collectors, and brain-local lookups. Each one is a bespoke script under scripts/ with its own error handling, retry logic, and output shape. GBrain has 3 ad-hoc wrappers (embedding.ts, transcription.ts, enrichment-service.ts) that don't share an interface.

Common consequences:

  • No uniform retry/backoff strategy (some scripts retry, most don't)
  • No cost tracking (Perplexity bills eaten silently when calls return no-substance results)
  • No confidence/provenance propagation (callers can't tell if an answer is verified or inferred)
  • Users can't add a resolver without forking GBrain

3.2 Interface

// src/core/resolvers/interface.ts

export type ResolverCost = 'free' | 'rate-limited' | 'paid';

export interface ResolverRequest<I> {
  input: I;
  context: ResolverContext;
  timeoutMs?: number;
}

export interface ResolverResult<O> {
  value: O;
  confidence: number;      // 0.01.0; 1.0 = deterministic from ground-truth API
  source: string;          // e.g. "x-api-v2", "perplexity-sonar", "brain-local"
  fetchedAt: Date;
  costEstimate?: number;   // dollars; 0 if free
  raw?: unknown;           // for sidecar preservation via put_raw_data
}

export interface Resolver<I, O> {
  readonly id: string;           // stable, slug-like: "x_handle_to_tweet"
  readonly cost: ResolverCost;
  readonly backend: string;      // "x-api-v2", "perplexity", "brain-local"
  readonly inputSchema: JSONSchema;
  readonly outputSchema: JSONSchema;

  available(ctx: ResolverContext): Promise<boolean>;
  resolve(req: ResolverRequest<I>): Promise<ResolverResult<O>>;
}

3.3 Context

export interface ResolverContext {
  engine: BrainEngine;
  storage: StorageBackend;
  config: GBrainConfig;
  logger: Logger;
  metrics: MetricsRecorder;
  budget: BudgetLedger;       // hard spend caps, queried pre-resolve
  requestId: string;
  remote: boolean;            // trust boundary — untrusted callers get stricter validation
  deadline?: Date;
}

3.4 Registry + Factory (mirrors src/core/storage.ts)

// src/core/resolvers/registry.ts
export class ResolverRegistry {
  register<I, O>(r: Resolver<I, O>): void;
  get(id: string): Resolver<unknown, unknown>;
  list(filter?: { cost?: ResolverCost; backend?: string }): Resolver[];
  async resolve<I, O>(id: string, input: I, ctx: ResolverContext): Promise<ResolverResult<O>>;
}

// src/core/resolvers/factory.ts (dynamic import like engine-factory)
export async function createResolver(
  type: 'x-api' | 'perplexity' | 'mistral-ocr' | 'brain-local' | 'plugin',
  config: ResolverConfig,
): Promise<Resolver>;

3.5 Plugin format (unifies recipes/ + data-research formats)

A plugin is YAML + JS module, discovered via filesystem scan of ~/.gbrain/resolvers/ and recipes/.

# Example: resolvers/x-api/handle-to-tweet.yaml
id: x_handle_to_tweet
version: 1
category: lookup
cost: rate-limited
backend: x-api-v2
module: ./handle-to-tweet.ts
input_schema:
  type: object
  properties:
    handle:   { type: string, pattern: "^[A-Za-z0-9_]{1,15}$" }
    keywords: { type: string }
  required: [handle]
output_schema:
  type: object
  properties:
    url:        { type: string, format: uri }
    tweet_id:   { type: string }
    text:       { type: string }
    created_at: { type: string, format: date-time }
requires:
  env: [X_API_BEARER_TOKEN]
health_check:
  kind: http
  url: https://api.twitter.com/2/tweets/1
  expect: { status: [200, 401] }   # 401 = auth failure but endpoint reachable
tests:
  - input:  { handle: "garrytan" }
    expect: { url: { pattern: "^https://x\\.com/garrytan/status/\\d+$" } }

Trust flagging follows the existing src/commands/integrations.ts pattern: only package-bundled resolvers are embedded=true and may run arbitrary commands; user-provided resolvers are restricted to http and validated schemas.

3.6 Wraps every resolver with FailImproveLoop

Existing src/core/fail-improve.ts is the deterministic-first/LLM-fallback pattern. Every resolver automatically gets wrapped: if the deterministic path (e.g. X API) returns a valid result, use it; if it fails, optionally fall back to an LLM-based resolver; log both paths for future pattern analysis and auto-test generation.

3.7 Reference implementations to ship

The OpenClaw survey inventoried 69 resolver shapes. Shipping all of them is wrong (over-scoped); shipping zero is under-scoped. The dogfood set:

# Resolver Purpose Used by
1 x_handle_to_tweet Bare-tweet citation repair (original Phase A) gbrain integrity
2 url_reachable Dead-link detection gbrain integrity
3 brain_slug_lookup Name/email → slug (wraps existing resolveSlugs) Output Builder
4 openai_embedding Refactor of src/core/embedding.ts into Resolver Import pipeline
5 perplexity_query Query → synthesis + citations Enrichment Orchestrator
6 text_to_entities LLM entity extraction (structured JSON) Enrichment Orchestrator

The remaining 63 OpenClaw patterns port incrementally, driven by user need. Each port is a new YAML + module under recipes/ or ~/.gbrain/resolvers/ with no framework changes.


4. Layer 2 — Enrichment Orchestrator

4.1 What's broken today

Garry's OpenClaw's enrichment is polished at the data layer, hacky at the control layer:

  • Completeness = "length > 500 chars + no needs-enrichment tag" (lib/enrich.mjs:351-355). Naïve. A rich page of repetitive Perplexity summaries (see brain/people/0interestrates.md — 38 repeating blocks) passes this check.
  • 30-day auto-re-enrichment runs forever. No "done" state. A person met once in 2023 still gets re-researched monthly.
  • Cascade is convention-only. Person→company stubs are created automatically; company→investors, company→employees traversals are documented but never implemented.
  • No hard budget cap. Cost is estimated per batch, never enforced across batches or per day.
  • Failure is silent. A bad Perplexity response logs and continues; partial writes can leave a page with a timeline entry but no raw-data sidecar.

4.2 The orchestrator

// src/core/enrichment/orchestrator.ts

export interface EnrichmentRequest {
  entitySlug: string;
  trigger: 'mention' | 'stub-creation' | 'cron-sweep' | 'manual' | 'cascade';
  tier?: 1 | 2 | 3;                // optional override; auto-computed if absent
  cascadeDepth?: number;           // 0 = no cascade; default 1
}

export interface EnrichmentResult {
  entitySlug: string;
  completenessBefore: number;
  completenessAfter: number;
  resolversUsed: string[];         // e.g. ["perplexity_query", "x_handle_to_tweet"]
  costSpent: number;
  writtenTo: string[];             // page paths touched, for transaction audit
  cascadedTo: string[];            // related entities enriched
  status: 'enriched' | 'skipped' | 'failed' | 'budget-exhausted';
  reason?: string;
}

export class EnrichmentOrchestrator {
  constructor(
    private registry: ResolverRegistry,
    private writer: BrainWriter,
    private budget: BudgetLedger,
    private scorer: CompletenessScorer,
    private graph: EntityGraph,
  ) {}

  async enrich(req: EnrichmentRequest): Promise<EnrichmentResult>;
  async enrichBatch(reqs: EnrichmentRequest[]): Promise<EnrichmentResult[]>;
}

4.3 Evidence-weighted completeness (replaces length heuristic)

Completeness is a per-entity-type rubric, stored in frontmatter on write and recomputed on demand.

// src/core/enrichment/completeness.ts
export interface CompletenessRubric<Page> {
  entityType: PageType;
  dimensions: {
    name: string;
    weight: number;                // sum must = 1.0
    check: (page: Page) => number; // 0.01.0
  }[];
}

// Example rubric for persons:
//   - has_role_and_company   0.20
//   - has_source_urls        0.20  (≥1 URL with resolver-verified reachability)
//   - has_timeline_entries   0.15  (≥1)
//   - has_citations          0.15  (every claim has [Source: ...])
//   - has_backlinks          0.10  (every linked page links back)
//   - recency_score          0.10  (last_verified within 90 days)
//   - non_redundancy         0.10  (no repeated blocks; distinct-lines/total-lines > 0.8)

Key property: non_redundancy + recency_score explicitly kill the two brain pathologies observed in the audit (Wilco-style repeating blocks; stale pages without last_verified).

The completeness field goes in frontmatter as 0.01.0. It becomes queryable via list_pages(where: completeness < 0.5).

4.4 Tier routing with hard budget

Two-dimensional routing: importance (tier 1/2/3 from person-score) × budget state.

// src/core/enrichment/tiers.ts
export const TIER_CONFIG = {
  1: { models: ['opus', 'sonar-deep'], maxCostUsd: 0.10, cascadeDepth: 2 },
  2: { models: ['sonar'],              maxCostUsd: 0.02, cascadeDepth: 1 },
  3: { models: ['sonar'],              maxCostUsd: 0.005, cascadeDepth: 0 },
};

// src/core/enrichment/budget.ts
export class BudgetLedger {
  // Hard caps. Queryable pre-resolve.
  dailyCapUsd: number;
  perEntityCapUsd: number;
  perResolverCapUsd: Map<string, number>;

  async reserve(resolverId: string, estimateUsd: number): Promise<Reservation | 'exhausted'>;
  async commit(reservation: Reservation, actualUsd: number): Promise<void>;
  async rollback(reservation: Reservation): Promise<void>;
  async state(): Promise<{ spent: number; remaining: number; perResolver: Record<string, number> }>;
}

Property: if the daily cap is reached, orchestrator.enrich() returns status: 'budget-exhausted' immediately. No silent overages. Circuit-breaker resets at midnight in the user's configured TZ.

4.5 Cascade (entity graph traversal)

// src/core/enrichment/cascade.ts
export class EntityGraph {
  // Deterministic, no LLM. Uses engine.getLinks() + engine.getBacklinks().
  async neighbors(slug: string, depth: number): Promise<string[]>;
  async cascadeFrom(trigger: string, depth: number): Promise<EnrichmentRequest[]>;
}

If person X is enriched and gains a new company: Acme field, cascade checks: does companies/acme exist? If not, create stub + enqueue at tier 2. Does companies/acme link back to X? If not, write the back-link. Iron Law is machine-enforced, not skill-enforced.

4.6 Fail-safe transactions

Every enrichment is wrapped in a BrainWriter transaction (Layer 4). Partial writes are rolled back. No asymmetric state like timeline-entry-without-raw-sidecar.

await writer.transaction(async (tx) => {
  const research = await registry.resolve('perplexity_query', {...}, ctx);
  await tx.appendTimeline(slug, {...});
  await tx.putRawData(slug, 'perplexity', research.raw);
  await tx.setFrontmatterField(slug, 'completeness', score);
  // All-or-nothing commit on exit.
});

5. Layer 3 — Scheduler

5.1 What's broken today

Garry's OpenClaw's cron is externally-driven JSON (cron/jobs.json) with ~30 jobs manually stagger-offset at different minutes. GBrain has zero native schedulingsrc/commands/autopilot.ts is a single daemon loop, and docs/guides/cron-schedule.md is architectural guidance, not code.

Failures observed in Garry's OpenClaw's actual state:

  • X OAuth2 Token Refresh: 11 consecutive timeouts (critical-path silent failure)
  • flight-tracker daily scan: 5 consecutive timeouts
  • morning-briefing: 4 consecutive timeouts
  • Quiet hours are checked at runtime in skills, so a skill that forgets to check will DM at 3 a.m.
  • Staggering is manual convention; no protection against two jobs colliding after a config edit.

5.2 ScheduledResolver interface

// src/core/scheduling/scheduler.ts
export interface Schedule {
  kind: 'cron' | 'interval';
  expr?: string;                    // cron string
  intervalMs?: number;
  tz: string;                       // IANA: "America/Los_Angeles"
  quietHours?: {
    startHour: number;              // 22 = 10 PM local
    endHour: number;                // 7 = 7 AM local
    policy: 'skip' | 'defer' | 'silent-run';
  };
  staggerKey?: string;              // jobs with same key auto-offset
  maxConcurrent?: number;           // global concurrency cap
  maxDurationMs?: number;           // timeout
}

export interface ScheduledResolver extends Resolver<void, ScheduledResult> {
  schedule: Schedule;
  retryPolicy: { maxRetries: number; backoffMs: number };
  circuitBreaker: { failureThreshold: number; cooldownMs: number };
  state: DurableState;              // watermark, content-hash, idempotency key
}

5.3 Enforcement vs convention (the key delta from Garry's OpenClaw)

Concern Garry's OpenClaw today Knowledge Runtime
Quiet hours Checked inside each skill (trust-based) Enforced at scheduler, skill cannot override
Staggering Manual minute-offset in jobs.json Scheduler assigns slots via hashed staggerKey
Concurrency MAX_BATCH_PROCESSES=2 in backoff, ignored by cron Global semaphore in scheduler
Timeout Per-job string in JSON, not always respected Enforced via AbortController, timeout raises TimeoutError caught by orchestrator
Retry None at cron level retryPolicy with exponential backoff
Silent failure "11 consecutive timeouts" unnoticed Circuit breaker opens at threshold → escalation to user
Idempotency State files per job, no framework DurableState primitive: watermark/ID/content-hash

5.4 Native engine + OS cron adapter

The scheduler runs as either:

  1. Embedded (default for gbrain autopilot): native event loop inside the daemon process. One process, many ScheduledResolvers.
  2. OS-driven (for Railway/launchd/systemd): gbrain schedule run <id> invoked by OS cron, scheduler state is durable so cross-invocation dedup still works.

Both modes share the same Schedule config + state.

5.5 Observability

Every scheduled run emits structured events: started, skipped-quiet-hours, deferred-to-active-hours, failed-retrying, circuit-opened, completed. Events go to:

  • ~/.gbrain/scheduler/events.jsonl (local, always)
  • engine.logIngest (audit trail in brain DB)
  • Optional webhook (Slack/Telegram for the user)

gbrain doctor reads the event log and reports: current circuit-breaker state, any resolver with > 3 consecutive failures, any resolver that hasn't fired within 3× its interval (freshness SLA like Garry's OpenClaw's freshness-check.mjs but built-in).


6. Layer 4 — Deterministic Output Builder

6.1 The anti-hallucination invariant

Iron Law: LLM picks WHAT. Code guarantees WHERE and HOW.

Garry's OpenClaw's existing lib/enrich.mjs:buildTweetEntry is close to this — tweet URLs are built from tweet.id returned by the X API, never from LLM memory. But:

  • A past incident: "Sub-agent test #2 FAILED — hallucinated 'Philip Leung' entity links across all daily files. LLM rewriting of daily files is too error-prone." (Garry's OpenClaw memory log, 2026-04-13.)
  • Back-links depend on appendTimeline being called everywhere; skips are silent.
  • Slug collisions are unchecked (no conflict detection on slugify).
  • Citation format is post-hoc linted weekly, not pre-write enforced.

6.2 BrainWriter

// src/core/output/writer.ts
export class BrainWriter {
  constructor(
    private engine: BrainEngine,
    private slugRegistry: SlugRegistry,
    private scaffolder: Scaffolder,
  ) {}

  async transaction<T>(fn: (tx: WriteTx) => Promise<T>): Promise<T>;
}

export interface WriteTx {
  // High-level typed operations; never raw string writes.
  createEntity(input: EntityInput): Promise<string>;          // returns slug, conflict-checked
  appendTimeline(slug: string, entry: TimelineInput): Promise<void>;
  setCompiledTruth(slug: string, body: CompiledTruthInput): Promise<void>;
  setFrontmatterField(slug: string, key: string, value: unknown): Promise<void>;
  putRawData(slug: string, source: string, data: object): Promise<void>;
  addLink(from: string, to: string, context: string): Promise<void>;  // auto-creates reverse back-link

  // Validators (called implicitly on commit)
  validate(): Promise<ValidationReport>;
}

Every user-visible URL/link/citation is built by code from resolver outputs, not from LLM text.

// src/core/output/scaffold.ts
export class Scaffolder {
  tweetCitation(handle: string, tweetId: string, dateISO: string): string {
    // "[Source: [X/garrytan, 2026-04-18](https://x.com/garrytan/status/123456)]"
  }
  emailCitation(account: string, messageId: string, subject: string): string {
    // deterministic Gmail URL per OpenClaw pattern
  }
  sourceCitation(resolverResult: ResolverResult<unknown>): string {
    // pulls .source, .fetchedAt, .raw from the result
  }
  entityLink(slug: string): string {
    // slugRegistry checks existence; returns resolvable wikilink
  }
}

6.4 SlugRegistry — conflict detection

// src/core/output/slug-registry.ts
export class SlugRegistry {
  async create(desiredSlug: string, displayName: string, type: PageType): Promise<CreatedSlug>;
  // Throws SlugCollision if another entity already occupies desiredSlug and isn't
  // confirmed as the same person (via email / x_handle / disambiguator).
  // Auto-resolves near-collisions by appending disambiguator.

  async confirmSame(slugA: string, slugB: string, confidence: number): Promise<void>;
  async merge(canonical: string, duplicate: string): Promise<void>;
}

6.5 Pre-write validators (fail-closed for integrity)

On WriteTx.validate() before commit:

  1. Citation validator. Every factual sentence in compiled_truth must have an inline [Source: ...] within N lines. Non-compliant paragraphs are flagged. Configurable: strict-mode rejects the transaction, lint-mode warns.
  2. Link validator. Every [text](path) must point to a page that exists OR to a URL the Scaffolder built (so it's guaranteed-valid). No raw LLM-composed URLs.
  3. Back-link validator. Every outbound link must have a reverse link written in the same transaction.
  4. Triple-HR validator. Compiled truth / timeline split enforced at the schema level.

Fails closed: the default is strict-mode. Loosening requires explicit writer.transaction({ strictMode: false }, ...) and logs a warning to the ingest log.

6.6 LLM output sanitization

Any LLM output destined for a brain page passes through a JSON-Schema-validated parser first. No free-form markdown goes to disk.

  • Entity extraction: JSON array of { name, type, context } per existing extractEntities pattern — strict validation.
  • Compiled-truth synthesis: LLM emits structured { sections: [{heading, paragraphs: [{text, sources: [...]}]}]}, scaffolder renders to markdown.
  • Timeline entries: LLM emits { date, summary, detail, sources }, scaffolder renders.

LLM never sees file paths, never writes files, never emits finished markdown.


7. Integration with existing GBrain

7.1 Reuse (already polished)

Existing Used by Change
src/core/fail-improve.ts (9/10) Wraps every Resolver in L1 None; becomes default wrapper
src/core/backoff.ts (9/10) ResolverContext.backoff None
src/core/storage.ts (9/10) Template for Resolver factory pattern None; serves as pattern reference
src/core/check-resolvable.ts (9/10) Extend to validate Resolver plugins Add checkResolvers() mode
src/commands/publish.ts (9/10) Uses BrainWriter under the hood Minor: route through L4
src/commands/backlinks.ts (8/10) Folded into L4 validator Keep as CLI-facing lint entry point
src/core/operations.ts validators Reused in ResolverContext trust enforcement None
src/core/engine.ts BrainEngine (35 methods) ResolverContext.engine Extend with getResolverRegistry()

7.2 Replace (ad-hoc today)

Existing Replace with
src/core/enrichment-service.ts (5/10) src/core/enrichment/orchestrator.ts (L2)
src/core/embedding.ts (monolithic) src/core/resolvers/builtin/embedding/openai.ts
src/core/transcription.ts (monolithic) src/core/resolvers/builtin/transcription/{groq,openai}.ts
src/commands/integrations.ts recipe format Unified Resolver plugin format (§3.5)
src/core/data-research.ts recipe format Same unified format
src/commands/autopilot.ts hard-coded daemon loop Wraps a set of ScheduledResolvers

7.3 Extend

  • src/core/engine.ts: add getResolverRegistry(), getWriter(), getScheduler(). Engine becomes the runtime's root container.
  • src/core/operations.ts: OperationContext inherits from ResolverContext (or vice-versa). Trust flags unified.
  • src/core/types.ts: add completeness: number to Page, sourcedBy: string[] for provenance.

8. Migration Path (phased, shippable)

Each phase ships independently, passes full E2E, is feature-flagged, and is reversible. No big-bang.

Phase 0 — Foundation (human: ~1 wk / CC: ~4 h)

  • Define Resolver<I,O>, ResolverContext, ResolverRegistry, ResolverResult (§3.23.4).
  • Add src/core/resolvers/index.ts wiring + tests for registry (register/get/list).
  • No behavioral change; ship as v0.11.0-alpha with feature flag.

Phase 1 — Three reference resolvers (human: ~1 wk / CC: ~4 h)

  • Port src/core/embedding.tsresolvers/builtin/embedding/openai.ts.
  • Implement resolvers/builtin/brain-local/slug-lookup.ts (wraps engine.resolveSlugs).
  • Implement resolvers/builtin/url-reachable.ts (HEAD-check).
  • Prove the interface: old callers swap to registry.resolve('openai_embedding', ...).

Phase 2 — BrainWriter + Slug Registry (human: ~1.5 wk / CC: ~6 h)

  • L4 core: BrainWriter.transaction, Scaffolder, SlugRegistry with conflict detection.
  • Pre-write validators: citation, link, back-link, triple-HR.
  • Migrate src/commands/publish.ts + src/commands/backlinks.ts to route through BrainWriter.
  • Now Garry's OpenClaw's "Philip Leung" hallucination is structurally impossible — LLM output passes through JSON-Schema validator before reaching Scaffolder.

Phase 3 — gbrain integrity command (human: ~0.5 wk / CC: ~2 h)

  • Ship the originally-scoped user-facing feature on top of the new foundation.
  • Uses Resolver SDK: x_handle_to_tweet + url_reachable.
  • Uses BrainWriter: all auto-repairs go through validated writes.
  • --auto --confidence 0.8 mode as user approved in cherry-pick #1.
  • User-visible value ships in Phase 3, not Phase 7.

Phase 4 — Enrichment Orchestrator (human: ~2 wk / CC: ~8 h)

  • L2 core: EnrichmentOrchestrator, BudgetLedger, CompletenessScorer, EntityGraph.cascadeFrom.
  • Migrate src/core/enrichment-service.ts callers (deprecate the old file after).
  • Completeness score in frontmatter on every write (dogfooding cascades).

Phase 5 — Scheduler (human: ~2 wk / CC: ~8 h)

  • L3 core: Scheduler, ScheduledResolver, DurableState, circuit breaker, quiet-hours enforcer.
  • Migrate src/commands/autopilot.ts to a ScheduledResolver set.
  • Ship gbrain schedule list|run|pause|tail CLI for observability.

Phase 6 — Port 58 OpenClaw resolvers (human: ~1.5 wk / CC: ~6 h)

  • perplexity_query, text_to_entities, mistral_ocr_pdf, x_search_all, x_user_to_tweets, gmail_query_to_threads, calendar_date_to_events.
  • Each ships as YAML + TS module under resolvers/builtin/proof of the plugin format.

Phase 7 — OpenClaw Adoption Integration (human: ~1 wk / CC: ~4 h)

  • Write docs/openclaw/ADOPTION.md showing your OpenClaw how to replace its 69 bespoke scripts with calls to gbrain registry.resolve(...).
  • Ship a gbrain claw-bridge subcommand that proxies Garry's OpenClaw's current script invocations to the resolver registry — zero-edit adoption path.
  • This is the test of the north star. If your OpenClaw can stand up a 1-line shim and drop scripts/x-api-client.mjs, the abstraction succeeded.

Total: human: ~10 weeks / CC: ~42 hours / calendar with single implementer: ~34 weeks.


9. Critical Files

New directories / files

src/core/
  runtime/
    index.ts                       # RuntimeContext (engine, storage, config, logger, metrics, budget)
    registry.ts                    # ResolverRegistry
    factory.ts                     # createResolver()
  resolvers/
    interface.ts                   # Resolver<I, O>
    fail-improve-wrapper.ts        # auto-wraps every resolver in FailImproveLoop
    builtin/
      x-api/
        handle-to-tweet.ts
        handle-to-tweet.yaml
      perplexity/
        query.ts
        query.yaml
      brain-local/
        slug-lookup.ts
        url-reachable.ts
      embedding/
        openai.ts                  # refactored from src/core/embedding.ts
      transcription/
        groq.ts
        openai.ts
  enrichment/
    orchestrator.ts                # EnrichmentOrchestrator
    tiers.ts                       # TIER_CONFIG
    budget.ts                      # BudgetLedger
    completeness.ts                # CompletenessScorer + per-type rubrics
    cascade.ts                     # EntityGraph
  scheduling/
    scheduler.ts                   # Scheduler + ScheduledResolver
    schedule.ts                    # Schedule type, cron expr parser
    state.ts                       # DurableState primitives
    quiet-hours.ts                 # TZ-aware enforcement
    stagger.ts                     # deterministic slot assignment
  output/
    writer.ts                      # BrainWriter
    scaffold.ts                    # Scaffolder (typed URL builders)
    slug-registry.ts               # SlugRegistry (conflict detection)
    validators/
      citation.ts
      link.ts
      back-link.ts
      triple-hr.ts

src/commands/
  integrity.ts                     # ships in Phase 3, replaces Feynman Phase A/B
  schedule.ts                      # gbrain schedule list|run|pause|tail (Phase 5)

docs/openclaw/
  ADOPTION.md                      # written in Phase 7

Replaced / removed

  • src/core/enrichment-service.ts — folded into enrichment/orchestrator.ts
  • src/core/embedding.ts — moved into resolvers/builtin/embedding/openai.ts
  • src/core/transcription.ts — moved into resolvers/builtin/transcription/

Extended

  • src/core/engine.ts — add getResolverRegistry(), getWriter(), getScheduler()
  • src/core/operations.ts — unify with ResolverContext; every operation validator reusable by resolvers
  • src/core/types.ts — add completeness: number, sourcedBy: string[], lastVerified: Date

10. Testing Strategy

Contract tests

Every Resolver implementation tested against the interface spec. Table-driven: run the same suite against openai_embedding, x_handle_to_tweet, etc. Ensures plugin authors can't ship broken resolvers.

Property tests

  • Idempotency: running a ScheduledResolver twice with the same state produces the same output and doesn't double-write.
  • Atomicity: a BrainWriter transaction that throws mid-flight leaves the brain bit-for-bit identical to pre-transaction.
  • Deterministic scaffolds: given the same resolver outputs, the Scaffolder produces byte-identical citations/links.

Integration tests

  • EnrichmentOrchestrator end-to-end against PGLite (in-memory, no API keys) with mocked resolver registry.
  • Scheduler with fake clock + quiet-hours scenarios.
  • BrainWriter transaction rollback on validator failure.

Chaos tests

  • Kill the process mid-enrichment; next run must resume cleanly.
  • Simulate API timeout mid-transaction; transaction must roll back completely.
  • Corrupted state file; scheduler must escalate, not silently skip.

Regression tests vs. Garry's OpenClaw behavior

For each OpenClaw pattern we port (e.g. X-handle → tweet URL), a regression test proves the new resolver produces the same answer on real-world inputs from the brain audit. This is the "your OpenClaw would adopt" proof.


11. Open Questions (flagged for CEO re-review)

  1. Scope shape. Is this the right four-layer decomposition, or are some layers better left to OpenClaw (e.g. Scheduling lives above GBrain, not in it)?
  2. Phase 3 user-value break. Does Phase 3 (user-visible gbrain integrity) ship early enough, or do we need an even smaller MVP?
  3. LLM-as-resolver. Should text_to_entities be a Resolver, or does that blur the "code vs LLM" line the invariant relies on?
  4. Plugin format. YAML + TS module (§3.5) vs. pure TS module with decorator-style metadata. Latter is more type-safe; former is more discoverable.
  5. Cross-resolver transactions. Do we support "atomic fetch-from-Perplexity + write-to-brain" at the L2 layer? Current design says yes; implementation is tricky (Perplexity call isn't rollbackable).
  6. OpenClaw bridge scope. Phase 7 gbrain claw-bridge — is that worth a phase of its own, or should adoption be documentation-only?
  7. Completeness rubric coverage. Do we define rubrics for all 9 PageTypes upfront, or ship people/company/meeting first and extend incrementally?
  8. Budget config UX. Hard daily cap is strict; should we also expose a soft-cap warning mode, and how is the cap set (env var? config file? prompt on first use?)
  9. Backwards compat. src/commands/publish.ts and src/commands/backlinks.ts have been running cleanly for weeks. Refactoring through BrainWriter carries migration risk. Acceptable?
  10. Existing TODOS alignment. TODOS.md has P0 "Runtime MCP access control" and P2 security hardening. The new RuntimeContext.remote flag interacts with both — do we fold MCP access control into Phase 0 or keep separate?

12. Verification (the "your OpenClaw would adopt" test)

The design succeeds iff:

  • A user can add a new resolver by dropping a YAML + TS module in ~/.gbrain/resolvers/ without editing GBrain source.
  • Your OpenClaw can delete scripts/x-api-client.mjs and replace all callers with 1-line await registry.resolve('x_handle_to_tweet', ...).
  • No brain page can be written with a bare tweet reference, a missing back-link, or an unverified URL (validators catch it pre-commit).
  • Running gbrain integrity --auto --confidence 0.8 over a real brain fixes ≥1,000 of the 1,424 known bare-tweet citations without human review.
  • Full E2E test suite passes on both PGLite + Postgres engines.
  • The Knowledge Runtime ships across 7 phases with each phase individually shippable and reversible.