* v0.28 schema: takes + synthesis_evidence (v31) + access_tokens.permissions (v32)
Migration v31 adds the takes table (typed/weighted/attributed claims) and
synthesis_evidence (provenance for `gbrain think` outputs). Page-scoped via
page_id FK (slug isn't unique alone in v0.18+ multi-source). HNSW partial
index on embedding for active rows. ON DELETE CASCADE on synthesis_evidence
so deleting a source take cascades the provenance row.
Migration v32 adds access_tokens.permissions JSONB with safe-default
backfill (`{"takes_holders":["world"]}`). Default keeps non-world holders
hidden from MCP-bound tokens until the operator explicitly grants access
via the v0.28 auth permissions CLI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28 engine: addTakesBatch, listTakes, searchTakes/Vector, supersede, resolve, synthesis_evidence
Extends BrainEngine with the takes domain object. Both engines implement the
same surface; PGLite uses manual `$N` placeholders, Postgres uses postgres-js
unnest() — same shape as addLinksBatch and addTimelineEntriesBatch.
Methods:
- addTakesBatch (upsert via ON CONFLICT (page_id, row_num) DO UPDATE)
- listTakes (filter by holder/kind/active/resolved, takesHoldersAllowList
for MCP-bound calls, sortBy weight/since_date/created_at)
- searchTakes / searchTakesVector (pg_trgm + cosine; honor allow-list)
- countStaleTakes / listStaleTakes (mirror countStaleChunks pattern;
embedding column intentionally omitted from listStale payload)
- updateTake (mutable fields only; throws TAKE_ROW_NOT_FOUND)
- supersedeTake (transactional: insert new at next row_num, mark old
active=false, set superseded_by; throws TAKE_RESOLVED_IMMUTABLE on
resolved bets)
- resolveTake (sets resolved_*; throws TAKE_ALREADY_RESOLVED on re-resolve;
resolution is immutable per Codex P1 #13 fold)
- addSynthesisEvidence (provenance persist; ON CONFLICT DO NOTHING)
- getTakeEmbeddings (parallel to getEmbeddingsByChunkIds)
Types live in src/core/engine.ts adjacent to LinkBatchInput. Page-scoped
via page_id (slug not unique in v0.18+ multi-source). PageType gains
'synthesis'. takeRowToTake mapper in utils.ts handles Date → ISO string
normalization.
Tests: test/takes-engine.test.ts — 16 cases against PGLite covering
upsert/list/filter/search happy paths, takesHoldersAllowList isolation,
the four invariant errors (TAKE_ROW_NOT_FOUND, TAKES_WEIGHT_CLAMPED,
TAKE_RESOLVED_IMMUTABLE, TAKE_ALREADY_RESOLVED), supersede flow, resolve
metadata round-trip, FK CASCADE on synthesis_evidence when source take
deletes. All pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28 model-config: unified resolveModel with 6-tier precedence + alias resolution
Replaces every hardcoded `claude-*-X` and per-phase `dream.<phase>.model`
config key with a single resolver. Hierarchy:
1. CLI flag (--model)
2. New-key config (e.g. models.dream.synthesize)
3. Old-key config (deprecated dream.synthesize.model, dream.patterns.model)
— read with stderr deprecation warning, one-per-process
4. Global default (models.default)
5. Env var (GBRAIN_MODEL or caller-supplied)
6. Hardcoded fallback
Aliases (`opus`, `sonnet`, `haiku`, `gemini`, `gpt`) resolve at the end so
any tier can use a short name. User-defined `models.aliases.<name>` config
overrides built-ins. Cycle-safe (depth 2 break). Unknown alias passes
through unchanged so users can pass full provider IDs without registering.
When new-key + old-key are BOTH set (Codex P1 #11 fix), new-key wins and
stderr warns "deprecated config X ignored; Y is set and wins". When only
old-key is set, it's honored with a softer "rename to Y before v0.30"
warning. Both warnings emit once per (key, process) — a Set memo prevents
log spam in long-running daemons.
Migrated call sites: synthesize.ts (model + verdictModel), patterns.ts
(model). subagent.ts and search/expansion.ts to be migrated later in v0.28
(staying compatible until then).
Tests: test/model-config.test.ts — 11 cases pinning the 6-tier ordering,
alias resolution + cycle break, deprecated-key warning emit-once, and
unknown-alias pass-through. All pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28 takes-fence: parser/renderer/upserter + chunker strip (privacy P0 fix)
src/core/takes-fence.ts — pure functions for the fenced markdown surface:
- parseTakesFence(body) — extracts ParsedTake[] from `<!--- gbrain:takes:begin/end -->`
blocks. Strict on canonical form, lenient on hand-edits with warnings
(TAKES_FENCE_UNBALANCED, TAKES_TABLE_MALFORMED, TAKES_ROW_NUM_COLLISION).
Strikethrough `~~claim~~` → active=false; date ranges `since → until`
split into sinceDate/untilDate.
- renderTakesFence(takes) — round-trip safe with parseTakesFence.
- upsertTakeRow(body, row) — append-only per CEO-D6 + eng-D9. Creates a
fresh `## Takes` section if no fence present. row_num is monotonic
(max + 1, never gap-filled — keeps cross-page refs and synthesis_evidence
stable forever).
- supersedeRow(body, oldRow, replacement) — strikes through old row's claim
AND appends the new row at end. Both rows preserved in markdown for
git-blame archaeology.
- stripTakesFence(body) — removes the fenced block entirely. Used by the
chunker so takes content lives ONLY in the takes table.
Codex P0 #3 fix: src/core/chunkers/recursive.ts now calls stripTakesFence()
before computing chunk boundaries. Without this, page chunks would contain
the rendered takes table and the per-token MCP allow-list would be
bypassed at the index layer (token bound to takes_holders=['world'] would
see garry's hunches via page hits). Doctor's takes_fence_chunk_leak check
(plan-side) asserts no chunk contains the begin marker.
Tests: 15 cases covering canonical parse, strikethrough, date range, fence
unbalanced detection, malformed-row skip + warning, row_num collision
detection, round-trip render, append-only upsert into existing fence,
fresh-section creation, monotonic row_num under hand-edit gaps, supersede
flow, stripTakesFence verifying takes content removed AND surrounding
prose preserved. Existing chunker tests still pass (15 + 15 = 30).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28 page-lock: PID-liveness file lock for atomic markdown read-modify-write
src/core/page-lock.ts — per-page file lock at
~/.gbrain/page-locks/<sha256-of-slug>.lock so two concurrent `gbrain takes
add` calls or `takes seed --refresh` from autopilot can't race on the
same `<slug>.md` read-modify-write. Eng-review fold: reuses the v0.17
cycle.lock pattern (mtime + PID liveness) but per-slug.
Differences from cycle.ts's lock:
- SHA-256 of slug for safe filenames (slashes, unicode, etc.)
- Same-pid + fresh mtime = LIVE (cycle.ts assumes one lock per process and
reclaims same-pid; page-lock allows concurrent locks for DIFFERENT slugs
in one process). mtime expiry still rescues post-crash leftovers.
- 5-min TTL (vs cycle's 30 min — page edits are short)
- `withPageLock(slug, fn)` convenience wrapper with default 30s timeout
API:
- acquirePageLock(slug, opts) → handle | null (poll-with-timeout)
- handle.refresh() / handle.release() (idempotent — only releases if pid matches)
- withPageLock(slug, fn, opts) — acquire + run + release-in-finally
Tests: 10 cases — fresh acquire, live holder returns null, stale-mtime
reclaim, dead-PID reclaim, refresh updates timestamp, foreign-pid release
is no-op, withPageLock callback runs and releases on success/failure,
timeout-throws when held, SHA-256 filename safety for slashes/unicode.
All pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28 extract-takes: dual-path phase (fs|db) + since/until_date as TEXT
src/core/cycle/extract-takes.ts — new phase that materializes the takes
table from fenced markdown blocks. Two paths mirror src/commands/extract.ts:
- extractTakesFromFs: walk *.md under repoPath, parse fences, batch upsert
- extractTakesFromDb: iterate engine.getAllSlugs(), parse each page's
compiled_truth+timeline, batch upsert (mutation-immune snapshot iteration)
Single dispatcher extractTakes(opts) routes by source. Honors:
- slugs filter for incremental re-extract (pipes from sync→extract)
- dryRun: count would-be upserts, write nothing
- rebuild: DELETE FROM takes WHERE page_id = $1 before re-insert (clean
slate when markdown is canonical and DB has drifted)
Schema fix: since_date/until_date were DATE in the original v31 migration.
Spec uses partial dates ('2017-01', '2026-04-29 → 2026-06') that Postgres
DATE rejects. Changed to TEXT in both the Postgres and PGLite blocks so
parser-rendered ranges round-trip cleanly. Loses the ability to do
date-range arithmetic in SQL, but date math on opinion timelines is
out of scope for v0.28 anyway. utils.ts dateOrNull now annotated as
v0.28 TEXT-aware.
Migration v31 has not been deployed yet (this branch is the v0.28 release
candidate), so the type swap is free. No data migration needed.
Tests: test/extract-takes.test.ts — 5 cases against PGLite covering full
walk + fence-skip on no-fence pages, takes-table populated post-extract,
incremental slugs filter, dry-run no-write, rebuild=true clears + re-inserts
ad-hoc rows. test/takes-engine.test.ts (16), test/takes-fence.test.ts (15)
all still pass — 36/36 takes tests green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28 takes CLI: list, search, add, update, supersede, resolve
src/commands/takes.ts — surfaces the engine methods + takes-fence library
through a single `gbrain takes <subcommand>` entrypoint:
takes <slug> list with filters + sort
takes search "<query>" pg_trgm keyword search across all takes
takes add <slug> --claim ... ... append (markdown + DB, atomic via lock)
takes update <slug> --row N ... mutable-fields update (markdown + DB)
takes supersede <slug> --row N ... strikethrough old + append new
takes resolve <slug> --row N --outcome record bet resolution (immutable)
Markdown is canonical. Every mutate command:
1. acquires the per-page file lock (withPageLock)
2. re-reads the .md file
3. applies the edit via takes-fence (upsertTakeRow / supersedeRow)
4. writes the .md file back
5. mirrors to the DB via the engine method
6. releases the lock (auto via finally)
Resolve currently writes only to DB — surfacing resolved_* in the markdown
table is deferred to v0.29 (the takes-fence renderer's column set is
fixed at # | claim | kind | who | weight | since | source per spec).
Wired into src/cli.ts dispatch + CLI_ONLY allowlist. Help text follows the
project convention (orphans/embed/extract pattern). --dir flag overrides
sync.repo_path config when working outside the configured brain.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28 MCP + auth: takes_list / takes_search / think ops + per-token allow-list
OperationContext gains takesHoldersAllowList — server-side filter for
takes.holder field threaded from access_tokens.permissions through dispatch
into the engine SQL. Closes Codex P0 #3 at the dispatch layer (chunker
strip already closed the page-content side in the previous commit).
src/core/operations.ts — three new ops:
- takes_list: lists takes with holder/kind/active/resolved filters; honors
ctx.takesHoldersAllowList for MCP-bound calls
- takes_search: pg_trgm keyword search; honors allow-list
- think: op surface registered (returns not_implemented envelope until
Lane D's pipeline lands). Remote callers cannot save/take per Codex P1 #7.
src/mcp/dispatch.ts — DispatchOpts.takesHoldersAllowList threads into
buildOperationContext.
src/mcp/http-transport.ts — validateToken now reads
access_tokens.permissions.takes_holders, defaults to ['world'] when the
column is absent or malformed (default-deny on private hunches).
auth.takesHoldersAllowList passed to dispatchToolCall.
src/mcp/server.ts (stdio) — defaults to takesHoldersAllowList: ['world']
since stdio has no per-token auth. Operators wanting full visibility use
`gbrain call <op>` directly (sets remote=false).
src/commands/auth.ts — `gbrain auth create <name> --takes-holders w,g,b`
flag persists the per-token list; new `auth permissions <name>
set-takes-holders <list>` updates an existing token.
Tests: test/takes-mcp-allowlist.test.ts — 8 cases against PGLite proving
the threading: local-CLI sees all holders, ['world'] returns only public,
['world','garry'] returns 2/3, no-overlap returns empty (no fallback),
search honors allow-list, remote save/take on think rejected with
not_implemented envelope.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28.0: ship-prep — VERSION, CHANGELOG, migration orchestrator, skill
Closes the v0.28 ship-prep cycle. Bumps VERSION + package.json + bun.lock
to 0.28.0. v0_28_0 migration orchestrator runs three idempotent phases on
upgrade:
- Schema verify: asserts schema_version >= 32 (migrations v31 + v32 already
applied by the schema runner during gbrain upgrade); fails clean if not.
- Backfill takes: inline runs `extractTakes(engine, { source: 'db' })` so
any pre-existing fenced takes tables in markdown populate the takes
index. Idempotent; ON CONFLICT DO UPDATE keeps the table in sync.
- Re-chunk TODO: queues a pending-host-work entry asking the host agent
to re-import pages with takes content so the v0.28 chunker-strip rule
(Codex P0 #3 fix) applies retroactively. Pages imported under v0.28+
already have takes content stripped from chunks at index time; this
TODO catches up legacy pages.
skills/migrations/v0.28.0.md — agent-readable upgrade guide. Walks
through doctor verification, deprecated-key migration, MCP token
visibility configuration, and a "try the takes layer" smoke test.
CHANGELOG.md — v0.28.0 release-summary in the GStack voice (no AI
vocabulary, no em dashes, real numbers from git diff stat) + the
mandatory "To take advantage of v0.28.0" block + itemized changes by
subsystem (schema, engine, markdown surface, model config, MCP+auth,
CLI, tests, accepted risks).
Final test sweep: 65/65 v0.28 tests pass across 6 files. typecheck clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28 think pipeline: gather → sanitize → synthesize → cite-render → CLI
src/core/think/sanitize.ts — prompt-injection defense for take claims:
14 jailbreak patterns (ignore-prior, role-jailbreak, close-take tag,
DAN, system-prompt overrides, eval-shell hooks) plus structural framing
(takes wrapped in <take id="..."> tags the model is told to treat as
DATA). Length-cap at 500 chars. Renders evidence blocks for the prompt.
src/core/think/prompt.ts — system prompt + structured-output schema.
Hard rules: cite every claim, mark hunches/low-weight explicitly,
surface conflicts (never silently pick), surface gaps. JSON schema
with answer + citations[] + gaps[]. Prompt adapts to anchor / time
window / save flag.
src/core/think/cite-render.ts — structured citations + regex fallback
(Codex P1 #4 fold). normalizeStructuredCitations validates the model's
structured output; parseInlineCitations is the body-scan fallback when
the model omits the structured field. resolveCitations dispatches and
records CITATIONS_REGEX_FALLBACK warning when used.
src/core/think/gather.ts — 4-stream parallel retrieval:
1. hybridSearch (pages, existing primitive)
2. searchTakes (keyword, pg_trgm)
3. searchTakesVector (vector, when embedQuestion fn supplied)
4. traversePaths (graph, when --anchor set)
RRF fusion (k=60). Each stream wrapped in try/catch — partial gather
beats no synthesis. Honors takesHoldersAllowList for MCP-bound calls.
src/core/think/index.ts — runThink orchestrator + persistSynthesis:
INTENT (regex classify) → GATHER → render evidence blocks → resolveModel
('models.think' → 'models.default' → GBRAIN_MODEL → opus) → LLM call
(injectable client) → JSON parse with code-fence + fallback strip →
resolveCitations → ThinkResult. persistSynthesis writes a synthesis
page + synthesis_evidence rows (page_id resolved per slug; page-level
citations skip evidence). Degrades gracefully without ANTHROPIC_API_KEY.
Round-loop scaffolding in place (rounds=1 only path exercised in v0.28).
src/commands/think.ts — `gbrain think "<question>"` CLI. Flag parsing
strips --anchor, --rounds, --save, --take, --model, --since, --until,
--json. Local CLI = remote=false, so save/take honored. Human-readable
output by default; --json for agent consumption.
operations.ts — `think` op now calls runThink (was a not_implemented
stub). Remote callers can't save/take per Codex P1 #7. Returns full
ThinkResult plus saved_slug + evidence_inserted.
cli.ts — wired into dispatch + CLI_ONLY allowlist.
Tests: test/think-pipeline.test.ts — 18 cases against PGLite covering
sanitize patterns, structural rendering, citation parsing (structured +
regex fallback + dedup + invalid-slug rejection), gather streams +
allow-list filter, full pipeline with stub client, malformed-LLM
fallback path, no-API-key graceful degradation, persistSynthesis writes
page + evidence rows. All pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28 dream phases: auto-think + drift + budget meter (Codex P1 #10 fold)
src/core/anthropic-pricing.ts — USD/1M-tokens map for Claude 4.7 family
plus older aliases. estimateMaxCostUsd returns null on unpriced models so
the meter caller can warn-once and bypass the gate.
src/core/cycle/budget-meter.ts — cumulative cost ledger. Each submit
estimates max-cost from (model + estimatedInputTokens + maxOutputTokens),
accumulates per-cycle, refuses next submit when projected > cap. Codex
P1 #10 fold: non-Anthropic models (gemini, gpt) bypass with one stderr
warn per process and `unpriced=true` on the result. Budget=0 disables
the gate. Audit trail at ~/.gbrain/audit/dream-budget-YYYY-Www.jsonl.
src/core/cycle/auto-think.ts — auto_think dream phase. Reads
dream.auto_think.{enabled,questions,max_per_cycle,budget,cooldown_days,
auto_commit}. Iterates configured questions through runThink with the
BudgetMeter pre-checking each submit. Cooldown timestamp written ONLY on
success (matches v0.23 synthesize pattern — retries after partial
failures pick back up). When auto_commit=true, persists synthesis pages
via persistSynthesis. Default-disabled.
src/core/cycle/drift.ts — drift dream phase scaffold. Reads
dream.drift.{enabled,lookback_days,budget,auto_update}. Surfaces takes
in the soft band (weight 0.3-0.85, unresolved) that have recent timeline
evidence on the same page. v0.28 ships the orchestration; the LLM judge
that proposes weight adjustments lands in v0.29. modelId + meter wired
now so the ledger captures gate state for callers that opt in.
Tests:
- test/budget-meter.test.ts (7 cases) — pricing-map coverage, allow path,
cumulative-deny, budget=0 disabled, unpriced bypass+warn-once, ledger
captures all events, ISO-week filename branch.
- test/auto-think-phase.test.ts (9 cases) — auto_think enable/skip,
questions empty, success → cooldown ts written, cooldown blocks rerun,
budget exhausted → partial. drift not_enabled, soft-band candidate
detection, complete + dry-run paths.
All pass. Typecheck clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28 e2e Postgres: takes engine + extract + MCP allow-list (12 cases)
test/e2e/takes-postgres.test.ts — full v0.28 takes pipeline against real
Postgres (gated on DATABASE_URL). 12 cases:
- addTakesBatch upsert via unnest() bind path (Postgres-specific)
- listTakes filters: holder, kind, sort=weight, takesHoldersAllowList
- searchTakes pg_trgm + allow-list filter
- supersedeTake transactional path (BEGIN/COMMIT semantics)
- resolveTake immutability — second resolve throws TAKE_ALREADY_RESOLVED
- synthesis_evidence FK CASCADE on take delete
- countStaleTakes + listStaleTakes filter active+null
- extractTakesFromDb populates takes from fenced markdown
- MCP dispatch with takesHoldersAllowList=['world'] returns only world
- MCP dispatch local-CLI path returns all holders
- MCP dispatch takes_search honors allow-list
- think op forces remote_persisted_blocked even for save+take
postgres-engine.ts: addTakesBatch boolean[] serialization fix.
postgres-js auto-detects element type from JS arrays; for booleans it
mis-detects as scalar. Cast through text[] (`'true' | 'false'`) then
SQL-cast to boolean[] — same pattern other batch methods rely on for
type-stable bind shapes.
test/e2e/helpers.ts: setupDB now (a) tolerates non-existent tables in
TRUNCATE (for fresh DBs where v31 hasn't yet created takes/synthesis_evidence)
and (b) calls engine.initSchema() to actually run migrations.
test/takes-mcp-allowlist.test.ts: updated 2 think-op cases to match
Lane D's landed pipeline. They previously asserted not_implemented
envelopes; now they assert remote_persisted_blocked + NO_ANTHROPIC_API_KEY
graceful-degrade behavior.
Run: DATABASE_URL=postgres://localhost:5435/gbrain_test bun test test/e2e/takes-postgres.test.ts
Result: 12/12 pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28 dream phases: local DreamPhaseResult type (avoid premature CyclePhase enum extension)
cycle.ts's PhaseResult is shaped {phase, status, summary, details} with a
narrow PhaseStatus enum ('ok'|'warn'|'fail'|'skipped') and CyclePhase enum
that doesn't yet include 'auto_think'/'drift'. The phases ship standalone
in v0.28 (cycle.ts dispatcher integration is v0.28.x); using PhaseResult
forced premature enum extension.
Introduces DreamPhaseResult exported from auto-think.ts:
{ name: 'auto_think'|'drift'; status: 'complete'|'partial'|'failed'|'skipped';
detail: string; totals?: Record<string,number>; duration_ms: number }
drift.ts re-exports the same type. When v0.28.x wires the dispatcher, the
adapter at the call site can map DreamPhaseResult → PhaseResult cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28 e2e: access_tokens.permissions JSONB end-to-end (5 cases)
test/e2e/auth-permissions.test.ts — closes the v0.28 token-allow-list
verification loop against real Postgres. Exercises:
- Migration v32 default backfill: new tokens created without a permissions
column get {takes_holders: ["world"]} via the schema DEFAULT clause.
- Explicit ["world","garry"] → dispatch.takes_list filters to those
holders only; brain hunches stay hidden from this token.
- ["world"] default-deny token → takes_search hits filtered to public claims.
- {} permissions row (operator tampered) gracefully defaults to ["world"]
via the HTTP transport's validateToken parsing.
- revoked_at IS NOT NULL → token excluded from active token query.
Avoids the postgres-js JSONB double-encode trap (CLAUDE.md memory): pass
the object directly to executeRaw, no JSON.stringify, no ::jsonb cast.
All 5 pass against pgvector/pgvector:pg16 on port 5435. Combined v0.28
test sweep: 116/116 across 11 files.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28 e2e: chunker takes-strip integration test (Codex P0 #3 verification)
test/e2e/chunker-takes-strip.test.ts — verifies the chunker actually
strips fenced takes content end-to-end through the import pipeline.
This is the Codex P0 #3 fix's verification path: takes content lives
ONLY in the takes table for retrieval, never duplicated in
content_chunks where the per-token MCP allow-list cannot reach.
5 cases:
- chunkText (unit) output never contains TAKES_FENCE_BEGIN/END markers
- chunkText output never contains fenced claim text
- chunkText output retains non-fence prose (no over-stripping)
- importFromContent end-to-end: imported page has chunks but none
contain fenced content
- takes_fence_chunk_leak doctor invariant: zero rows globally where
chunk_text matches `<!--- gbrain:takes:%`
Final v0.28 test sweep:
121 pass, 0 fail, 336 expect() calls, 12 files
Coverage: schema migrations, engine methods (PGLite + Postgres),
takes-fence parser, page-lock, extract phase, takes CLI engine
surface, model config 6-tier resolver, MCP+auth allow-list,
think pipeline (gather + sanitize + cite-render + synthesize),
auto-think + drift + budget meter, JSONB end-to-end, chunker
strip integration. ~95% of v0.28 surface area covered.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix CI: apply-migrations skippedFuture arrays + http-transport SQL mock
Two CI failures from PR #563:
test/apply-migrations.test.ts (2 fails) — `buildPlan` tests assert exact
skippedFuture arrays at fixed installed-version stamps. Adding v0.28.0 to
the migration registry means it shows up in skippedFuture when the test
runs at installed=0.11.1 / installed=0.12.0. Append '0.28.0' to both
hardcoded arrays.
test/http-transport.test.ts (8 fails) — the FakeEngine mock string-prefix
matches `SELECT id, name FROM access_tokens` to return a row. v0.28's
validateToken now selects `SELECT id, name, permissions FROM access_tokens`
to read the per-token takes_holders allow-list. Mock returned [] on the
new query → validateToken treated every token as invalid → 401.
Fix: mock now matches both query shapes. validTokens row gets a default
`{takes_holders: ['world']}` permission injected when caller didn't
supply one (mirrors the migration v33 column DEFAULT). Updated
FakeEngineConfig type to allow tests to pass explicit permissions.
Verification:
bun test test/apply-migrations.test.ts → 18/18 pass
bun test test/http-transport.test.ts → 24/24 pass
bun run typecheck → clean
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix CI: add scope annotations to v0.28 ops (takes_list/takes_search/think)
test/oauth.test.ts enforces an invariant from master's v0.26 OAuth landing:
every Operation must have `scope: 'read' | 'write' | 'admin'`, and any op
flagged `mutating: true` must be 'write' or 'admin'. My v0.28 ops were added
before master shipped v0.26 + the new invariant; the merge surfaced the gap.
Annotations:
- takes_list → read
- takes_search → read
- think → write (mutating: true; --save persists synthesis page)
Verification:
bun test test/oauth.test.ts → 42/42 pass
bun run typecheck → clean
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(v0.28.1): export INJECTION_PATTERNS for shared sanitization
The same pattern set protects takes from prompt-injection (think/sanitize.ts)
and now retrieved chat content in the LongMemEval harness. One source of
truth for both surfaces; adding a new pattern in this file automatically
covers benchmarks too.
Existing consumers (sanitizeTakeForPrompt, renderTakesBlock) keep working
unchanged. Verified via test/think-pipeline.test.ts (18 pass, 0 fail).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(v0.28.1): longmemeval harness — reset-in-place over in-memory PGLite
One in-memory PGLiteEngine per benchmark run; TRUNCATE between questions
with runtime-enumerated tables via pg_tables so future schema migrations
don't silently leak across questions. Infrastructure tables (sources,
config, gbrain_cycle_locks, subagent_rate_leases) preserved across resets
so initSchema-seeded rows like sources.'default' survive (FK target for
pages.source_id).
Files:
- src/eval/longmemeval/harness.ts: createBenchmarkBrain + resetTables +
withBenchmarkBrain. ~50 lines, no class wrapper.
- src/eval/longmemeval/adapter.ts: pure haystackToPages() converter.
Slug prefix `chat/` (verified non-matching against DEFAULT_SOURCE_BOOSTS).
- src/eval/longmemeval/sanitize.ts: re-uses INJECTION_PATTERNS from
think/sanitize.ts; wraps each session in <chat_session id date> tags;
4000-char cap.
- test/longmemeval-sanitize.test.ts: 12 cases pinning the F8 contract.
Hermetic: no DATABASE_URL, no API keys.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(v0.28.1): gbrain eval longmemeval CLI command
Run the LongMemEval public benchmark against gbrain's hybrid retrieval.
Dataset is a positional path (download from xiaowu0162/longmemeval on HF).
Per-question loop wraps everything in try/catch; one bad question doesn't
kill the run, error JSONL line emitted instead.
Wiring:
- src/cli.ts: pre-dispatch bypass for `eval longmemeval` so the user's
~/.gbrain brain is never opened. Hermeticity gate verified: --help works
on machines with no gbrain config.
- src/commands/eval-longmemeval.ts: arg parsing, JSONL emit (LF + UTF-8
pinned), hybridSearch with optional expandQuery from search/expansion.ts,
resolveModel from model-config.ts (6-tier chain), ThinkLLMClient injection
seam from think/index.ts, structural <chat_session> framing.
- test/eval-longmemeval.test.ts: 12 cases covering harness lifecycle,
reset clears all tables, schema-migration robustness, p50/p99 speed gate
(warm reset+import+search target <500ms), adapter shape, source-boost
regression guard, end-to-end with stubbed LLM, JSONL format guard,
per-question failure handling.
- test/fixtures/longmemeval-mini.jsonl: 5 hand-authored questions with
keyword-friendly overlap so --keyword-only works in CI.
Speed: warm reset+import 5 pages+search p50=25.9ms p99=30.3ms locally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(v0.28.1): bump VERSION + CHANGELOG
VERSION + package.json synchronized at 0.28.1. CHANGELOG entry uses the
release-summary voice + "To take advantage of v0.28.1" block per CLAUDE.md.
Sequential release on garrytan/v0.28-release; lands after v0.28.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: surface v0.28.1 LongMemEval CLI across project docs
- README.md: add EVAL section to Commands reference (eval --qrels, export,
prune, replay, longmemeval); add v0.28.1 announce paragraph next to the
v0.25.0 BrainBench-Real intro.
- CLAUDE.md: add Key files entry for src/eval/longmemeval/ +
src/commands/eval-longmemeval.ts; add "Key commands added in v0.28.1"
subsection (mirrors the v0.26.5 / v0.25.0 pattern); inventory
test/eval-longmemeval.test.ts + test/longmemeval-sanitize.test.ts under
the unit-test list.
- docs/eval-bench.md: cross-link from the "What it actually does" section
to LongMemEval as the third evaluation axis (public benchmark,
ground-truth labels, full QA pipeline); append "Public benchmarks:
LongMemEval (v0.28.1)" section with architecture, flags table, and
perf numbers.
- CONTRIBUTING.md: append a paragraph after the eval-replay block pointing
contributors at gbrain eval longmemeval for public-benchmark coverage.
- AGENTS.md: extend the existing eval-retrieval bullet with a one-line
mention of gbrain eval longmemeval.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28.2 feat: remote-source MCP + scope hierarchy + whoami (#690)
* refactor(core): extract SSRF helpers from integrations.ts to core/url-safety.ts
src/core/git-remote.ts (next commit) needs isInternalUrl etc. but importing
from src/commands/ would invert the layering boundary (no existing
src/core/ file imports from src/commands/). Extract the SSRF helpers
(parseOctet, hostnameToOctets, isPrivateIpv4, isInternalUrl) into a new
src/core/url-safety.ts and have integrations.ts re-export for backward
compat. test/integrations.test.ts continues to pass without changes (110
existing tests, 214 expects).
Why this matters for v0.28: the upcoming sources --url feature reuses
this SSRF gate for git-clone URL validation. Codex review caught that
re-rolling weaker URL classification would regress on the IPv6/v4-mapped/
metadata/CGNAT bypass forms that integrations.ts already handles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(core): add git-remote module — SSRF-defensive clone/pull + state probe
New src/core/git-remote.ts (~210 lines) for v0.28's remote-source feature:
- GIT_SSRF_FLAGS exported const: -c http.followRedirects=false,
-c protocol.file.allow=never, -c protocol.ext.allow=never,
--no-recurse-submodules. Single source of truth shared by cloneRepo
and pullRepo so a future flag added to one path lands on both.
Closes the SSRF surfaces codex flagged: DNS rebinding via redirects,
.gitmodules as a second-fetch surface, file:// scheme in remotes.
- parseRemoteUrl: https-only, rejects embedded credentials and path
traversal, delegates internal-target classification to isInternalUrl
from url-safety.ts (covers RFC1918, link-local, loopback, IPv6, CGNAT
100.64/10, metadata hostnames, hex/octal/single-int bypass forms).
GBRAIN_ALLOW_PRIVATE_REMOTES=1 escape hatch with stderr warning is
needed for self-hosted git over Tailscale (CGNAT trips the gate).
- cloneRepo: --depth=1 default (full clone via depth: 0); refuses
non-empty destDirs; spawns git via execFileSync (no shell injection)
with GIT_TERMINAL_PROMPT=0 + askpass=/bin/false to prevent credential
prompts. timeoutMs default 600s.
- pullRepo: -C path + GIT_SSRF_FLAGS + pull --ff-only, same env confine.
- validateRepoState: 6-state decision tree (missing | not-a-dir |
no-git | corrupted | url-drift | healthy). Used by performSync's
re-clone branch to recover from rmd clone dirs and refuse syncs on
url-drift or corruption.
test/git-remote.test.ts (304 lines, 32 tests): GIT_SSRF_FLAGS exact
shape, all parseRemoteUrl rejection cases including dedicated CGNAT
100.64/10 with/without GBRAIN_ALLOW_PRIVATE_REMOTES (codex T3 case),
fake-git harness for argv assertions on cloneRepo/pullRepo, all 6
validateRepoState branches.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(core): add scope hierarchy + ALLOWED_SCOPES allowlist
New src/core/scope.ts (~120 lines) for v0.28's scoped MCP feature.
Hierarchy:
- admin implies all (escape hatch)
- write implies read
- sources_admin and users_admin are siblings (different axes —
sources-mgmt vs user-account-mgmt; neither implies the other)
Exported:
- hasScope(grantedScopes, requiredScope): the canonical scope check.
Replaces exact-string-match at three call sites in upcoming commits
(serve-http.ts:673, oauth-provider.ts:365 F3 refresh, oauth-provider.ts:498
token issuance). Without this rewrite, an admin-grant token would
fail to refresh down to sources_admin (codex finding).
- ALLOWED_SCOPES set + ALLOWED_SCOPES_LIST sorted array (deterministic
for OAuth metadata wire format and drift-check output).
- assertAllowedScopes / InvalidScopeError: registration-time gate so
tokens with bogus scope strings (read flying-unicorn) get rejected
with RFC 6749 §5.2 invalid_scope at auth.ts:296 + DCR /register +
registerClientManual. Today's behavior accepts any string silently.
- parseScopeString: space-separated wire format → array.
Forward-compat: hasScope ignores unknown granted scopes rather than
throwing, so pre-allowlist tokens with weird scope strings continue
working without crashes (registration is the gate, runtime is best-effort).
test/scope.test.ts (178 lines, 35 tests): hierarchy table including
all-implies for admin, sibling non-implication of *_admin scopes,
write→read but not the reverse, F3 refresh-token subset semantics
under hasScope, ALLOWED_SCOPES_LIST sorted-pinning, allowlist
rejection cases, parseScopeString edge cases (undefined/null/empty).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* build(admin): scope-constants mirror + drift CI for src/core/scope.ts
The admin React SPA's tsconfig.json scopes include: ['src'] to admin/src/,
so it cannot directly import ../../src/core/scope.ts. The plan considered
widening the include or generating a single source of truth; both options
either couple the SPA to the gbrain monorepo or add a build step. Eng
review picked the boring choice: hand-maintained mirror at
admin/src/lib/scope-constants.ts plus a CI drift check.
Files:
- admin/src/lib/scope-constants.ts: hand-maintained ALLOWED_SCOPES_LIST
duplicate, sorted alphabetically to match src/core/scope.ts.
- scripts/check-admin-scope-drift.sh: extracts the list from each file
via awk, normalizes via tr/sort, diffs. Exits 0 on match, 1 on drift
(with full breakdown of which scopes diverged), 2 on internal error.
Tested both passing and corrupted paths.
- package.json: wires check:admin-scope-drift into both `verify` and
`check:all` so any update to src/core/scope.ts that forgets the
admin-side mirror fails the build.
The Agents.tsx scope-checkbox sites (5 hardcoded locations) get updated
in a later commit to import from this constants file.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(oauth): hasScope hierarchy + ALLOWED_SCOPES allowlist at registration
Switch three call sites in oauth-provider.ts from exact-string-match to
hasScope() so the v0.28 sources_admin and users_admin scopes — and the
admin-implies-all + write-implies-read hierarchy in src/core/scope.ts —
work end to end:
- F3 refresh-token subset enforcement at line 365: previously rejected
admin → sources_admin refresh because exact-match treated them as
unrelated scopes. gstack /setup-gbrain Path 4 needs admin tokens to
refresh down to least-privilege sources_admin scope; this fix lands
that path.
- Token issuance intersection at line 498 (client_credentials grant):
same hasScope swap so a client whose stored grant is `admin` can mint
tokens including any implied scope.
- registerClient (DCR /register) and registerClientManual: validate
every scope string against ALLOWED_SCOPES via assertAllowedScopes.
Pre-fix the system silently accepted `--scopes "read flying-unicorn"`
and persisted the bogus string in oauth_clients.scope. Post-fix the
caller gets RFC 6749 §5.2 invalid_scope. Existing rows with
pre-allowlist scopes keep working (allowlist gates registration only).
Tests amended in test/oauth.test.ts:
- T1 (eng-review): admin grant CAN refresh down to sources_admin
- T1 sibling: write grant CANNOT refresh up to sources_admin
- ALLOWED_SCOPES allowlist coverage (manual + DCR paths, all 5 valid)
- Scope-annotation contract tests widened to accept the v0.28 union
62 OAuth tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(serve-http): hasScope at /mcp + advertise full ALLOWED_SCOPES
Two changes against src/commands/serve-http.ts:
- Line 195: scopesSupported on the mcpAuthRouter options switches from the
hardcoded ['read','write','admin'] to Array.from(ALLOWED_SCOPES_LIST).
Without this, /.well-known/oauth-authorization-server keeps reporting
the old triple, so MCP clients (Claude Desktop, ChatGPT, Perplexity)
cannot discover the v0.28 sources_admin and users_admin scopes via
standard discovery — they would have to be pre-configured out of band.
- Line 673: request-time scope check on /mcp swaps
authInfo.scopes.includes(requiredScope) for hasScope(...). This was
the most-cited codex finding: without it, sources_admin tokens could
not even satisfy a `read`-scoped op (sources_admin doesn't include
the literal string "read"). hasScope routes through the hierarchy
table in src/core/scope.ts so admin implies all and write implies
read at the gate too.
T2 amendment in test/e2e/serve-http-oauth.test.ts: assert
/.well-known/oauth-authorization-server includes all 5 scopes in
scopes_supported. Pre-v0.28 the list was hardcoded to ['read','write',
'admin'] and this assertion would have failed. (The test is
Postgres-gated; runs under bun run test:e2e with DATABASE_URL set.)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(core): sources-ops module — atomic clone + symlink-safe cleanup
src/core/sources-ops.ts (~470 lines): pure async functions extracted from
src/commands/sources.ts so the CLI handlers and the new MCP ops share
one implementation.
addSource: D3 atomicity contract from the eng review.
1. Validate id (matches existing SOURCE_ID_RE).
2. Q4 pre-flight SELECT — fail loudly with structured `source_id_taken`
before any clone work. Pre-fix the existing CLI used INSERT…ON
CONFLICT DO NOTHING which silently no-op'd; with clone-first that
would orphan the temp dir.
3. parseRemoteUrl gate (delegates to isInternalUrl from url-safety.ts).
4. Clone into $GBRAIN_HOME/clones/.tmp/<id>-<rand>/ via the new
git-remote helpers.
5. INSERT row with local_path=<final clone dir>, config.remote_url=<url>.
6. fs.renameSync(tmp/, final/). Rollback on either-side failure unlinks
the temp dir; rename-failed path also DELETEs the just-INSERTed row
best-effort.
removeSource: clone-cleanup with realpath+lstat confinement matching
validateUploadPath() shape at src/core/operations.ts:61. String startsWith
is symlink-unsafe and would let $GBRAIN_HOME/clones/<id> → /etc resolve
out of the confine. Two defenses layered:
- isPathContained (realpath-resolves both sides + parent-with-sep
string check) rejects symlinks whose target falls outside the
confine.
- lstat-then-isSymbolicLink check refuses symlinks whose realpath
happens to land back inside the confine (defense in depth).
getSourceStatus: returns clone_state via validateRepoState (the 6-state
decision tree from git-remote.ts). Lets a remote MCP caller diagnose
"healthy | missing | not-a-dir | no-git | url-drift | corrupted" without
SSH access to the brain host. listSources additionally exposes
remote_url so callers can see which sources are auto-managed.
recloneIfMissing: T4 follow-up for `gbrain sources restore` after the
clone dir was autopurged — re-clones via the same temp + rename
atomicity contract. Idempotent (returns false when clone is already
healthy).
test/sources-ops.test.ts (~470 lines, 24 tests): pre-flight collision
(Q4), happy paths for both --path and --url, all four D3 rollback paths
(clone-fail before INSERT, INSERT-fail after clone, rename-fail
post-INSERT, atomic temp-dir cleanup), symlink-target-OUTSIDE-clones
(realpath confinement), symlink-target-INSIDE-clones (lstat-check),
removeSource refuses to delete user-supplied paths, refuses "default"
source, getSourceStatus clone_state branches, T4 recloneIfMissing
recovery + idempotent + no-op for path-only sources, isPathContained
unit tests covering subtree / outside / symlink-escape / fail-closed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(operations): whoami + sources_{add,list,remove,status} MCP ops
Five new ops in src/core/operations.ts auto-flow through src/mcp/tool-defs.ts
so MCP clients (Claude Desktop, ChatGPT, Perplexity, OpenClaw) get them via
standard tools/list discovery — no SDK or transport code changes needed.
Operation.scope union widened to add 'sources_admin' and 'users_admin' (the
v0.28 hierarchy from src/core/scope.ts).
whoami (scope: read): introspect calling identity over MCP.
- Returns `{transport: 'oauth', client_id, client_name, scopes, expires_at}`
for OAuth clients (clientId starts with gbrain_cl_).
- Returns `{transport: 'legacy', token_name, scopes, expires_at: null}`
for grandfathered access_tokens.
- Returns `{transport: 'local', scopes: []}` when ctx.remote === false.
Empty scopes (NOT ['read','write','admin']) is the D2 decision —
returning OAuth-shaped scopes for local callers would resurrect the
v0.26.9 footgun where code conditionally trusted on
`auth.scopes.includes('admin')` instead of `ctx.remote === false`.
- Q3 fail-closed: throws unknown_transport when remote=true AND auth is
missing OR ctx.remote is the literal `undefined` (cast bypass guard).
A future transport that forgets to thread auth doesn't get a free
pass.
sources_add (sources_admin, mutating): register a source by --path
(existing v0.17 behavior) or --url (v0.28 federated remote-clone path).
Calls into addSource from sources-ops.ts which owns the temp-dir +
rename atomicity.
sources_list (read): list registered sources with page counts, federated
flag, and remote_url. The remote_url field is new — lets a remote MCP
caller see which sources are auto-managed.
sources_remove (sources_admin, mutating): cascade-delete a source +
symlink-safe clone cleanup. Requires confirm_destructive: true when the
source has data.
sources_status (read): per-source diagnostic returning clone_state
('healthy' | 'missing' | 'not-a-dir' | 'no-git' | 'url-drift' |
'corrupted' | 'not-applicable') — lets a remote MCP caller diagnose a
busted clone without SSH access to the brain host.
test/whoami.test.ts (9 tests): pinned transport-detection for all four
return shapes including Q3 fail-closed throw under both auth=undefined
and remote=undefined cast-bypass paths.
test/sources-mcp.test.ts (16 tests): op-metadata pins (scope, mutating,
localOnly), functional handler shape against PGLite, hasScope-driven
scope-enforcement smoke test simulating the serve-http.ts:673 gate
(read-only token rejected for sources_add; sources_admin token allowed;
admin token allowed for everything; gstack /setup-gbrain Path 4 token
covers all 4 ops), SSRF gate at the op layer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(sync): re-clone fallback when clone is missing/no-git/corrupted
src/commands/sync.ts gets a v0.28-aware front-half. When the source has
config.remote_url, performSync calls validateRepoState before the existing
fast-forward pull path:
- 'healthy' → fall through to existing pull (unchanged)
- 'missing' → loud stderr "auto-recovery: re-cloning <id>", then
'no-git' recloneIfMissing handles the temp-dir + rename. Sync
'not-a-dir' continues from the freshly-cloned head.
- 'corrupted' → throw with structured hint pointing at sources remove
+ add (no syncing wrong state).
- 'url-drift' → throw with hint pointing at the (deferred) sources
rebase-clone command.
Closes the operator-confidence gap: rm -rf $GBRAIN_HOME/clones/<id>/ no
longer breaks future syncs. The next sync sees the missing dir and
recovers via the recorded URL.
src/core/operations.ts: extend ErrorCode with 'unknown_transport' so
whoami's Q3 fail-closed path types check.
test/sources-resync-recovery.test.ts (12 tests): full validateRepoState
state matrix exercised under fake-git, recloneIfMissing recovery from
each degraded state, idempotent on healthy clones, the sync.ts:320
integration path that drives the recovery.
test/sources-ops.test.ts + test/sources-mcp.test.ts: drop the
GBRAIN_PGLITE_SNAPSHOT-disable line so these tests stop forcing cold
init across the parallel-shard runner. With snapshot allowed, init time
drops from 6+s to ~50ms and parallel runs stay under the 5s hook
timeout.
test/sources-mcp.test.ts: tighten scope literal-type so tsc keeps the
union narrow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(cli): sources add --url + restore re-clone, thin-wrapper refactor
src/commands/sources.ts now delegates the data-mutation work to
src/core/sources-ops.ts (added in the previous commit). The CLI handler
parses argv, calls into addSource, and formats output.
Two new flags on `gbrain sources add`:
- `--url <https-url>` : federated remote-clone path (clone + INSERT +
rename, atomic rollback on failure).
- `--clone-dir <path>` : override the default
$GBRAIN_HOME/clones/<id>/ destination.
Validation rejects mutually-exclusive `--url` + `--path`. Errors from
the ops layer (SourceOpError) propagate through the CLI's standard
error wrapper in src/cli.ts so existing tests that assert throw shape
keep passing.
`gbrain sources restore <id>` (T4 from eng review): if the source has a
remote_url AND the on-disk clone was autopurged, call recloneIfMissing
before declaring success. Clone errors print a WARN with recovery
hints rather than failing the restore — the DB row is what restore
guarantees; the clone is best-effort.
54 sources-related tests pass (existing test/sources.test.ts +
sources-ops + sources-mcp).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(doctor,cycle): orphan-clones surface + autopilot purge phase (P1)
addSource's atomicity contract uses a temp dir that gets renamed to the
final clone path. If the process is SIGKILL'd between clone-finish and
rename, the temp dir orphans on disk. Without sweeping these, a brain
server accumulates gigabytes over months of failed `sources add --url`
attempts.
Two layers:
1. `gbrain doctor` now surfaces stale entries. A new orphan_clones check
walks $GBRAIN_HOME/clones/.tmp/, names anything older than 24h, and
prints a warn with disk-byte estimate. Operators see the leak before
`df` complains.
2. The autopilot cycle's existing `purge` phase grows a substep that
nukes .tmp/ entries past the same 72h TTL the page-soft-delete purge
uses. Operator behavior stays uniform across all soft-delete-style
surfaces.
Both layers are filesystem-only (no DB). On a brain that never used
--url cloning, both are no-ops.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* build(admin): scope checkboxes source from scope-constants mirror + dist
admin/src/pages/Agents.tsx Register Client modal:
- useState default sources from ALLOWED_SCOPES_LIST (defaulting `read`
to true, others false; unchanged UX for the common case).
- Scope checkbox map iterates ALLOWED_SCOPES_LIST instead of the old
hardcoded ['read','write','admin'].
Without this commit, even with the v0.28.1 server-side scope hierarchy,
operators registering an OAuth client from the admin UI cannot tick the
new sources_admin / users_admin scopes — defeats the whole gstack
/setup-gbrain Path 4 unblock.
The drift-check CI gate (scripts/check-admin-scope-drift.sh) ensures
this list stays in sync with src/core/scope.ts going forward.
admin/dist/* rebuilt via `cd admin && bun run build`. Old hash bundle
removed; new bundle (224.96 kB / 68.70 kB gzip).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: v0.28.1 — remote-source MCP + scope hierarchy + whoami
VERSION + package.json: bump to 0.28.1 (per CLAUDE.md branch-scoped
versioning rule — this branch adds substantial new features on top of
v0.28.0).
CHANGELOG.md: new top-level entry for v0.28.1 in the gstack/Garry voice
(no AI vocabulary, no em dashes, real numbers + commands). Lead
paragraph names what the user can now do that they couldn't before.
"Numbers that matter" table calls out the +5 MCP ops, +2 OAuth scopes,
and the 4-to-0 SSH-step number for gstack /setup-gbrain Path 4. "What
this means for you" closer ties the work to the operator workflow shift.
"To take advantage of v0.28.1" block has paste-ready upgrade commands
including the admin SPA rebuild step. Itemized changes section
describes the architecture cleanly without exposing scope-string
internals to public attack-surface enumeration (per CLAUDE.md
responsible-disclosure rule).
TODOS.md: file 6 follow-ups under a new "Remote-source MCP follow-ups
(v0.28.1)" section: token rotation, migration introspection in
get_health, Accept-header friendliness, sources rebase-clone for
URL-drift recovery, --filter=blob:none partial-clone option, and the
chunker_version PGLite-schema parity codex caught.
README.md: short subsection under the existing sources CLI listing
that names the new --url flag and what auto-recovery does. Capability
framing (no scope-string enumeration).
llms.txt + llms-full.txt: regenerated via `bun run build:llms` so the
documentation bundle reflects the v0.28.1 entry. The build-llms
generator's drift check passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(e2e): sources-remote-mcp — full gstack /setup-gbrain Path 4 round-trip
Spins up `gbrain serve --http` against real Postgres with a fake-git binary
in PATH (so `git clone` is exercised end-to-end without network), registers
two OAuth clients (sources_admin + read-only), mints tokens, calls the new
v0.28.1 MCP ops via /mcp, and asserts the gstack /setup-gbrain Path 4 flow
works end to end.
12 tests cover the full lifecycle:
- whoami over HTTP MCP returns transport=oauth + the right scopes
- /.well-known/oauth-authorization-server advertises all 5 scopes
- sources_add: clone fires, INSERT lands, row carries config.remote_url
- sources_status: clone_state=healthy after add
- sources_list: surfaces remote_url for the new source
- SSRF rejection: sources_add with RFC1918 URL fails at parseRemoteUrl gate
- Scope enforcement: read-only token gets insufficient_scope on sources_add
- Read-only token CAN call sources_list (read-scoped op)
- ALLOWED_SCOPES allowlist: CLI register-client rejects bogus scope
- Recovery: rm clone dir + sources_status reports clone_state=missing
- sources_remove: cascades + cleans up the auto-managed clone dir
Subprocess env threading replicates the v0.26.2 bun execSync inheritance
pattern — bun does NOT inherit process.env mutations, so every CLI
subprocess call passes env: { ...process.env } explicitly.
Cleanup contract mirrors test/e2e/serve-http-oauth.test.ts: revoke any
clients we registered, force-kill the server subprocess on SIGTERM
timeout, surface cleanup failures to stderr without throwing so real
test failures aren't masked.
The base table list in helpers.ts (ALL_TABLES) doesn't include sources
or oauth_clients, so this test explicitly truncates them in beforeAll
to avoid Q4 pre-flight collisions on re-run.
Skipped gracefully when DATABASE_URL is unset.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: codex adversarial review — confine remote sources_admin + close SSRF gaps
Pre-ship adversarial review (codex exec) caught five issues. Four ship in
this commit; the fifth (DNS rebinding) is filed as v0.28.x follow-up.
CRITICAL — `sources_admin` tokens over HTTP MCP could plant content at any
host path. The MCP op exposed `path` and `clone_dir` to remote callers; the
op layer trusted them verbatim, then auto-recovery's rm -rf on degraded
state turned that into arbitrary delete primitives. src/core/operations.ts
sources_add handler now drops both fields when ctx.remote !== false. Local
CLI keeps the override (operator trust). Loud logger.warn when a remote
caller tries — visible in the SSE feed without leaking values.
HIGH — Steady-state `git pull --ff-only` bypassed GIT_SSRF_FLAGS entirely.
The legacy helper at src/commands/sync.ts:192 spawned git without the
-c http.followRedirects=false -c protocol.{file,ext}.allow=never
--no-recurse-submodules set that cloneRepo applies. Every recurring sync
was reopening the redirect/submodule/protocol bypass. Routed the call site
at sync.ts:381 through pullRepo from git-remote.ts so initial clone and
ongoing pull share one defensive flag set.
MEDIUM — listSources ignored its `include_archived` flag. The op
advertised the param but the function destructured it as `_opts` and
queried every row. Archived sources' ids, local_paths, and remote_urls
were leaking to read-scoped MCP callers by default. Filter in SQL
(`WHERE archived IS NOT TRUE` unless the flag is set) so archived rows
never reach the wire.
PARTIAL HIGH — IPv6 ULA fc00::/7 and link-local fe80::/10 were not in
the isInternalUrl bypass list. Only ::1/:: and IPv4-mapped IPv6 were
blocked. Added regex-based ULA + link-local rejection to url-safety.ts.
Test coverage:
- test/git-remote.test.ts: 4 new IPv6 cases (ULA fc-prefix + fd-prefix,
link-local fe80::, public IPv6 still allowed).
- test/sources-mcp.test.ts: 3 new cases pinning the remote/local
asymmetry (clone_dir override silently ignored over MCP, path nulled,
local CLI keeps the override).
- test/sources-mcp.test.ts: 2 new cases for include_archived honored.
DNS rebinding (codex finding #3): the current gate is lexical only.
A deliberate attacker who controls a hostname's A/AAAA records can still
resolve to an internal IP. Closing this requires async DNS resolution +
revalidation; filed as v0.28.x follow-up in TODOS.md so the API change
surface (parseRemoteUrl becomes async, every caller updates) lands in
its own PR.
323 tests pass (9 files); 4071 unit tests pass (full suite).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: rebump v0.28.1 → v0.28.2 (master collision)
Caught after PR creation. master is at v0.28.1 already; this branch
forked from garrytan/v0.28-release at v0.28.0 and naively bumped to
v0.28.1 without checking the master queue. CI version-gate would have
rejected at merge time (requires VERSION strictly greater than
master's).
Root cause: I bumped VERSION mechanically during plan implementation
(echo "0.28.1" > VERSION) without consulting the queue-aware allocator
at bin/gstack-next-version. /ship Step 12's idempotency check then
classified state as ALREADY_BUMPED and the workflow's "queue drift"
comparison was the safety net I should have hit — but I skipped it.
Files updated:
- VERSION + package.json: 0.28.1 → 0.28.2
- CHANGELOG.md: header + "To take advantage of v0.28.2" subsection
- README.md: sources --url note version reference
- TODOS.md: 7 follow-up entries' version references
- llms.txt + llms-full.txt: regenerated
PR title rewrite via gstack-pr-title-rewrite.sh handled in a separate
gh pr edit call; CI version-gate now passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(todos): close longmemeval-publication, file 4 follow-up TODOs
Full 500-question 4-adapter LongMemEval _s benchmark landed at
github.com/garrytan/gbrain-evals#main:ced01f0. gbrain-hybrid 97.60% R@5,
+1.0pt over MemPal raw 96.6%. Replacing the now-stale "needs full run"
TODO with closure + 4 grounded follow-ups:
1. Timeline-aware retrieval signal for temporal-reasoning questions
(P2 — closes the only category we lose to MemPal-raw)
2. Per-question batch consolidation for ~10x cold-cache speedup
(P3 — makes daily benchmark CI gate practical)
3. LongMemEval _m split run (P3 — differentiated, not yet published
by MemPal)
4. Cheaper-embedding-model recipe (P4 — recall-cost tradeoff curve)
Each TODO has the standard What/Why/Pros/Cons/Context/Depends-on shape per
the gbrain TODOS-format convention.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(llms): regenerate llms-full.txt to match merged CLAUDE.md
CI test/build-llms.test.ts asserts the committed llms.txt/llms-full.txt
are byte-for-byte identical to what scripts/build-llms.ts produces. The
master merge brought in v0.28.9/v0.28.10/v0.28.11 + multimodal embedding
notes that updated CLAUDE.md; the bundle was stale.
No content changes. Pure regeneration via `bun run build:llms`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(changelog): rewrite v0.28.12 entry — lead with the LongMemEval result
Old entry buried the headline ("LongMemEval lands in the box…") under
process detail (hermetic CI test count, 25.9ms p50, schema-table
runtime enumeration). The reader cares what gbrain DOES — not how we
plumbed the harness.
New entry leads with the actual number — 97.60% R@5 on the public
LongMemEval _s split, beating MemPalace raw by 1.0pt — followed by
the per-category win table that proves gbrain ties or beats MemPal in
5 of 6 question types and shows the +7.1pt assistant-voice lift.
Links to the full gbrain-evals report (97.60% headline + full
methodology + reproducible runner) so curious readers can dig deeper.
Two honest findings published in plain text: vector-only is
essentially tied with hybrid at K=5, and query expansion via Haiku is
a clean null result on this dataset. Better to publish the null than
hide it.
Reproduction block updated to match the actual gbrain-evals workflow
(clone + bun install + dataset download + bash batch runner). The
prior "download / run / hand to evaluate_qa.py" block stayed for the
in-tree CLI path.
Regenerated llms-full.txt to keep the build-llms regen-drift guard
green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
111 KiB
TODOS
LongMemEval benchmark follow-ups (v0.28.12)
Closed: full 500-question 4-adapter run published
The full 500-question, 4-adapter LongMemEval _s benchmark landed in
gbrain-evals#main:ced01f0.
gbrain-hybrid: 97.60% R@5, beating MemPal raw 96.6% by 1.0pt on the same
dataset, K, and n with no LLM in the retrieval loop. Honest null result on
query expansion (97.60% with vs without). Closing this entry; remaining
follow-ups below.
Timeline-aware retrieval signal for temporal-reasoning questions
Priority: P2
What: gbrain's links table + gbrain extract timeline already build a
graph of dated events. Feed that signal into searchKeyword / searchVector
ranking so questions like "what was the FIRST issue I had after my new
car's first service?" get a temporal boost on session ordering.
Why: LongMemEval temporal-reasoning is the only question type where MemPal-raw beats gbrain-hybrid (96.2% vs 94.7%, -1.5pt). Embeddings carry topic similarity; "first" / "before" / "last week" need ordering signal that vector cosine doesn't surface. We have the data infrastructure to fix this (the timeline extraction code), just don't pipe it into search ranking.
Pros: Closes the only categorical loss to MemPal on the public benchmark. Generalizes beyond LongMemEval — every personal-knowledge agent gets temporal questions and most fail them. This is a structural advantage.
Cons: Requires a new SQL ranking factor in src/core/search/sql-ranking.ts
and signal-extraction work in the query-time path (parsing temporal hints
from the question). Maybe ~200 lines + a benchmark line on the gbrain-evals
report once it ships.
Context: Per-type breakdown in
gbrain-evals/docs/benchmarks/2026-05-07-longmemeval-s.md shows we tie
or beat MemPal-raw on 5 of 6 types and lose temporal by 1.5pt. Also:
src/core/link-extraction.ts already extracts dated timeline entries via
parseTimelineEntries. They land in timeline_entries table but aren't
used during retrieval ranking.
Depends on: Nothing blocking.
Per-question batch consolidation (latency optimization)
Priority: P3
What: importFromContent calls embedBatch once per page. Each LongMemEval
question imports ~50 sessions = 50 separate API calls. Pre-chunk all sessions
for a question, embed in one OpenAI call, then bulk-write.
Why: Drops per-question latency from ~14s to ~3s on a cold cache. Currently the runner ships a 700MB SQLite warm-cache to avoid this; a faster cold path would let CI run the benchmark daily without a fixture.
Pros: Daily benchmark CI gate becomes practical. Cuts cold-cache cost by ~10x. Faster iteration when tuning ranking parameters.
Cons: ~80 lines of batch-consolidation code that lives in the runner, not
gbrain core. Touches eval/runner/longmemeval.ts:run() per-question loop.
Less generalizable than the timeline-aware ranker work.
Context: Right now the warm-cache mitigates this in practice (subsequent runs are sub-1-min). The optimization matters only when re-running with a different gbrain version that re-keys the cache.
Depends on: Nothing blocking.
LongMemEval _m split (200 distractor sessions per haystack)
Priority: P3
What: Run the existing 4-adapter benchmark against the harder _m split
where each haystack has ~200 distractor sessions instead of ~50.
Why: Pushes retrieval into the regime where gbrain's pipeline either
holds up or doesn't. MemPal hasn't published _m numbers; we'd have a
clean head-to-head once we run it. Also stresses the noise-rejection
(source-boost / hard-exclude) layer of gbrain harder than _s does.
Pros: Differentiated benchmark line. Forces signal-vs-noise behavior we
can't measure on _s. Free with our existing runner.
Cons: ~$10-20 in OpenAI embeddings (4x more chunks per question). Cache file grows to ~3GB. ~6-8 hours wall time for the embedding-heavy runs even parallel-3.
Depends on: Nothing blocking. Could ship same shape as _s report.
Cheaper embedding-model recipe for benchmarks
Priority: P4
What: Pin text-embedding-3-small (or Voyage-3-lite via the v0.27
pluggable provider stack) as a benchmark-only embedding model so the
cold-cache cost drops 10x. Compare recall against text-embedding-3-large
and publish the recall-cost tradeoff curve.
Why: "What's the cheapest embedding model that still wins this benchmark?" is a real builder question. We'd publish the answer.
Pros: Useful tradeoff line for users picking gbrain in a cost-sensitive deployment. Validates the v0.27 pluggable-provider work end-to-end.
Cons: Multiple full-benchmark runs ($30+ in API spend) to chart the curve.
Depends on: v0.27 pluggable embedding provider work (already shipped,
verify Voyage adapter integration in src/core/ai/recipes/voyage.ts).
multimodal embedding follow-ups (v0.28.11 / PR #719)
gbrain doctor: warn on misconfigured multimodal model
Priority: P2
What: Add two checks in src/commands/doctor.ts. (1) When embedding_multimodal_model is set, verify the recipe's required API key is present in the env. (2) When embedding_multimodal: true is set but no embedding_multimodal_model AND the primary embedding_model recipe doesn't declare supports_multimodal, surface that gap.
Why: Today these misconfigurations surface only on first image ingest, after the user has already pushed image content into the brain. Doctor catching them at install/upgrade time saves a round of confusion.
Pros: Both checks are read-only and cheap (one env probe + one recipe lookup). Same pattern as existing doctor checks. Surfaces problems before they ship.
Cons: Doctor's check list grows; needs a --fast opt-out path if added to the default scan. ~40 lines.
Context: PR #719 added the multimodal_model routing key. The recipe-level + model-level validation in embedMultimodal() already throws clear errors at runtime, but only when image content hits the gateway. v0.28.x candidate.
Depends on: None.
Reclassify Voyage HTTP 4xx as AIConfigError (Codex F2 from PR #719 review)
Priority: P2
What: src/core/ai/gateway.ts:626 currently throws AITransientError for any non-401/403 4xx response from Voyage's /multimodalembeddings endpoint. Replace with a 4xx-non-429 → AIConfigError branch matching normalizeAIError's contract at src/core/ai/errors.ts:54.
Why: A config bug (malformed body, unsupported field, model the caller forgot to add to multimodal_models) currently presents to the caller as transient and triggers retry storms. PR #719's Change 3 closes the specific wrong-multimodal-model case locally via the multimodal_models allow-list, but other 4xx reasons still misclassify.
Pros: Aligns the embedMultimodal error classifier with normalizeAIError. Eliminates retry-on-permanent-bug behavior. ~10 lines + 1 test.
Cons: Changes runtime error class for some failures; existing callers that catch AITransientError for these codes now must catch AIConfigError. Search before merging.
Context: Pre-existing in v0.27.1; surfaced because PR #719's new key makes the misclass more reachable. v0.28.x candidate.
Depends on: None.
gbrain config unset <key> subcommand (Codex F6 from PR #719 review)
Priority: P3
What: Add unset action alongside show|get|set in src/commands/config.ts. Calls engine.setConfig(key, '') (loadConfigWithEngine treats empty string as undefined) so a user who set a key by mistake can clear it. Empty-string write is the minimum-diff implementation; a real DELETE would be cleaner if the engine grows one.
Why: Once a user runs gbrain config set X val, there's no normal CLI path to clear it. Empty string is rejected by the current set validator (action === 'set' && key && value where value is truthy). PR #719 added another DB-merge key (embedding_multimodal_model) and surfaces this UX gap.
Pros: Closes a pre-existing UX hole that applies to every DB-merge key (embedding_multimodal, embedding_image_ocr*, now embedding_multimodal_model). Trivial implementation, ~15 lines.
Cons: Need to decide whether unset is a real DELETE (cleaner) or empty-string write (simpler).
Context: Pre-existing in v0.27.x. Worth doing alongside the doctor checks above so users have a working escape hatch.
Depends on: None.
cross-modal-eval (v0.27.x follow-ups from PR #674 plan)
--budget-usd hard cap + per-call cost telemetry (T11=B follow-up)
Priority: P2
What: gbrain eval cross-modal ships in v0.27.x with a partial cost guardrail: default --cycles 1 in non-TTY plus a stderr cost-estimate printed before each run. The full --budget-usd N hard cap (refuse to start the next cycle if estimated spend would exceed) and per-call actual-cost telemetry written into the receipt are intentionally deferred.
Why: Codex pushback on the original P2=B "defer everything" decision was right — even with >=2/3 success required for a verdict (Q3=A), 3 cycles × 3 calls = 9 frontier calls per run, repeated across N skills if anyone scripts a bulk audit. The TTY/non-TTY cycle default catches the worst case; the hard cap catches the next class of mistakes.
Pros: Deterministic spend ceiling. Real per-call cost in the receipt drives a feedback loop that lets us refine the price-table constant in src/core/cross-modal-eval/runner.ts:estimateCost. Future bulk-audit integrations get a safety net by default.
Cons: ~80 lines of pricing-table + parsing + threading. Pricing values drift; the file becomes a small maintenance burden between model-family bumps.
Context: Pricing table lives at src/core/cross-modal-eval/runner.ts:estimateCost. Once we have real telemetry from a few weeks of usage, we can switch the table to "last observed" instead of "list price" and get more accurate caps. v0.27.x candidate.
Depends on: Nothing.
Subagent integration (recovers cross-process rate-leases — T4 deferred)
Priority: P2
What: Wire gbrain eval cross-modal to be invokable as a gbrain agent run child job. Today the CLI runs synchronously and bypasses src/core/minions/rate-leases.ts because the lease helper requires a minion_jobs.id that the CLI path doesn't have (T4=A in plans/radiant-napping-lerdorf.md).
Why: Cross-process concurrency cap. A user running gbrain eval cross-modal in one terminal alongside gbrain agent run in another can hit Anthropic 429s due to combined load. As a minion job, the eval gets the rate-lease behavior for free, plus stagger / quiet-hours / retry surface from the existing Minions queue.
Pros: No new helper API; reuses what's already there. Closes the cross-process gap that today's Promise.allSettled design intentionally leaves open.
Cons: Requires a job handler registration + receipt-path threading through job context. Probably ~150 lines plus tests. Behavior parity (verdict / receipt shape) needs to be pinned with a parametrized test.
Context: Pattern is the same as src/core/minions/handlers/subagent.ts. v0.27.x candidate.
Depends on: Nothing.
Skill adoption telemetry (revisit T7=C with data)
Priority: P3
What: Track how many skills land cross-modal eval receipts. If adoption stalls at, say, <30% of skills after 30 days, consider flipping the 11th item from required:false (T7=C, current) to required:true (T7=A) in v0.28.x.
Why: T7=C ships the gate as informational so existing audits don't regress. The forcing function is documentation alone. We don't yet know if that's enough.
Pros: Data-driven decision instead of guessing. Lightweight: count receipt files in gbrainPath('eval-receipts') against the count of skills under skills/*/SKILL.md.
Cons: "Adoption stalled" is a judgment call without a baseline. Could become a debate.
Context: New check in gbrain doctor would surface the count. v0.28.x candidate.
Depends on: None.
docs/cross-modal-eval.md user guide
Priority: P3
What: Add a user-facing guide. Cover the gateway-config flow, receipt forensics, the <slug>-<sha8>.json filename convention, default models + how to override them, the relationship to skills/cross-modal-review/SKILL.md, and worked examples on a real skill.
Why: SKILL.md teaches the workflow but lives under skills/skillify/. CLAUDE.md "Key files" entries are agent-facing, not human-facing. A docs/cross-modal-eval.md is the natural home for "I'm a user, how do I use this command?" answers.
Pros: Discoverable from CLAUDE.md "Key files" reference. Mirrors docs/eval-bench.md precedent.
Cons: Doc-write task; ~250 lines of prose.
Context: v0.27.x candidate.
Depends on: None.
/health endpoint hardening (v0.28.1 follow-up)
Cancel engine.getStats() when /health times out
Priority: P2
What: probeHealth() in src/commands/serve-http.ts races engine.getStats() against a 3s timeout. When the timeout wins, the original getStats() keeps running on a saturated pool. Under sustained probe traffic with a slow DB, timed-out probes pile up expensive count(*) queries that turn a partial slowdown into a total outage.
Why: Both adversarial reviewers (Claude + Codex) flagged this independently during the v0.28.1 ship. Deferred because cancellation requires AbortController plumbing through BrainEngine.getStats() which doesn't exist yet — wider blast radius than v0.28.1's zombie-reaping scope justified.
Pros: Closes the self-DoS path. /health returning 503 stops contributing to pool saturation.
Cons: Touches the BrainEngine interface (PostgresEngine + PGLiteEngine implementations). Needs postgres.js or PgBouncer-level query cancellation. Wider blast radius.
Context: Drop-in replacement for Promise.race([getStats(), timeout]) is getStats({ signal }) consumed via AbortController. Reviewer findings: see PR #637 (v0.28.1) adversarial review section.
Depends on: AbortController plumbing in BrainEngine interface.
Replace /health with a lighter liveness probe
Priority: P3
What: engine.getStats() does count(*) FROM pages, content_chunks, links, tags, timeline_entries plus GROUP BY type. On a large but otherwise healthy brain, this can normally exceed 3s and cause false-positive 503s + orchestrator restart loops.
Why: Codex flagged that the new 3s timeout is aggressive for the cost of the probe. Pre-existing behavior (the /health endpoint was already doing full stats in v0.27 with no timeout). Worth splitting probe purpose: /health for liveness (SELECT 1), /stats for the full counts.
Pros: Liveness probe stays under 100ms even on saturated pools. Operators get a separate /stats for the count breakdown when they actually want it.
Cons: Behavior change for orchestrator setups that scrape /health as both liveness AND count source.
Context: PR #637 (v0.28.1) adversarial review. Pair with the AbortController follow-up above.
Remote-source MCP follow-ups (v0.28.2)
Token rotation: gbrain auth rotate <name> + rotate_token MCP op
Priority: P2
What: Atomic rotate for legacy + OAuth tokens. Issue a new token in the same TX as the revocation of the old, no overlap window. Refresh-token rotation already exists for OAuth; this is the unified user-facing surface (CLI + MCP).
Why: Today rotation is revoke + create, with a window where neither token works. For long-lived bearer keys handed to agents, that's a reload outage every time the key gets rotated.
Pros: Single command does the right thing. Atomic cutover. Operators stop scripting around the gap.
Cons: Needs careful testing of the legacy access_tokens UPDATE path (returns single-use new token before the row mutates) plus an MCP op that grants a new token bound to the original client_id without requiring a new authorize round trip.
Context: Item 4 from the gstack /setup-gbrain v1.28.1.0 enhancement request. v0.28.x candidate.
Depends on: Nothing.
Migration introspection in get_health
Priority: P3
What: Extend BrainEngine.getHealth() return shape with migrations: { pending: [...], wedged: [...] }. gbrain doctor already shows this; expose it via the MCP op so remote agents can detect partial-migration state without invoking doctor separately.
Why: Closes a remote-diagnostic gap. gstack /setup-gbrain Path 4 hit a wedged-migration brain mid-session; the only readback was SSH + gbrain doctor. With this, the same diagnostic flows through MCP.
Pros: Pure additive change to the get_health op shape. No new op surface. Consumers ignore the new field if they don't care.
Cons: Wedged detection logic lives in gbrain doctor's code today; need to extract or duplicate. Care needed not to leak migration internals to non-admin scopes (current op is admin-only — fine).
Context: Item 5 from the gstack /setup-gbrain v1.28.1.0 enhancement request.
Depends on: Nothing.
Accept-header friendliness on /mcp
Priority: P3
What: MCP SDK rejects requests missing text/event-stream in the Accept header with a generic 406 Not Acceptable. Pre-check the header at the express middleware layer and return a 400 with a descriptive hint pointing at the spec.
Why: Other MCP clients (curl scripts, custom integrations) hit the SDK's 406 and get no diagnostic. gstack's verify-helper sets both headers correctly so the headline path works.
Pros: Operator UX improvement. Faster debugging when clients fail discovery. Cons: Tight coupling to the SDK behavior — if it later loosens, the pre-check becomes redundant. Context: Item 6 from the gstack /setup-gbrain v1.28.1.0 enhancement request. Depends on: Nothing.
gbrain sources rebase-clone <id>
Priority: P3
What: Recover from url-drift (config.remote_url updated but the on-disk clone still points at the old origin). Currently sync refuses with a structured error pointing at this command — but the command itself doesn't exist yet. Implement: prompt for confirmation (rm-rf the clone is destructive), then re-clone via the same temp-dir + rename atomicity contract as sources add --url.
Why: Closes the loop on the URL-drift code path the v0.28.2 sync added. Without it, operators have to sources remove --confirm-destructive + sources add --url (loses page count, history).
Pros: Cleaner UX for URL changes. Preserves the source row + history.
Cons: Destructive on-disk; needs --confirm-destructive gate. Edge case: what if sync is mid-run when rebase fires? The existing sync-lock guards this, but worth pinning in tests.
Context: v0.28.2 plan filed this explicitly as a follow-up.
Depends on: Nothing.
--filter=blob:none partial-clone option for federated sources
Priority: P3
What: v0.28.2 defaults gbrain sources add --url to --depth=1 (no history). For users who want commit-aware features later (page-state-at-commit-X, blame, who-edited-what), expose --filter=blob:none as an opt-in: keeps full graph metadata, lazy-fetches blobs.
Why: --depth=1 is a one-way door — once cloned, you can't reconstruct history without re-cloning the whole repo. Partial clones preserve history while staying small.
Pros: Forward-compat for commit-aware brain features. Negligible cost on first clone for typical brain repos. Better than the alternative (full clones for everyone).
Cons: First-clone latency is higher on long-history repos. Adds one more flag to the add surface.
Context: Eng review A5 — the boring choice for v0.28.2 was --depth=1. This is the unboring follow-up.
Depends on: Nothing.
DNS rebinding defense for parseRemoteUrl
Priority: P3
What: isInternalUrl (src/core/url-safety.ts) does lexical/string-based classification only — no DNS resolution. An attacker who controls a public hostname's A/AAAA records can resolve to internal IPs (127.0.0.1, 169.254.169.254, RFC 1918) and bypass the SSRF gate. The gate catches direct IP literals + metadata hostnames; it doesn't catch https://attacker-controlled.example/repo.git where DNS points internal.
Why: Defense in depth. The current gate is sufficient for naive abuse (typing 192.168.1.1 directly), but a deliberate attacker with DNS control can bypass it. Adding async DNS resolution + revalidation closes the hole.
Pros: Closes the cleanest remaining SSRF bypass. Mirrors the redirect-revalidation pattern at integrations.ts:289. Pinned by a future test using a mock resolver.
Cons: Async DNS makes parseRemoteUrl async. Every caller (CLI, MCP op, test) needs to update. ~50-line change.
Context: Codex finding from v0.28.2 ship adversarial review. The IPv6 ULA + link-local portion of the same finding shipped in v0.28.2; DNS rebinding deferred.
Depends on: Nothing.
sources.chunker_version PGLite-schema parity
Priority: P3
What: src/schema.sql:33 declares sources.chunker_version and src/commands/sync.ts:253 reads/writes it, but src/core/pglite-schema.ts:28 omits the column. PGLite users hit a schema-mismatch error on the sync write path.
Why: Pre-existing bug surfaced during the v0.28.2 codex review. Not introduced by remote-source work, but adjacent to source-sync code. Worth fixing as a small parity PR before more source-local state lands.
Pros: Closes a quiet schema drift between the two engine implementations. ~10 lines. Cons: Needs a migration entry to add the column to existing PGLite brains. Migration version bump. Context: Codex D5 from v0.28.2 plan review. Depends on: Nothing.
OAuth/MCP hardening (v0.26.7 follow-up)
F11 — auth register-client --redirect-uri flag
Priority: P3
What: gbrain auth register-client always passes [] for redirect URIs; there is no CLI flag to set them. Operators who want to register an authorization_code client without DCR have to hand-edit the database.
Why: Operator UX gap, not a trust-boundary issue. Codex C11 correctly flagged it as scope creep on the v0.26.7 hardening pass — kept out of that PR but worth doing.
Pros: Closes the operator-experience gap. Validates https:// or loopback per RFC 6749 §3.1.2.1 at registration time. Repeatable flag.
Cons: ~30 lines of argv parsing + URL validation. Adds one more flag to the auth register-client surface. Low value relative to the OAuth provider hardening that already shipped.
Context: Eva-brain has the implementation under src/commands/auth.ts:registerClient. Lift verbatim — the localhost/127.0.0.1/::1 exact-match validation is correct; codex spot-check confirmed it does NOT match localhost.evil.com. v0.27 candidate.
Depends on: Nothing.
F13 — gbrain serve --http argv positive-int validator
Priority: P3
What: parseInt(args[idx + 1]) on --port and --token-ttl accepts the next flag as the value if the argument is missing (e.g., --port --token-ttl 100 parses port as NaN → fallback 3131). Negative integers like --port -1 parse to -1, server fails to bind with a confusing error.
Why: Hygiene, not security. Codex C11 flagged as scope creep. Cheap to do later.
Pros: Replaces parseInt(...) || fallback with a parsePositiveIntOption(args, flag, fallback, {max?}) helper that validates the next arg isn't a flag, matches ^[1-9]\d*$, and clamps to a max. Exits 2 with a clear error.
Cons: ~20 lines of helper + threading through serve.ts. Behavior change: previously-silent bad input now exits loud. Probably fine; no consumer relies on the silent fallback.
Context: Eva-brain has the helper at src/commands/serve.ts. v0.27 candidate.
Depends on: Nothing.
destructive-guard (v0.26.5 follow-up)
Adjacent 2 — Storage objects orphan on hard purge
Priority: P2
What: When purgeExpiredSources (sources cascade) or purgeDeletedPages (page-level) deletes rows, the underlying object-storage payloads referenced by files.storage_uri (S3 / Supabase Storage) are NOT torn down. The cascade FK on files.source_id removes the DB row that points at the object; the object itself stays.
Why: Bound today by most brains carrying Files: 0 (operator preview boxes confirm this in the wild). The leak compounds the moment attachments / images / audio start landing — every soft-delete + 72h TTL purge silently abandons object-storage bytes.
Pros: Closes a real data-leak path. Operators stop paying for orphaned bytes. Aligns sources/pages purge with the file lifecycle.
Cons: Storage backend code is non-trivial (S3 vs Supabase vs local-fs paths each have different cleanup APIs). Single-flight delete + retries on 5xx; needs an audit log.
Context: Plan calls this out explicitly in v0.26.5 CEO review (~/.claude/plans/take-a-look-and-gentle-pine.md Adjacent 2). Targets: src/core/storage.ts for the object-storage interface, src/core/destructive-guard.ts purgeExpiredSources for the call site, plus a new sweep in the cycle's purge phase. v0.26.6 candidate.
Depends on: Schema is fine (already has files.storage_uri). Just needs the storage delete plumbing.
Adjacent 3 — sources remove + sources purge race against gbrain sync
Priority: P3
What: gbrain sources remove <id> and the new gbrain sources purge <id> paths don't acquire SYNC_LOCK_ID (the gbrain-sync writer lock from PR #490). If gbrain sync is mid-import for the same source, the parent row can DELETE while sync is INSERTing children, surfacing as a loud FK violation.
Why: Failure mode is loud (FK violation, not data corruption), and the race window is narrow. Worth closing while the destructive surface is touched, not before.
Pros: Single line at the top of runRemove and runPurge. Reuses tryAcquireDbLock(engine, SYNC_LOCK_ID, 5). No design surface.
Cons: Adds an extra "couldn't acquire lock" exit path the operator has to recognize and retry.
Context: Plan calls this out in CEO review Adjacent 3. Targets: src/commands/sources.ts runRemove and runPurge. v0.26.6 candidate. Pattern: try { await fn() } finally { await release() } mirrors the cycle.ts use of the same primitive.
Depends on: Nothing.
Auth revoke-client gets the destructive-guard pattern
Priority: P3
What: gbrain auth revoke-client <client_id> (v0.26.2) lands without an impact preview or --confirm-destructive gate. CASCADE-purges every active token + auth code in one transaction; one stray client_id wipes a production integration.
Why: Lower urgency than sources/pages because operators run this explicitly with a known client_id, not reflexively. But if the v0.26.5 posture is "every destructive surface gets the same gate," this surface should adopt it.
Pros: Posture consistency — every destructive verb in the gbrain CLI follows one pattern. Operators get the impact preview before nuking a production OAuth client.
Cons: Marginal — single-row delete with cascade. The CASCADE is the blast radius, not the verb itself.
Context: Plan flags this in CEO review. Targets: src/commands/auth.ts runRevokeClient (current shape: atomic DELETE...RETURNING with CASCADE on oauth_tokens + oauth_codes). Add an impact preview that counts oauth_tokens and oauth_codes for the client, then gate behind --confirm-destructive.
Depends on: Nothing.
test infra (v0.26.4 follow-up — intra-file parallelism)
Sweep cross-file shared-state contention; enable bun test --concurrent for another 2-3x speedup
Priority: P0 Status: v0.26.7 shipped foundation slice (helpers + lint + mock.module quarantine). v0.26.8 (env sweep) and v0.26.9 (PGLite sweep + codemod + measurement) carry the rest.
What: v0.26.4 shipped file-level parallel fan-out (8 shards) and got bun run test from 18 minutes to ~85s — a 12x speedup. The next layer is intra-file parallelism via Bun's --concurrent flag (or per-test test.concurrent() markers). This requires every test file to be safe under concurrent execution within the same bun test process.
The constraint: when multiple test files load into the same bun process (which is what bun test foo.test.ts bar.test.ts ... does inside a shard), they share module-level state. Three contention surfaces today:
- ~58 PGLiteEngine instantiations across
test/(per codex's grep). Many use module-levellet engine: PGLiteEnginepatterns. Race when multiple test files load and each invokesnew PGLiteEngine().connect({}). (carrying to v0.26.9) - ~40 process.env mutations without restore.
process.env.X = '...'not paired withafterEachcleanup leaks across files in the same process. (carrying to v0.26.8 —withEnvhelper shipped in v0.26.7) 2 top-level(quarantined asmock.module(...)calls intest/core/cycle.test.ts:26andtest/embed.test.ts. Top-level mocks affect every other test file in the same process.*.serial.test.tsin v0.26.7)
The repo already has the right helper: test/helpers/reset-pglite.ts exports resetPgliteState(engine) which is "two orders of magnitude faster" than fresh-engine-per-test (per the helper's own comment). Sweep all PGLite sites to use one shared engine + this reset in beforeEach. Do NOT introduce a freshPglite() allocator — codex correctly flagged that the repo already rejected that direction.
Two flakes already known and quarantined as *.serial.test.ts (run after parallel pass at --max-concurrency=1):
test/brain-registry.serial.test.ts(wasbrain-registry.test.ts)test/reconcile-links.serial.test.ts(wasreconcile-links.test.ts)
After the sweep, both should be fixable and renameable back to plain *.test.ts.
Why:
- 2-3x additional speedup on top of v0.26.4's 12x. Target:
bun run test< 30s on a Mac dev box. - Forces the test architecture to be principled (no shared mutable state across files in the same process).
- The empirical proof point: when
bun run testwas first measured at v0.26.4, two flakes surfaced under cross-file pressure that pass cleanly in isolation. That same pattern WILL surface more flakes if the suite grows. Better to sweep proactively than to keep growing the*.serial.test.tsquarantine.
Pros:
- Real architectural win, not just speed: tests become composable.
- Existing helper (
test/helpers/reset-pglite.ts) already validates the pattern. - Quarantined flakes auto-resolve: rename back to
*.test.tsafter the sweep.
Cons:
- 1-2 weeks of careful refactoring across ~100 test files.
- Some tests genuinely need shared file-wide state (top-level mocks for module-replacement tests). Those stay quarantined as
*.serial.test.tspermanently — but the count should shrink to a known small set, not grow.
Context: v0.26.4 plan considered doing this in scope (Codex Tension #2 = C). After empirical measurement showed --max-concurrency=4 does nothing on tests not marked test.concurrent(), the user chose to ship v0.26.4 as file-level-only and file this as the v0.27+ project. Plan file: ~/.claude/plans/system-instruction-you-are-working-tranquil-ladybug.md. Codex critical findings #2, #3, #6 are all relevant.
Acceptance criteria:
- All ~58 PGLiteEngine sites use shared-engine +
resetPgliteState()inbeforeEach. (v0.26.9) - All ~40
process.envmutations use awithEnv(...)helper that saves + restores. (v0.26.8 — helper shipped v0.26.7) The 2 top-levelDONE in v0.26.7 (quarantined)mock.module()calls scoped tobeforeEach/afterEach, OR the file moves to*.serial.test.ts.- Wrapper passes
--concurrent(or every test marked.concurrent()). (v0.26.9 — codemod withfindrecursive per Codex F3) bun run testruns 5 times consecutively without flakes. (v0.26.9)- Quarantine count
≤10after the sweep (raised from 5 per D15; v0.26.7 added 2, currently 4: brain-registry, reconcile-links, cycle, embed). - Wallclock target:
bun run test≤60s informational (per D9, dropped from <30s after Codex F1: marking only ~92 cheap files concurrent doesn't unblock the heavy 56 PGLite + 49 env files). Pinned config: SHARDS=8, MAX_CONCURRENCY=4, document Mac model. (v0.26.9)
Decisions ledger (v0.26.7 plan): D1 reversed→D16 sliced, D5 quarantine, D6 no helper wrapper, D7 grep+quarantine, D9 ≤60s informational, D10 ESM-cache claim dropped, D11 codemod uses find recursive, D12 lint wired into verify not test, D13 unquarantine attempt dropped, D14 extended grep patterns, D15 cap raised to 10.
Estimated effort: 1-2 weeks of one engineer's focused work. Could parallelize by sub-area (env-mutation sweep is independent of PGLite sweep).
Speed up E2E via Postgres template databases
Priority: P1
What: E2E tests (bun run test:e2e) currently run sequentially in one shared Postgres container, each test file calling initSchema() from scratch (~5-20s each on cold init). Speed-up: build the schema ONCE into a template DB (gbrain_template), then have each test file CREATE DATABASE foo TEMPLATE gbrain_template (~50ms per clone). With per-shard DATABASE_URL overrides, E2E can fan out to N parallel shards too.
Why: Current E2E wallclock is ~5-10 min in CI. Template DB clones could bring that to ~1-2 min. Critical for the inner loop on E2E-bearing PRs (currently a real friction point per /ship workflow).
Sketch:
- Build template DB once via
initSchema()againstgbrain_template. - Per-test-file:
CREATE DATABASE gbrain_test_clone_<n> TEMPLATE gbrain_template(50ms vs 5-20s). - Per-shard isolation via
DATABASE_URLenv override. - Schema-version stamp on the template so it invalidates when
migrate.tschanges. - Cleanup via
DROP DATABASEin afterAll.
Estimated effort: 1-2 days. Filed during v0.26.4 plan as a deferred follow-up (D4 = B).
test infra (v0.26.2 follow-up — pre-existing failures triage)
Fix 22 pre-existing test failures unrelated to OAuth
Priority: P0
What: A bun test run on top of master at v0.26.2 surfaces 22 pre-existing failures across these suites — none touch v0.26.2's diff (oauth-provider.ts, auth.ts, oauth tests). They reproduce on a clean checkout against master:
- 12 cases in
test/e2e/sync.test.ts(Git-to-DB Sync Pipeline) —result.status === 'first_sync'vs actual'synced'state-machine drift; same root cause across all 12. - 3 cases in
test/e2e/multi-source.test.ts(cascade delete + 2 sync routing) — performSync sourceId/local_path resolution. test/e2e/sync-parallel.test.ts(60-file Postgres concurrency=4) — connection-leak probe regression.test/e2e/sync.test.ts--skip-failedstructured summary loop (v0.22.12 #500).test/e2e/dream.test.ts(no --dry-run syncs pages) — runCycle DB write path.test/e2e/cycle.test.ts(live cycle + chunks + lock cleanup).test/e2e/doctor.test.ts(gbrain doctor exits 0 on healthy DB) — possibly related to v0.26.2 schema changes since CHANGELOG mentions extension of doctor checks.test/brain-registry.test.ts(empty/null/undefined id routes to host) — unrelated to OAuth surface.test/e2e/claw-test.test.ts(fresh-install scripted scenario) — needs investigation; took 3.9s and reported "produces zero error/blocker friction" failure.
Why: These failures pre-date v0.26.2 (CHANGELOG already documents "18 pre-existing master timeouts" from v0.26.0 merge). v0.26.2 brings the count to 22, suggesting a 4-test drift on master between v0.26.0 ship and now. Fixing inside v0.26.2 would balloon scope from a 6-file OAuth fix-wave to a 30+ file test-infra repair. The fix-wave deserves its own PR with focused triage.
Likely root causes worth investigating:
- bun execSync env inheritance (already discovered + fixed in test/e2e/serve-http-oauth.test.ts during v0.26.2): bun's
execSyncdoes NOT inherit env mutations done viaprocess.env.X = ..., only OS-level env from before bun started. helpers.ts loads.env.testingand setsDATABASE_URLviaprocess.envmutation, which is invisible to subprocesses unlessenv: { ...process.env }is passed explicitly. Several of the failing E2E tests (sync, cycle, dream, claw-test) spawn subprocesses via execSync — likely the same bug. - Test ordering / DB state pollution: full-suite runs in bun test happen in a deterministic order; isolated runs of these test files may pass while suite runs fail. Could indicate beforeAll/afterAll cleanup gaps.
- Schema drift: doctor/multi-source tests may rely on specific schema state that v0.26 OAuth tables changed.
Pros:
- Separating from v0.26.2 keeps the OAuth ship focused and auditable; the 22 failures aren't blocking real-world OAuth functionality.
- The execSync env-inheritance pattern is now documented in test/e2e/serve-http-oauth.test.ts as a reference fix for the next maintainer.
- Unblocks v0.26.2 ship while preserving the failure inventory for the follow-up.
Cons:
- 22 failing tests on master is real test-infra debt.
- Some may be load-bearing (sync pipeline failures could mask real regressions in
performSync). bun run ci:local(full E2E gate) won't pass cleanly until these are addressed.
Context: Discovered during v0.26.2 ship audit. Reproduce with bun test 2>&1 | grep "^(fail)" after copying .env.testing from a sibling worktree (port 5435 test DB running). The 17/17 OAuth E2E suite passes in isolation AND in full-suite after the env-inheritance fix landed.
Effort: L (human ~4-8h; CC ~30-60min once env-inheritance fix is applied across all tests).
Depends on / blocked by: None — independent of v0.26.2.
ci-local-mirror
CI-skip artifact + signature for stages 1+2 follow-up
Priority: P0
What: After a successful local CI run via bun run ci:local, write .ci-cache/passed-<commit-sha>.json containing {commit, test_set_hash, bun_version, schema_hash, signature}. Push to a ci-cache orphan branch (or GH Releases). CI's first step fetches the artifact for the current SHA and skips the test job if (a) signature matches Garry's GPG/SSH key, and (b) test_set_hash matches what CI would have run.
Why: Stages 1+2 (shipped in this branch) give a strong local CI gate, but PR CI still re-runs every test on every push. Stage 3 closes the loop and trades ~10 min of CI wall-time for sub-second artifact verification on Garry's own pushes. External PRs are unaffected because the signature won't match — they hit the normal CI path.
Pros:
- ~10 min/PR saved on Garry's own pushes; the local gate becomes the source of truth.
- External contributor PRs untouched (no security regression).
- Forces a clear test-set-hash contract: any drift in what local-vs-CI run is caught at verification time.
Cons:
- Trust model needs careful design: signature scheme, key rotation, what happens when signature verification fails.
- Cache invalidation is real — if env or service version drifts between local run and CI, a stale local pass could ship to master.
- Adds a
ci-cachebranch / artifact storage surface to maintain.
Context:
- Discussed during the eng-review of the local CI mirror plan at
~/.claude/plans/lets-do-1-2-dockerfile-ci-zany-charm.md. - Don't start until stages 1+2 have been used for ~2 weeks AND the
scripts/e2e-test-map.tshas stabilized (so test_set_hash is a meaningful identity). - Initial trust-but-verify: run both local and CI in parallel for ~1 week before flipping the skip; alert on any disagreement.
Effort: M (human ~2-3 days + ~1 week trust-but-verify period running both local + CI in parallel; CC ~1 day for the mechanics).
Depends on / blocked by: Stages 1+2 (this PR) landing first.
test/e2e/multi-source.test.ts cascade test isn't isolated
Priority: P1
What: The "sources remove cascades to pages + chunks + timeline + links + files" test in test/e2e/multi-source.test.ts:281 fails when the file runs after other E2E files in the sequential bash scripts/run-e2e.sh order, but passes 20/20 on a fresh Postgres volume. The failing assertion is SELECT COUNT(*) FROM links WHERE from_page_id = aliceId expecting 0, getting 1 — so a prior file's setup left a links row that references a page id the cascade test happens to reuse. The test's own setupDB() truncates but doesn't sweep all referencing rows back when ids collide.
Why: Surfaced when bun run ci:local (this PR's local CI gate) ran the full sequential E2E. CI never catches it because .github/workflows/e2e.yml:40 only runs mechanical.test.ts + mcp.test.ts on PRs and nightly Tier 1. So 27 of 29 E2E files including this one aren't actually exercised by CI today. The local gate is stronger and surfaces real cross-file isolation gaps.
Pros:
- Fixing isolation makes
bun run ci:local(full E2E) reliably green. - Same fix likely to harden other E2E files that share id namespaces.
- Lets us turn
bun run ci:localinto a real ship gate.
Cons:
- Could require a per-file "namespace your test ids" pattern, ~30 min per affected file across the suite.
Context:
- Repro:
bash scripts/run-e2e.sh test/e2e/multi-source.test.tsagainst a stale DB after other E2E files have run → fails. Same against a freshdocker compose down -v && up -d postgres→ passes 20/20. - The test inserts a hardcoded
cascadetestsource id andaliceIdpage id; collisions across runs are predictable. - Likely fix: use
mkdtemp-style randomized source/page ids per test, OR have the test do a deeper reset (DELETE FROM all five tables in beforeEach) instead of relying onsetupDB's TRUNCATE behavior.
Effort: S (CC ~30 min for the multi-source.test.ts fix; M if we audit all 29 E2E files for similar id-collision risk).
Depends on / blocked by: Nothing.
scripts/run-e2e.sh:71 echo overflows on large-output failing tests
Priority: P2
What: When an E2E test fails AND prints lots of output (e.g., multi-source.test.ts floods postgres NOTICE objects), scripts/run-e2e.sh:71 does echo "$output" against a multi-megabyte shell variable. The host pipe to docker-compose-run hits EAGAIN and fails with echo: write error: Resource temporarily unavailable. With set -e, the script aborts at that point, skipping the remaining E2E files and the final SUMMARY block.
Why: When the local CI gate finds a real failure (per the multi-source.test.ts entry above), the user wants to see it AND see how the rest of the suite did. Currently the failure shadows the rest.
Pros:
- See all E2E failures from a single run instead of needing to bisect.
- Quick win, ~5 lines.
Cons:
- None worth listing.
Context:
- Reproduced live during plan verification on 2026-04-29. Previous
multi-source.test.tsfailure killed the script before postgres-bootstrap, postgres-jsonb, etc. could run. - Likely fix: replace
echo "$output"withprintf '%s\n' "$output", or write$outputto a tmpfile andcatit (handles large blobs better than echo over pipes), or pipe throughstdbuf -o0. - Don't suppress the postgres NOTICE flood at the test layer — that's separate; here we just want the script to not die when bun's stderr is verbose.
Effort: S (human or CC: ~10 min).
Depends on / blocked by: Nothing.
claw-test E2E (v0.22.16 follow-ups)
Hermes runner — src/core/claw-test/runners/hermes.ts
Priority: P2
What: Add a Hermes implementation of the AgentRunner interface. v1 ships only OpenClaw; v1.1 lands hermes once we have real friction reports from openclaw to validate the contract against.
Why: Cross-agent diff (gbrain friction diff --base openclaw --compare hermes) is the highest-leverage next signal. Friction unique to one agent vs common-to-both separates "agent contract bug" from "gbrain bug" automatically.
Effort: S (CC ~30m). Depends on: v1 openclaw runner producing real friction reports first.
Friction analytics suite — diff / trend / migration-stub
Priority: P2
What: Three new gbrain friction subcommands deferred from v1:
gbrain friction diff --base <run-or-agent> --compare <run-or-agent>(cross-agent comparison; ~80 LOC)gbrain friction trend [--since <version-or-date>] [--phase <name>](time-series across runs; ~60 LOC)gbrain friction migration-stub [--threshold N](clusters friction by phase + tokens, emitsskills/migrations/v[N+1].mdstub; ~150 LOC)
Why: Turns point-in-time reports into a slope. Pairs with the v1.1 public scoreboard.
Effort: M (CC ~2h total).
Scenario expansion — supabase-migration and supervisor-restart
Priority: P2
What: Two more scenarios under test/fixtures/claw-test-scenarios/:
supabase-migration—gbrain init --pglitethengbrain migrate --to supabase; verifies the cross-engine migration pathsupervisor-restart— kill worker mid-job; verify supervisor recovers without data loss
Why: These are the other highest-historical-pain regression points (per CLAUDE.md fix-wave history). v1 ships only fresh-install + upgrade-from-v0.18 because Codex flagged that mixing them dilutes the fresh-install signal; v1.1 lands them as separate scenarios.
Effort: M (CC ~1h each).
Real v0.18 SQL dump for upgrade scenario
Priority: P2
What: The upgrade-from-v0.18 scenario ships scaffolded — seed/dump.sql is missing. The harness gracefully no-ops the seed phase when absent, so the scenario currently behaves like fresh-install. v1.1: generate a real v0.18-shape PGLite dump per the procedure documented in test/fixtures/claw-test-scenarios/upgrade-from-v0.18/seed/README.md.
Why: Without a real seed, the scenario doesn't actually exercise the migration chain forward-walk. That's the whole point of the upgrade scenario — proves issue #239/#243/#266/#357 class regressions stay fixed.
Effort: S (CC ~30m once a v0.18 checkout is handy). Depends on: ability to run a v0.18 gbrain build.
Public scoreboard — gbrain-evals.io/friction
Priority: P3
What: Sibling-repo PR in garrytan/gbrain-evals that renders friction JSONL into a public dashboard. Friction count per version per agent, line charts over time. v1's JSONL already includes gbrain_version + agent tags so the scoreboard is a thin layer on top.
Why: Marketing surface. Proves install quality is improving release-over-release. The friction loop becomes visible to the world, not just maintainers.
Effort: M. Depends on: a working live mode and ≥10 real friction reports.
PTY-mode transcript capture
Priority: P3
What: transcript-capture.ts currently uses plain child_process.spawn pipes. Some agents only emit ANSI colors / progress UI on a TTY. v1.1 adds a PTY mode (likely via node-pty) so live-mode transcripts capture the full agent UX.
Why: Faithful transcripts make the friction → reasoning link more useful. v1 accepts that some agent UI is lost.
Effort: S (CC ~30m). Mostly a ~30 LOC swap inside spawnWithCapture.
Read-side host-isolation ($GBRAIN_HOST_HOME)
Priority: P3
What: v0.22.16 confined every ~/.gbrain write site to honor $GBRAIN_HOME. But src/commands/init.ts:299-313 still reads real ~/.claude / ~/.openclaw / ~/.codex / ~/.factory / ~/.kiro for module fingerprinting (host detection). Even with write-isolation, a claw-test running on a developer's box discovers their real installed mods. v1.1: add a separate $GBRAIN_HOST_HOME override for the read-side detection so the claw-test can run truly hermetic.
Why: v1's hermeticity contract is "writes are isolated, reads are not." v1.1 closes the read-side gap.
Effort: S (CC ~30m).
Routing-callout sweep — annotate skills the claw-test exercises
Priority: P3
What: skills/_friction-protocol.md is a cross-cutting convention. v1.1: sweep the 4–6 skills the claw-test actually exercises (setup, brain-ops, query, ingest, smoke-test, the migrations the test covers) and add a > **Convention:** see [skills/_friction-protocol.md](_friction-protocol.md). callout via the existing src/core/dry-fix.ts shape so DRY auto-fix doesn't fight it.
Why: Right now agents only call gbrain friction log if they find the protocol skill on their own. The callouts route them there proactively from any harness-exercised skill.
Effort: S (CC ~15m).
minions / worker (v0.22.14 follow-ups)
v0.22.15 — Embed cooperative-abort (HIGHEST PRIORITY — daily pain)
Priority: P0
What: Plumb signal: AbortSignal through runPhaseEmbed →
src/commands/embed.ts → embedBatch in src/core/embedding.ts. Check
signal?.aborted between OpenAI batch calls (every ~100 texts, ~2s
real-time) and between slugs in the per-slug loop.
Why: Embed phase ignores signal.aborted between batches today. Job
wall-clock timeout fires → handler keeps running → cycle's finally block
unreachable → gbrain_cycle_locks row stays held indefinitely. Every
subsequent autopilot cron cycle sees cycle_already_running → skips. Lock
TTL is 30 min; new cycles give up before that. Doctor reports UNHEALTHY.
The chain in production: ~5min cron submits cycle → 22K stale pages → embed phase takes 10–15 min → 600s timeout fires → job dead-lettered → embed keeps running → lock held → all subsequent cycles skip. Garry hits this DAILY on his production brain.
Pros: Closes the daily wedge. Makes timeouts actually effective. Lets operators bump worker timeouts confidently knowing abort actually stops work.
Cons: Touching the embed hot path; small risk of botching the abort checks. Mitigation: between-batch granularity (~2s), not per-text (too fine) or per-slug (too coarse for 500+ chunk slugs).
Context: PR #503 (v0.22.14) catches the SYMPTOM (worker stalled, queue piling up) via self-health-monitoring. This PR catches the CAUSE for one specific failure class. Both fixes are needed; they're complementary, not duplicative.
Files to touch:
src/core/cycle.ts:579—runPhaseEmbed(engine, dryRun)→ addsignal?: AbortSignalargsrc/core/cycle.ts:803— passopts.signalthroughsrc/commands/embed.ts:~363— accept signal, check between slugssrc/core/embedding.ts:51-56—embedBatch(texts, onProgress?, signal?), check between for-loop iterations ofBATCH_SIZEslices
Tests required:
- embedBatch checks signal between OpenAI calls; aborts within one batch (~2s)
- Per-slug loop in
embed.tschecks signal between slugs - End-to-end: cycle handler with embed phase + signal aborted mid-flight →
finally runs →
gbrain_cycle_locksrow deleted - Regression: 1K+ chunks scenario — embed does NOT block lock release when timeout fires
Effort: M (human: ~3 hr / CC: ~30 min).
Depends on / blocked by: Nothing. v0.22.14 ships first.
v0.23+ — Bare-worker engine reconnect parity with supervisor
Priority: P2
What: Extract the supervisor's reconnect-then-fail pattern into
MinionWorker so bare workers can retry transient DB blips before exiting.
Today the supervisor calls engine.reconnect() after 3 consecutive DB health
failures (#406); the bare worker just emits 'unhealthy' and the CLI calls
process.exit(1).
Why: Bare-worker behavior is more disruptive than supervised behavior on transient PgBouncer blips. A bare worker restarts the entire process; a supervised worker just reconnects the pool. Operationally the supervisor approach is gentler (no in-flight job loss, no PM restart latency).
Pros: Unifies bare and supervised behavior. Reduces process churn on transient network blips.
Cons: More code in MinionWorker; risk of reconnect masking a real
problem. Mitigation: cap retry attempts, fall through to 'unhealthy'
emission after the cap.
Context: Filed during v0.22.14 plan-eng-review. The asymmetry is documented in v0.22.14 CHANGELOG as deliberate; this TODO captures the "unify someday" intent.
Effort: S (human: ~2 hr / CC: ~20 min).
Depends on / blocked by: Nothing.
v0.23+ — minion_workers heartbeat table for queue_health doctor (B7)
Priority: P3
What: Add a minion_workers table (worker_id PK, hostname,
last_heartbeat, queue, concurrency, started_at) so the existing
queue_health doctor check (Postgres path) can detect dead workers via
heartbeat staleness instead of relying on the indirect lock_until proxy.
Why: v0.19.1 added queue_health checks for stalled-active jobs and
waiting-depth threshold. The worker-heartbeat subcheck was deferred (B7)
because the lock_until-on-active-jobs proxy can't distinguish "worker
exited cleanly" from "worker idle" — a check that cries wolf erodes trust
in every doctor check. With a real heartbeat row, doctor can say "no worker
seen in N intervals" with confidence.
Pros: Doctor's queue_health becomes ground-truth. Detects "worker
container died but cron didn't restart it" scenario.
Cons: New table, schema migration, every health-tick UPSERTs. Costs a write per worker per minute (default).
Context: Filed during v0.22.14 plan-eng-review. PR #503's self-health monitoring is the worker-side liveness; this would be the queue-side ground-truth.
Effort: M (human: ~1 day / CC: ~1 hr).
Depends on / blocked by: Schema migration system; nothing else.
sync (v0.22.13 follow-up — PR #490 review)
D-PR490-1 — Plumb resolved database_url through SyncOpts
Priority: P3
What: Add database_url?: string (or a richer resolvedConnection shape) to
SyncOpts and have the caller (runSync, the cycle handler, the jobs handler)
populate it from the active engine instead of having performSync /
performFullSync / import.ts each call loadConfig() separately. Today every
sync run hits the config file three times.
Why: v0.18 multi-source brains can in principle run different sources against
different database_url endpoints (or different per-source overrides via
sources.config_jsonb). Right now loadConfig() returns the global config, and
that always matches the engine in practice — but the convention papers over a
real divergence the moment someone wants per-source connection settings. Folding
the resolution into SyncOpts makes the worker-engine creation in sync.ts and
import.ts deterministic from SyncOpts alone.
Pros:
- Removes 3 redundant
loadConfig()calls per sync. - Makes
performSync/performFullSyncside-effect-free with respect to the on-disk config file. - Sets up for per-source
database_urloverrides without further refactor. - Makes the v0.22.13 belt-and-suspenders fallback (PR #490 Q3) cleaner — no
more
!config?.database_urlshort-circuit inside the parallel branch.
Cons:
- API-shape change to
SyncOpts(mild; not externally exported). - Touching three callers (
runSync, jobs handler,cycle.tsrunPhaseSync). - Only worth doing when paired with a per-source override story; otherwise it's just plumbing.
Context: Surfaced during the PR #490 plan-eng-review (parallel sync). Deferred because it isn't on the v0.22.13 critical path. The same pattern would benefit the cycle handler and the autopilot daemon. See the plan-eng-review decisions log: A4 = "Defer; file as TODO."
Depends on / blocked by: Nothing structural. Best paired with the v0.18
per-source config_jsonb work if/when that lands.
sync error-code classification (PR #501 follow-ups)
Plumb structured ParseValidationCode through ImportResult
Priority: P2
What: Replace the regex-on-error-message path in src/core/sync.ts:classifyErrorCode
with a structured code field threaded through ImportResult from the parse layer.
Three changes:
src/core/import-file.ts:362— callparseMarkdown(content, relativePath, { validate: true, expectedSlug })soparsed.errors[0].codeis populated.src/core/import-file.ts— addcode?: stringtoImportResult. Promote the structured code (or'SLUG_MISMATCH'when the existing expectedSlug check trips) into the result envelope alongsideerror.src/commands/sync.ts:488— extendfailedFilesshape withcode?: string.recordSyncFailuresalready accepts the field; the only thing missing is the capture site populating it.src/core/sync.ts:classifyErrorCode— keep as a fallback for un-coded errors (DB exceptions, generic catches). Primary path reads the structured code.
Why: The repo already has ParseValidationCode + ParseValidationError in
src/core/markdown.ts:5-18, and three other consumers (src/commands/lint.ts:72,
src/commands/frontmatter.ts:148, src/core/brain-writer.ts:314) read structured
errors directly. Sync is the outlier — it calls parseMarkdown without validation
and reverse-engineers codes via regex. PR #501 shipped that regex out of pragmatism;
this TODO removes ~50% of classifyErrorCode and eliminates a class of false-positives.
Pros:
- One source of truth for parse codes (the enum in
markdown.ts). - Eliminates regex fragility — adding a new validation code in
markdown.tsautomatically flows to sync without a new regex. - Closes the case where canonical messages (
File is empty...,No closing ---...) don't match aspirational regex patterns.
Cons: Touches ImportResult interface, which ripples through src/commands/import.ts:105,
src/commands/sync.ts:498-510, src/core/cycle.ts, brain-writer reconciler.
Context: PR #501 documented this as P3 in the eng review at
~/.claude/plans/then-codex-synchronous-toucan.md. Codex's outside-voice review
agreed independently. The fix is small — ~50 lines including tests + downstream
call sites — and it's the correct architectural endpoint.
Effort: M (human: ~2 hr / CC: ~20 min).
Depends on / blocked by: Nothing.
CHANGELOG migration note for acknowledgeSyncFailures() shape change
Priority: P0 — required at /ship time
What: When PR #501 ships, the release CHANGELOG entry MUST include this
### For contributors block:
### For contributors
`acknowledgeSyncFailures()` now returns `{count, summary}` instead of `number`.
If you import this directly from `gbrain/sync`, replace `n` with `result.count`
and use `result.summary` for the new code-grouped breakdown.
Why: The function is exported from src/core/sync.ts:433 and reachable via
the package exports map. External TS consumers (gbrain-evals, host agent forks)
that imported it got number and now get an object — silent type break.
Effort: XS (human: ~1 min). Just don't forget.
Depends on / blocked by: PR #501 ship.
Concurrent-safe ack of ~/.gbrain/sync-failures.jsonl
Priority: P3
What: Two concurrent gbrain sync runs hitting acknowledgeSyncFailures()
can clobber each other. The function does a whole-file writeFileSync rewrite
(src/core/sync.ts:433-455); recordSyncFailures() does independent
appendFileSync (src/core/sync.ts:395-416). Concurrent ack + append can lose rows.
Why: Pre-existing — predates PR #501. Real risk only on autopilot setups where multiple sync invocations might overlap (rare today, more likely as multi-source sync matures).
Fix sketch: Atomic rename pattern (write to sync-failures.jsonl.tmp, then
renameSync) plus a file lock for the read-modify-write cycle. Or move the
acknowledged-set to the DB.
Effort: S (human: ~1 hr / CC: ~10 min).
Depends on / blocked by: Nothing.
test-infra
Parallel-load timeout flake on v0.21 PGLite-heavy tests
Priority: P0
What: 22 tests added in v0.21.0 (Code Cathedral II) consistently fail in the full bun test run with timeout-pattern elapsed times of 7-10s, but pass in isolation. Every failing test calls engine.initSchema() in beforeAll without a timeout extension. Under parallel load (168 test files now run concurrently after v0.21 added ~24 new files), initSchema exceeds bun's default 5s beforeAll timeout.
Affected files include (non-exhaustive): test/sync-strategy.test.ts, test/cathedral-ii-brainbench.test.ts, test/code-edges.test.ts, test/reindex-code.test.ts, test/reconcile-links.test.ts, test/two-pass.test.ts, test/parent-symbol-path.test.ts, test/pglite-v0_19.test.ts.
Why: Currently triaged as "skip pre-existing, ship anyway" but that's not a real fix. Blocks /ship for anyone whose CHANGELOG-time test run sees them.
Pros: Fixing it lets /ship run cleanly without manual triage every release.
Cons: ~22 file edits adding beforeAll(async () => {...}, 30000) is mechanical but dull.
Context: Same pattern fixed in v0.20.5 wave for test/e2e/minions-shell-pglite.test.ts. Single-file repro: each fails in bun test, passes in bun test <file>. Reproduces with my changes stashed, so it's on master.
Effort: S (human: ~30 min / CC: ~5 min). Mechanical: grep for beforeAll(async () => { in affected files, add , 30000) argument.
Depends on / blocked by: Nothing.
resolver / check-resolvable (v0.22.4 follow-ups)
D10 — Extend check-resolvable to parse RESOLVER.md disambiguation rules
Priority: P2
What: Extend src/core/check-resolvable.ts:357-390 to parse a structured
disambiguation block in RESOLVER.md (e.g. a ## Disambiguation rules
numbered list with parseable <trigger> → <winning-skill> shape) and treat
resolved overlaps as non-issues. Then the action message at
src/core/check-resolvable.ts:388 ("Add disambiguation rule in RESOLVER.md OR
narrow triggers") stops lying about the OR — currently only the second branch
silences the warning.
Why: The current MECE-overlap fix path forces authors to delete user-facing
triggers from skill frontmatter. That's wrong for cases where two skills
legitimately respond to the same phrase under different contexts (e.g.
"citation audit" → focused fix vs broader brain health). A real
disambiguation parser would let RESOLVER.md carry the resolution while
keeping both skills' triggers intact for chaining.
Pros:
- The action message stops misleading users.
- v0.22.4 D2 used the "narrow triggers" path because the disambiguation parser doesn't exist yet; landing this would let v0.23+ keep dual triggers for genuinely-overlapping skills.
- Aligns RESOLVER.md's stated role (the dispatcher) with what the checker actually reads.
Cons:
- Introduces a new
RESOLVER.mdsyntactic contract that other tooling now has to respect (parser, lint, downstream forks reading the same file). - Risk of false-positive resolution if the parser is loose.
- ~80 lines of parser + tests; not blocking anything in v0.22.4.
Context:
- The "OR" in the action message is misleading today. Confirmed at
src/core/check-resolvable.ts:388. - The MECE detector loop is at
src/core/check-resolvable.ts:357-390. - The disambiguation rules already exist as prose in
skills/RESOLVER.md(the citation-audit row added in v0.22.4 is the pattern). They're agent-facing routing hints today, not parsed structure.
Effort: S (human: ~4-6 hours / CC: ~30 min for parser + 12-16 test cases).
Depends on / blocked by: Nothing.
code-indexing (v0.21.0 Cathedral II follow-ups)
B2 — Magika auto-detect for extension-less files (Layer 9 deferred)
Priority: P2
What: Embed Google's Magika ML classifier (~1MB ONNX) as a bundled asset. Wire into detectCodeLanguage as the fallback for files with no recognized extension (Dockerfile, Makefile, .envrc, shell scripts with shebangs but no .sh). The chunker already has setLanguageFallback(fn) as a module-level hook.
Why: v0.20.0 widens the file classifier from 9 to 35 extensions (Layer 2), covering most real-world cases. Extension-less files still slip through to recursive chunks. Magika would close the last common case.
Pros: Completes the file-classification story. Unblocks chunker on real-world configs + build scripts.
Cons: ~1MB asset bundled with bun --compile. Integration risk: Magika's ONNX runtime needs WASM compat with bun. The plan explicitly allowed deferring B2 because bundling surprises late in implementation are costly.
Context:
src/core/chunkers/code.tsexportssetLanguageFallback(fn: LanguageFallback | null)— call at process start with a Magika-powered classifier.detectCodeLanguage(filePath, content?)already accepts optional content for fallback paths.- The NPM
magikapackage is the first thing to try; needs bun-compile compatibility verification.
Effort: M (human: ~2-3 days / CC: ~2 hours for the integration + CI guard).
Depends on / blocked by: Nothing. Hook is in place as of v0.20.0.
A4 — full doc_comment extraction at chunk time
Priority: P2
What: When the chunker emits a method/class/function, look at the comment node(s) immediately preceding the declaration and persist them as content_chunks.doc_comment. The FTS trigger from Layer 1b already weights doc_comment 'A' above chunk_text 'B' — the ranking is ready, the column is populated NULL today.
Why: "how does X handle N+1" should rank the docstring that explains N+1 above the function body or any prose paragraph. Layer 1b paved the ranking half; extraction is the remaining half.
Pros: Material MRR lift on natural-language queries. Zero schema work (column + trigger already in place).
Cons: Per-language convention detection — JSDoc blocks, Python docstrings (first string expression in a function body), C-style doc comments, etc. Not hard but each language has edge cases.
Context:
src/core/chunkers/code.tsemits chunks inchunkCodeTextFull. Walk each declaration's preceding sibling(s) for comment nodes.- ChunkInput already has
doc_comment?: string. Populate at chunk time and it flows throughupsertChunks(Layer 6 wired those columns). - Per-language config: leading-comment type names per language (
comment,line_comment,block_comment,documentation_comment). - Test hook:
test/cathedral-ii-brainbench.test.tshas adoc_comment_matchingplaceholder — flesh it out end-to-end.
Effort: M (human: ~2 days / CC: ~90 min for the 8 Layer-5 langs).
Depends on / blocked by: Nothing. Layer 1b + Layer 6 both in place.
C6 — gbrain code-signature "(A, B) => C"
Priority: P3 (stretch)
What: Type-signature retrieval via tree-sitter type captures per language. "Find every function whose signature returns a Promise" or "(string, number) => boolean".
Why: Each language's type system is its own mini-cathedral. Ship per-language rather than as one item.
Effort: L per language (typescript-first).
Depends on / blocked by: Nothing — additive on the Layer 5 edge schema.
Cross-file edge resolution (Layer 5 precision upgrade)
Priority: P3
What: Today every call edge lands unresolved in code_edges_symbol with to_symbol_qualified = bare callee name. Second-pass resolution: after all code files import, walk every code_edges_symbol row and try to resolve to_symbol_qualified via symbol_name_qualified join; if found within the same source, write a resolved row to code_edges_chunk.
Why: getCallersOf("searchKeyword") currently returns the Layer 6 ambiguity — every searchKeyword call site in any class. Receiver-type analysis lifts this.
Effort: L. Needs receiver-type inference; can ship per-language.
Depends on / blocked by: Nothing — UNION-on-read path keeps unresolved edges surfaced even without this.
P3 — Dev experience: test suite parallelism on fast multi-core machines
Context: bun test on M-series Macs spawns ~1 worker per core. test/dream.test.ts (5 describe blocks, 11 tests) and test/orphans.test.ts create a fresh PGLite engine in beforeEach that runs ~20 schema migrations per test. Under parallel load, WASM-instance contention causes ~18 beforeEach timeouts at 5–9s.
Evidence: CI (ubuntu-latest, fewer cores) is green on every PR. Running the suspect files in isolation (bun test test/dream.test.ts test/orphans.test.ts) is also green. Reproduces only on fast multi-core local machines running the full 136-file parallel suite.
Fix: move engine creation from beforeEach to beforeAll per describe block; add a data-reset helper (delete-all-rows-in-relevant-tables) between tests. ~80 LOC change across two test files.
Priority: P3 because production CI is unaffected. Hits local dev iteration speed on fast Macs.
Found: 2026-04-24 during v0.19.0 production-readiness review.
Completed
Checks 5 + 6 for check-resolvable
Completed: v0.19.0 (2026-04-22)
Both checks shipped as real implementations, not just filed issues:
- Check 5 (trigger routing eval):
src/core/routing-eval.ts+gbrain routing-evalCLI. Structural layer runs incheck-resolvableby default;--llmopts into LLM tie-break. Fixtures live atskills/<name>/routing-eval.jsonl. - Check 6 (brain filing):
src/core/filing-audit.ts+skills/_brain-filing-rules.json. Newwrites_pages:+writes_to:frontmatter. Warning-only in v0.19, error in v0.20.
DEFERRED[] in src/commands/check-resolvable.ts is now empty — v0.19 shipped both deferred checks as working code paths, not as issue URLs. The export stays in place for future deferred checks.
BrainBench Cats 5/6/8/9/11 — shipped to sibling repo
Completed: v0.20.0 (2026-04-23)
All five previously-deferred BrainBench categories shipped as working runners in the sibling repo github.com/garrytan/gbrain-evals:
- Cat 5 Provenance —
eval/runner/cat5-provenance.tswith dedicatedclassify_claimtool (3-way label:supported | unsupported | over-generalized) - Cat 6 Prose-scale auto-link precision —
eval/runner/cat6-prose-scale.ts(baseline-only) +eval/runner/adversarial-injections.ts(6 injection kinds) - Cat 8 Skill Compliance —
eval/runner/cat8-skill-compliance.ts(brain-first / back-link / citation-format / tier-escalation, deterministic from tool-bridge trace) - Cat 9 End-to-End Workflows —
eval/runner/cat9-workflows.ts(rubric-graded) - Cat 11 Multi-modal Ingestion —
eval/runner/cat11-multimodal.ts(PDF/audio/HTML)
Plus supporting infrastructure: agent adapter (Sonnet + 12 read + 3 dry_run tools), structured-evidence Haiku judge contract, PublicPage/PublicQuery sealed qrels, 6-artifact flight-recorder, 6 portable JSON schemas for v1→v2 driver swap.
Scope pivot: originally planned for in-tree v1.1 delta; mid-PR pivoted to extract the entire eval harness so gbrain users don't download the ~5MB corpus at install time. BrainBench is now a public sibling benchmark; gbrain ships clean.
v0.10.5: inferLinkType residuals (works_at, advises)
Completed: v0.20.0 (2026-04-23)
src/core/link-extraction.ts — WORKS_AT_RE and ADVISES_RE expanded with
rank-prefixed engineer patterns ("senior/staff/principal/lead engineer at"),
discipline-prefixed ("backend/frontend/ML/security engineer at"), broader role
verbs ("manages engineering at", "running product at", "heads up X at"),
possessive time ("his/her/their time at"), role-noun forms ("tenure as",
"stint as", "role at"), advisory capacity phrasings, "as an advisor" forms,
and qualifier-specific advisors. New EMPLOYEE_ROLE_RE prior fires for
self-identified employees at the page level, biasing outbound company refs
toward works_at when per-edge verbs are absent. Precedence: investor > advisor
employee. Existing tests in
test/link-extraction.test.tscover the new patterns.
P1 (BrainBench v1.1 — remaining categories)
Cats 5/6/8/9/11 shipped to the sibling repo in v0.20.0 — see the Completed section above. One remaining scope item:
BrainBench Cat 1+2 at full scale
What: Existing benchmark-search-quality.ts (29 pages, 20 queries) and benchmark-graph-quality.ts (80 pages, 5 queries) currently pass at small scale. v1.1 extends both to 2-3K rich-prose pages generated via Opus to surface scale-dependent failures (tied keyword clusters, hub-node fan-out, prose-noise extraction precision).
Why deferred from PR #188: Needs ~$200-300 of Opus tokens for the rich corpus. The 80-page version already proves algorithmic correctness; scale-up proves it survives real-world load.
Threshold: maintain v1 metrics at 30x scale.
v0.10.4: inferLinkType prose precision fix
Shipped in PR #188. BrainBench Cat 2 rich-corpus type accuracy went from 70.7% → 88.5%. Fix: widened verb regexes (added "led the seed/Series A", "early investor", "invests in", "portfolio company", etc.), tightened ADVISES_RE to require explicit advisor rooting (generic "board member" matches investors too), widened context window 80→240 chars, added person-page role prior (partner-bio language → invested_in for outbound company refs only). Per-type after fix: invested_in 91.7% (was 0%), mentions 100%, attended 100%. works_at 58% and advises 41% are next iteration's residuals.
v0.10.4: gbrain alias resolution feature (driven by Cat 3)
What: Add an alias table to gbrain so "Sarah Chen" / "S. Chen" / "@schen" / "sarah.chen@example.com" resolve to one canonical entity. Schema: aliases (id, slug, alias_text) with a unique index. Search blends alias matches into hybrid scoring.
Why: BrainBench Cat 3 measured 31% recall on undocumented aliases — that's the v0.10.x baseline. With alias table, should jump to 80%+.
Depends on: Cat 3 baseline (shipped in PR #188).
P1
Minions shell jobs — Phase 2 scheduling (deferred from v0.13.0)
What: minion_schedules table + autopilot-cycle scanner that submits due shell jobs.
Why: v0.13.0 moves shell scripts to Minions but still leaves scheduling in the host crontab. Your OpenClaw's scripts/service-manager.sh + crontab is the only piece left on the host side. A DB-driven scheduler would mean a single gbrain autopilot --install replaces the host crontab entirely, scheduling is visible via gbrain jobs list --scheduled, and downtime-on-one-machine tolerance improves (schedule is shared DB state, not per-host crontab).
Pros: Canonical host-agnostic deployment. No more host-specific crontab.
Cons: Cross-engine migration complexity (new table on both PGLite + Postgres). Autopilot-cycle scanner needs to handle missed-schedule semantics (fire-once-on-startup or skip-if-past-now), and this is where every other cron-like system has historically accrued bugs.
Depends on: v0.13.0 shell jobs shipped. ✅
gbrain crontab-to-minions <file> migration helper (deferred from v0.13.0)
What: Parse an existing crontab file, emit a proposed rewrite using gbrain jobs submit shell ... for each deterministic entry, keep LLM-requiring entries as-is.
Why: Hand-rewriting ~14 OpenClaw cron entries is error-prone and one-shot. A helper would make the migration reversible and auditable (diff the before/after crontab, dry-run the first N, commit).
Pros: Removes the "rewrite 14 lines by hand" tax every agent operator pays on adoption.
Cons: Crontab parsing is historically fiddly (5-field vs 6-field, @hourly aliases, Vixie extensions, env vars in crontab). Could misrewrite entries with shell substitution.
Depends on: v0.13.0 shell jobs shipped. ✅
Batch the DB-source extract read path (deferred from v0.12.1)
What: extractLinksFromDB and extractTimelineFromDB at src/commands/extract.ts:447, 504 issue one engine.getPage(slug) per slug after engine.getAllSlugs(). On a 47K-page brain that's still 47K serial reads over the Supabase pooler.
Why: v0.12.1 fixed the write-side N+1 with batched INSERTs (~100x fewer round-trips). The read side still does serial getPage() calls — each fetches compiled_truth + timeline + frontmatter (tens of KB per page). On a 47K-page Supabase brain that's ~10-20 minutes of read latency before any work happens. The v0.12.0 orchestrator's backfill uses --source db, so this stays slow until fixed.
Pros: Mirrors the write-side fix on the read path. Combined with batched writes, full re-extract on a 47K-page brain should drop from "minutes" to "seconds" end-to-end. Eliminates the implicit listPages-pagination-mutation learning risk by giving you a snapshot read.
Cons: New engine method (getPagesBatch(slugs: string[]) → Promise<Page[]> or a streaming cursor) needs to land on both PGLite and Postgres. Memory budget — a 47K-page brain with ~30KB/page is ~1.4GB if loaded all at once; needs chunked iteration (e.g., 500 slugs/query, stream-process).
Context: Codex's plan-time review and the testing/performance specialists at ship time both flagged this. Filed during v0.12.1 to ship the bug fix without scope creep. Approach: add getPagesBatch(slugs) returning chunked results, then update the 4 DB-source extract paths to consume it.
Depends on: v0.12.1 ships first.
Batch embedding queue across files
What: Shared embedding queue that collects chunks from all parallel import workers and flushes to OpenAI in batches of 100, instead of each worker batching independently.
Why: With 4 workers importing files that average 5 chunks each, you get 4 concurrent OpenAI API calls with small batches (5-10 chunks). A shared queue would batch 100 chunks across workers into one API call, cutting embedding cost and latency roughly in half.
Pros: Fewer API calls (500 chunks = 5 calls instead of ~100), lower cost, faster embedding.
Cons: Adds coordination complexity: backpressure when queue is full, error attribution back to source file, worker pausing. Medium implementation effort.
Context: Deferred during eng review because per-worker embedding is simpler and the parallel workers themselves are the bigger speed win (network round-trips). Revisit after profiling real import workloads to confirm embedding is actually the bottleneck. If most imports use --no-embed, this matters less.
Implementation sketch: src/core/embedding-queue.ts with a Promise-based semaphore. Workers await queue.submit(chunks) which resolves when the queue has room. Queue flushes to OpenAI in batches of 100 with max 2-3 concurrent API calls. Track source file per chunk for error propagation.
Depends on: Part 5 (parallel import with per-worker engines) -- already shipped.
P0
PGLite test-runner concurrency flake (~27 false failures in full bun test)
What: Fix the concurrent-PGLite-init flake that surfaces ~27 error: PGLite not connected. Call connect() first. failures when bun test runs all 174 unit-test files together. Each failing file passes in isolation; failures only appear under full-suite parallelism.
Why: The failures are masking real signal. /ship and any solo dev running bun test has to manually triage 27 results every time. Today they're all in test/cathedral-ii-pglite.test.ts, test/cathedral-ii-brainbench.test.ts (Layer 5/6/7/8 + parent_scope_coverage + call_graph_recall), test/sync.test.ts (4 dry-run cases), test/reindex-code.test.ts (Layer 13 E2). All exist on master and date back to v0.12.3-v0.21.0 — pre-existing, not caused by any one branch.
Context: Confirmed pre-existing on master via git diff origin/master...HEAD --stat -- <failing files> returning empty. Tests pass cleanly in 1-3-file batches. Wall clock for the full suite is 596s. Likely root causes: (a) PGLite has a singleton or shared OPFS-like state that races under parallel PGlite.create() calls, (b) test/cathedral-ii-pglite.test.ts "fresh-install schema" tests assume exclusive PGLite access, (c) bun test concurrency exceeds what PGLite's WASM init can handle.
Pros: Green suite signal. Faster shipping. Stops eroding trust in bun test.
Cons: Likely needs PGLite engine-per-test isolation (each test gets its own dedicated engine instance via tmpdir) or a bun test --concurrency=N cap. Both touch test infra used by 50+ files.
Effort: M (human: 1 day to root-cause + implement / CC: ~2-3 hours via /investigate).
Discovered: v0.25.0 ship, 2026-04-25.
Fix bun build --compile WASM embedding for PGLite
What: Submit PR to oven-sh/bun fixing WASM file embedding in bun build --compile (issue oven-sh/bun#15032).
Why: PGLite's WASM files (~3MB) can't be embedded in the compiled binary. Users who install via bun install -g gbrain are fine (WASM resolves from node_modules), but the compiled binary can't use PGLite. Jarred Sumner (Bun founder, YC W22) would likely be receptive.
Pros: Single-binary distribution includes PGLite. No sidecar files needed.
Cons: Requires understanding Bun's bundler internals. May be a large PR.
Context: Issue has been open since Nov 2024. The root cause is that bun build --compile generates virtual filesystem paths (/$bunfs/root/...) that PGLite can't resolve. Multiple users have reported this. A fix would benefit any WASM-dependent package, not just PGLite.
Depends on: PGLite engine shipping (to have a real use case for the PR).
Runtime MCP access control
What: Add sender identity checking to MCP operations. Brain ops return filtered data based on access tier (Full/Work/Family/None).
Why: ACCESS_POLICY.md is prompt-layer enforcement (agent reads policy before responding). A direct MCP caller can bypass it. Runtime enforcement in the MCP server is the real security boundary for multi-user and remote deployments.
Pros: Real security boundary. ACCESS_POLICY.md becomes enforceable, not advisory.
Cons: Requires adding sender_id or access_tier to OperationContext. Each mutating operation needs a permission check. Medium implementation effort.
Context: From CEO review + Codex outside voice (2026-04-13). Prompt-layer access control works in practice (same model as Garry's OpenClaw) but is not sufficient for remote MCP where direct tool calls bypass the agent's prompt.
Depends on: v0.10.0 GStackBrain skill layer (shipped).
P1 (new from v0.25.0 — eval-capture adversarial review)
v0.25.0 eval-capture follow-ups (6 surgical hardenings)
Priority: P1
What: Six targeted hardenings on the v0.25.0 eval-capture surface, all surfaced by the /ship adversarial review and triaged out of the v0.25.0 PR to keep scope tight:
gbrain eval prune --dry-run: replace thelistEvalCandidates(limit:100k) + filtercount with a realengine.countEvalCandidatesBefore(date)method. Today the warning ateval-prune.ts:107-109honestly tells the user the count may be undercounted, but a brain with > 100k rows + old data could still confuse a careful operator. NewBrainEnginemethod on both engines, ~30 LOC, lifts the floor count to a true count.- PII scrubber CC false-positive rate: 16-digit Luhn-valid order IDs / invoice numbers get redacted as
[REDACTED]. Either require a contextual prefix (card,cc,credit) within N chars, or document the tradeoff explicitly indocs/eval-capture.md. The two approaches differ in coverage so list them as alternatives. eval_capture_failures.reasonenum:'scrubber_exception'is dead telemetry — no realistic path emits it (the scrubber is regex-only and never throws). Either remove the value from the schema CHECK + enum, OR wrapscrubPiiin a try-catch insidebuildEvalCandidateInputso the value is actually reachable.id DESCtiebreaker docs: CLAUDE.md says "stable id-desc tiebreaker so--sincewindows never dupe/miss rows". This is true within a single call but doesn't prevent dupe/miss across overlapping windows when LIMIT < total. Either add a realid-cursor (WHERE id < $cursor) for export, or scope the doc claim to "within a single export call".- Public-exports canaries: 6 of 17 subpaths (
gbrainroot,/minions,/engine-factory,/transcription,/backoff,/extract) havecanary: []— the test only checks the import resolves, so a barrel module accidentally losing its named exports would still pass. Pin one stable canary symbol per subpath. EXPECTED_COUNTduplication:scripts/check-exports-count.shandtest/public-exports.test.tsboth hardcode17. Drift risk. Make one read the other (or both compute frompackage.json).
Why: All 6 are real (some informational, some footgun-class) but each is small and surgical. Bundling into one v0.25.1 follow-up PR keeps the v0.25.0 ship clean and lets the fixes land with their own dedicated tests + CHANGELOG entry.
Effort: S total (human: ~half day / CC: ~1.5 hours).
Discovered: v0.25.0 ship adversarial review, 2026-04-25.
P1 (new from v0.7.0)
Constrained health_check DSL for third-party recipes
Completed: v0.9.3 (2026-04-12). Typed DSL with 4 check types (http, env_exists, command, any_of). All 7 first-party recipes migrated. String health checks accepted with deprecation warning + metachar validation for non-embedded recipes.
P1 (new from v0.18.0 — test flakiness)
beforeAll hook timeouts under parallel test runner
What: 17 tests across 9 files (dream, orphans, brain-allowlist, extract-db, multi-source-integration, core/cycle, migrations-v0_12_2, migrations-v0_13_1, oauth) fail with beforeEach/afterEach hook timed out for this test at the 7-10 second threshold when run via bun run test (parallel). Every test passes in isolation (bun test path/to/file.test.ts → 0 fail). Root cause is PGLite schema init racing under concurrent test files.
Why: bun run test is the pre-ship gate and reports these as failures, forcing manual triage on every /ship. The tests themselves are correct — the runner is stressing PGLite boot. Bumping the hook timeout or running E2E-like tests with --bail or serial execution would clear the 18 false positives.
Fix options:
- Bump per-test hook timeout to 30s in
bunfig.toml(quick fix, low risk) - Move PGLite-init-heavy tests to
test/e2e/so they run serially viascripts/run-e2e.sh(follows existing pattern) - Share a module-scoped PGLite instance across describe blocks within a file (biggest win — most fixture setup is identical)
Effort: 30 min for option 1, ~2 hours for option 3.
Context: Noticed during /ship merge wave on garrytan/mcp-key-mgmt (2026-04-16 branch merge of v0.18.0). Failure set stayed exactly 17-18 tests across multiple /ship runs, confirming deterministic flakes rather than real regressions. Blocking workaround: run the specific test file to verify after any suite change.
P1 (new from v0.11.0 — Minions)
Per-queue rate limiting for Minions
What: Token-bucket rate limiting per queue via a new minion_rate_limits table (queue, capacity, refill_rate, tokens, updated_at), with acquire/release in claim().
Why: The #1 daily OpenClaw pain is spawn storms hitting OpenAI/Anthropic rate limits. max_children caps fan-out per parent, but a queue with 50 ready jobs will still slam the API. Every Minions consumer currently reinvents token-bucket in user code.
Pros: First-class rate limiting means no consumer has to roll their own. Composes with max_children (which is per-parent) to give two orthogonal throttles.
Cons: Adds a write hotspot on the rate-limit row. Mitigate by keeping it a simple UPDATE ... WHERE tokens > 0 RETURNING that fails fast and puts the claim back in the pool.
Effort: ~2 hours. Deferred from v0.11.0 to keep the parity PR at a reviewable size.
Depends on: Minions (shipped in v0.11.0).
Minions repeat/cron scheduler
What: BullMQ-style repeatable jobs. queue.add(name, data, { repeat: { cron: '0 * * * *' } }).
Why: Idempotency keys (shipped in v0.11.0) are the foundation. Consumers currently use launchd/cron to fire gbrain jobs submit, but a native scheduler inside the worker would be cleaner and portable across deployments.
Pros: One mental model for both immediate and scheduled work. Idempotency prevents double-fire.
Cons: Every cron library has edge cases (DST, missed intervals on worker restart). Use a battle-tested parser.
Effort: ~1 day.
Depends on: Idempotency keys (shipped in v0.11.0).
Minions worker event emitter
What: worker.on('job:completed', handler) / worker.on('job:failed', ...) instead of polling.
Why: Consumers currently poll getJob(id) to watch state changes. An event API is the ergonomic BullMQ has and Minions doesn't.
Effort: ~4 hours.
waitForChildren(parent_id, n) / collectResults(parent_id) helpers
What: Convenience wrappers over readChildCompletions for common fan-in patterns.
Why: The child_done inbox primitive shipped in v0.11.0. Now add the ergonomic API on top so orchestrators don't have to write the polling loop.
Effort: ~2 hours.
Depends on: child_done inbox primitive (shipped in v0.11.0).
P2
Orchestrator + runner double-write to migrations ledger (deferred from v0.18.2 codex review)
What: src/commands/migrations/v0_18_0.ts:200-208 appends an entry to ~/.gbrain/migrations/completed.jsonl while src/commands/apply-migrations.ts:374-386 also appends one for the same orchestrator run. The dedupe guard in src/core/preferences.ts:120-131 only suppresses duplicate complete entries, not partial entries. Result: distorted wedge counting (3-consecutive-partials-triggers-wedge logic sees 6 partials when it should see 3).
Why: Codex plan-review caught this during PR #356 while verifying the two-migration-systems resume boundary. Not blocking v0.18.2 shipping because it only affects the wedge detection threshold, not correctness of the migration itself.
Fix: Pick one writer (prefer apply-migrations.ts runner as the single source of truth, remove the orchestrator-side append). Fold into feat/agent-migration-devex follow-up PR, which already touches both files for the migrate-command consolidation work.
Depends on: v0.18.2 shipped. ✅
22K-page resync is 30+ minutes on large brains (deferred from v0.18.2 codex review)
What: When a schema migration requires data backfill (e.g., computing page_id from page_slug across all files rows), src/commands/sync.ts:248-251, 311-337 iterates per-file. None of v0.18.2's hardening work shrinks this path. On a 22K-page brain the resync takes 30+ minutes; at 500K pages it would be several hours.
Why: Codex explicitly called out that none of PR #356 or the two follow-up PRs addresses the resync execution model. This is a separate performance-design problem.
Options to explore:
- (a) Parallel page import via worker pool (Minions-based).
- (b) Bulk COPY-based import replacing the per-file INSERT.
- (c) Incremental resync that only rewrites changed rows (needs content hash or updated_at gating).
Priority: P2 now, upgrade to P1 if another heavy migration ships that needs backfill at this scale.
Depends on: v0.18.2 shipped. ✅
Minions: gbrain jobs stats --orphaned (deferred from v0.13.0)
What: New CLI flag / output column surfacing jobs that are waiting with no registered handler on any live worker.
Why: v0.13.0 adds shell jobs that require GBRAIN_ALLOW_SHELL_JOBS=1 on the worker. If an operator submits a shell job but no worker with the flag is running, the row sits in waiting silently. The CLI's starvation warning + docs help at submit time; this TODO surfaces the problem at operational-check time.
Pros: Closes the "did my cron actually run" ambiguity for multi-machine deployments.
Cons: Knowing "no worker has this handler registered" requires worker heartbeat tracking, which Minions doesn't have yet (it's stateless at DB level beyond lock_token). Could be approximated by "no jobs of this name have completed in last N minutes AND count of waiting is > 0."
Depends on: v0.13.0 shell jobs shipped. ✅
Minions: AbortReason plumbing on MinionJobContext (deferred from v0.13.0)
What: Handlers today can't distinguish whether ctx.signal.aborted fired due to timeout, cancel, or lock-loss. v0.13.0 derives this at worker-catch-time from abort.signal.reason, but the handler can't see it directly. Expose ctx.abortReason?: 'timeout' | 'cancel' | 'lock-lost' | 'shutdown' on the context.
Why: Shell handler's kill-sequence today can't decide "retry this" (lock-lost) vs "don't retry, user cancelled" (cancel) — they look the same. A typed AbortReason lets handlers make that decision for themselves.
Pros: Handlers get richer signals.
Cons: Small surface-area addition to the handler API. Not strictly required since the worker already makes the retry/dead decision for them.
Depends on: v0.13.0 shell jobs shipped. ✅
Minions: blocking-mode audit log for true forensic integrity (deferred from v0.13.0)
What: Opt-in mode for shell-audit where appendFileSync failures DO block submission instead of logging-and-continuing.
Why: v0.13.0 ships the audit log in best-effort mode, which means a disk-full attacker can silently disable the forensic trail. Acceptable for v0.13.0 because the primary use is operational ("what did this cron do last Tuesday"), not security forensics. Operators who want fail-closed semantics should have a flag.
Pros: Enables true forensic integrity for deployments that need it.
Cons: Fail-closed means a transient disk issue blocks shell submissions, which can be worse than a missing log line for most operators. Opt-in is the right shape but adds surface area.
Depends on: v0.13.0 shell jobs shipped. ✅
Minions: configurable per-job output buffer sizes (deferred from v0.13.0)
What: Add max_stdout_bytes / max_stderr_bytes to ShellJobParams; override the 64KB/16KB defaults.
Why: 64KB/16KB covers typical OpenClaw scripts today but a verbose benchmark or a debug-dump script could need more.
Depends on: First shell-job author who actually needs it. Don't pre-build the flag.
Security hardening follow-ups (deferred from security-wave-3)
What: Close remaining security gaps identified during the v0.9.4 Codex outside-voice review that didn't make the wave's in-scope cut.
Why: Wave 3 closed 5 blockers + 4 mediums. These are the known residuals. Each is an independent hardening item that becomes trivial as Runtime MCP access control (P0 above) lands.
Items (each a separate small task):
- DNS rebinding protection for HTTP health_checks. Current
isInternalUrlvalidates the hostname string; DNS resolution happens later insidefetch. A malicious DNS server can return a public IP on first lookup and an internal IP on the actual request. Fix: resolve hostname viadns.lookupbefore fetch, pin the IP with a customhttp.Agentlookupoverride, re-validate post-resolution. Alternative: usessrf-req-filterlibrary. - Extended IPv6 private-range coverage. Block
fc00::/7(Unique Local Addresses),fe80::/10(link-local),2002::/16(6to4),2001::/32(Teredo),::/128. Current code covers::1,::, and IPv4-mapped (::ffff:*) via hex hextet parsing. - IPv4 shorthand parsing.
127.1(legacy 2-octet form = 127.0.0.1),127.0.1(3-octet), mixed-radix with trailing dots. Current code handles hex/octal/decimal integer-form IPs but not these shorthand variants. - Broader operation-layer limit caps.
traverse_graphdepthparam, plusget_chunks,get_links,get_backlinks,get_timeline,get_versions,get_raw_data,resolve_slugs— all currently accept unboundedlimit/depth. Wave 3 only clampedlist_pagesandget_ingest_log. sync_brainrepo path validation. Therepoparameter accepts an arbitrary filesystem path. Same threat model asfile_uploadbefore wave 3. AddvalidateUploadPath(strict) for remote callers.file_uploadsize limit.readFileSyncloads the entire file into memory. Trivial memory-DoS from MCP. Add ~100MB cap (matches CLI's TUS routing threshold) and stream for larger files.file_uploadregular-file check. Reject directories, devices, FIFOs, Unix sockets viastat.isFile()beforereadFileSync.- Explicit confinement root (H2).
file_uploadstrict mode currently usesprocess.cwd(). Move toctx.config.upload_root(or derive from where the brain's schema lives) so MCP server cwd can't be the wrong anchor.
Effort: M total (human: ~1 day / CC: ~1-2 hrs).
Priority: P2 — deferred consciously. Wave 3 closed the easily-exploitable paths. These are the defense-in-depth follow-ups.
Depends on: Security wave 3 shipped. None are blockers for Runtime MCP access control, but all three security workstreams (this, that P0, and the health-check DSL) converge on the same zero-trust MCP goal.
Community recipe submission (gbrain integrations submit)
What: Package a user's custom integration recipe as a PR to the GBrain repo. Validates frontmatter, checks constrained DSL health_checks, creates PR with template.
Why: Turns GBrain from a single-author integration set into a community ecosystem. The recipe format IS the contribution format.
Pros: Community-driven integration library. Users build Slack-to-brain, RSS-to-brain, Discord-to-brain.
Cons: Support burden. Need constrained DSL (P1) before accepting third-party recipes. Need review process for recipe quality.
Context: From CEO review (2026-04-11). User explicitly deferred due to bandwidth constraints. Target v0.9.0.
Depends on: Constrained health_check DSL (P1) — SHIPPED in v0.9.3.
Always-on deployment recipes (Fly.io, Railway)
What: Alternative deployment recipes for voice-to-brain and future integrations that run on cloud servers instead of local + ngrok.
Why: ngrok free URLs are ephemeral (change on restart). Always-on deployment eliminates the watchdog complexity and gives a stable webhook URL.
Pros: Stable URLs, no ngrok dependency, production-grade uptime.
Cons: Costs $5-10/mo per integration. Requires cloud account.
Context: From DX review (2026-04-11). v0.7.0 ships local+ngrok as v1 deployment path.
Depends on: v0.7.0 recipe format (shipped).
gbrain serve --http + Fly.io/Railway deployment
What: Add gbrain serve --http as a thin HTTP wrapper around the stdio MCP server. Include a Dockerfile/fly.toml for cloud deployment.
Why: The Edge Function deployment was removed in v0.8.0. Remote MCP now requires a custom HTTP wrapper around gbrain serve. A built-in --http flag would make this zero-effort. Bun runs natively, no bundling seam, no 60s timeout, no cold start.
Pros: Simpler remote MCP setup. Users run gbrain serve --http behind ngrok instead of building a custom server. Supports all 30 operations remotely (including sync_brain and file_upload).
Cons: Users need ngrok ($8/mo) or a cloud host (Fly.io $5/mo, Railway $5/mo). Not zero-infra.
Context: Production deployments use a custom Hono server wrapping gbrain serve. This TODO would formalize that pattern into the CLI. ChatGPT OAuth 2.1 support depends on this.
Depends on: v0.8.0 (Edge Function removal shipped).
P2 (knowledge graph follow-ups)
Auto-link skipped writes generate redundant SQL
What: When gbrain put is called with identical content (status=skipped), runAutoLink still does a full getLinks + per-candidate addLink loop. On N identical writes of a 50-entity page that's 50N round trips.
Why: Defensive reconciliation catches drift between page text and links table, but on truly idempotent writes it's wasted work.
Pros: Lower DB load on cron-style re-syncs. Keeps put_page latency tight under bulk MCP usage.
Cons: Need to track whether links could have drifted independent of content (e.g., a target page was deleted). Conservative approach: only skip auto-link reconciliation if status=skipped AND existing links match desired set (which still requires the getLinks call).
Context: Caught in /ship adversarial review (2026-04-18). Acceptable for v0.10.3 because auto-link runs in a transaction with row locks, so amplification cost is bounded.
Effort estimate: S (CC: ~10min) Priority: P2 Depends on: Nothing.
Audit extract --source db against auto_link config flag
What: gbrain extract links --source db writes to the same links table that auto_link=false is supposed to opt out of. The two are conceptually distinct (extract is intentional batch op, auto_link is implicit on write), but a user who turned off auto_link expecting "no automatic link writes" might be surprised.
Why: Either the behavior should match (extract checks auto_link too) or the docs should explicitly state extract is a superset.
Pros: Less surprise for users who treat auto_link as a master switch.
Cons: Some users want extract to work even when auto_link is off (e.g. one-time backfill).
Context: Caught in /ship adversarial review (2026-04-18). Documenting for now.
Effort estimate: S (CC: ~10min for docs OR ~20min for code change). Priority: P2 Depends on: Nothing.
Doctor --fix polish from v0.14.1 adversarial review
What: Six deferred findings from v0.14.1 ship-time adversarial review on src/core/dry-fix.ts:
- TOCTOU between read and write.
attemptFixreads once, writes later. Concurrent editor saves silently overwritten. Fix: re-read immediately before write and compare snapshot, orO_EXCLtempfile + rename. - Fence detection misses 4-backtick and
~~~fences.isInsideCodeFenceonly catches^```$. CommonMark-legal alternates slip through. expandBulletwalk-up is dead code. Loop breaks immediately becausebaseIndentmatches the current line. Remove or make it actually walk up.- Multi-match guard too strict. Skills with the pattern in a table-of-contents AND body get
ambiguous_multiple_matchesforever. Consider: fix first, re-scan, repeat until fixed-point. - Subprocess spam.
getWorkingTreeStatusspawnsgit statusN×M times perdoctor --fix. Cache per-skill per-invocation. doctor --fix --jsonswallows the auto-fix report.printAutoFixReportreturns early onjsonOutput; agents don't see fix outcomes. Emitauto_fixas a top-level key.
Why: None are ship-blockers; all surfaced during v0.14.1 Codex adversarial review. Bundle into one follow-up PR.
Pros: Closes the adversarial findings loop. Better correctness under concurrent edits and JSON-consumer agents.
Cons: Concurrent-edit test is finicky.
Context: v0.14.1 shipped with the 4 critical fixes (shell-injection via execFileSync, no-git-backup detection, EOF newline preservation, proximity-window consistency). These six are the deferred remainder.
Effort estimate: M (CC: ~45min for all six + tests). Priority: P2 Depends on: Nothing.
Completed
ChatGPT MCP support (OAuth 2.1)
Completed: v0.26.0 (2026-04-25) — gbrain serve --http ships full OAuth 2.1 via MCP SDK's mcpAuthRouter + OAuthServerProvider. Authorization code flow with PKCE unblocks ChatGPT. Client credentials flow unblocks Perplexity/Claude. Dynamic Client Registration available behind --enable-dcr flag (off by default). See docs/mcp/CHATGPT.md for connector setup. Closed the P0 that had been blocking the "every AI client" promise since v0.6.
Implement AWS Signature V4 for S3 storage backend
Completed: v0.6.0 (2026-04-10) — replaced with @aws-sdk/client-s3 for proper SigV4 signing.
Caller-opt-in retry for executeRaw (D3 follow-up from v0.22.1)
What: Add PostgresEngine.executeRawIdempotent(sql, params) (or a {retry: true} parameter flag on executeRaw) so callers explicitly opt into auto-retry for statements they know are idempotent. Audit existing call sites and migrate the read-only ones (search, page fetches, etc.) to the new method.
Why: Closes the gap left by D3's drop-the-wrapper decision in v0.22.1. The original #406 wrapped executeRaw in a regex-gated retry that was unsound for writable CTEs and side-effecting SELECTs. Recovery moved up to the supervisor watchdog, but per-call recovery for reads (the bulk of executeRaw traffic from MCP, search, page fetches) is gone. A caller-opt-in flag puts the idempotency decision where it belongs (at the call site, with full statement context).
Pros: Restores per-call auto-recovery for reads without the phantom-write risk on mutations. Explicit > clever: each call site declares its own idempotency posture. Future caller-added mutations get safe-by-default behavior.
Cons: Touches every existing executeRaw call site (~25). Requires careful audit — accidentally tagging a mutation as idempotent re-introduces the phantom-write bug.
Context: Codex F3 demonstrated that READ_ONLY_PREFIX = /^(\s|--.*\n)*(SELECT|WITH)\b/i is unsound — WITH x AS (UPDATE … RETURNING …) SELECT … matches the prefix but updates a row; SELECT pg_advisory_xact_lock(...) is a SELECT with side effects. The plan-eng-review wrap-up in ~/.claude/plans/system-instruction-you-are-working-tender-horizon.md has the full discussion.
Effort estimate: M (human: ~1 day / CC: ~30 min including call-site audit). Priority: P2 — current behavior (no retry, supervisor recovers within ~3 min) is acceptable but per-call recovery is a real ergonomic win. Depends on: Nothing.
Replace walkMarkdownFiles with engine.getAllSlugs() in extractForSlugs (F1 follow-up from v0.22.1)
What: The cycle path's extractForSlugs() at src/commands/extract.ts:455 still does a walkMarkdownFiles(brainDir) to build the allSlugs set for link resolution. On a 54K-page brain that's a single readdir traversal (~hundreds of ms — acceptable, dominated by the file-content-read elimination from #417). But engine.getAllSlugs() exists at extract.ts:728 and produces the same set via a single SQL query (~tens of ms).
Why: Eliminates the residual directory walk on every cycle. Codex F1 noted that the v0.22.1 plan's "cycle never re-walks the whole tree again" claim was overstated — it stops READING file contents but still walks the directory. This TODO closes that gap honestly.
Pros: Cycle becomes O(slugs sync touched), not O(total brain size). No more readdir on a growing brain. ~5 LOC change.
Cons: Crosses an FS-vs-DB consistency boundary in the FS-source extract path. Edge case: a file deleted from disk but still in DB. Currently extractForSlugs skips with if (!existsSync(fullPath)) continue — unchanged. But if a markdown file references a slug whose page exists in DB but file was deleted, the link would resolve via DB but the original extractor caught it. Needs a careful test for this case.
Context: Codex plan-review during v0.22.1 wrap, verified at extract.ts:455-456. The plan-eng-review session captured the rationale.
Effort estimate: S (human: ~2 hr / CC: ~10 min including the consistency-edge-case test). Priority: P3 — pure perf, no correctness gap. Depends on: Nothing.
err.code-based connection-error matching in postgres-engine.ts (B1 follow-up from v0.22.1)
What: The CONNECTION_ERROR_PATTERNS array (~12 strings: ECONNREFUSED, connection terminated, password authentication failed, etc.) matched against err.message and err.code. Replace with structured matching against err.code only, using postgres.js's typed error classes (PostgresError with structured codes).
Why: String matching against error messages breaks on library upgrades (postgres.js could change its error message phrasing without bumping major). Code matching is durable. The Layer 1 cleanup follows: gbrain itself doesn't define connection-error codes; it should defer to postgres.js's classification.
Pros: More durable across library updates. Less code (drop the 12-string array). Follows the typed-errors pattern v0.21.0 introduced (src/core/errors.ts).
Cons: Requires verifying which err.code values postgres.js actually exposes for each connection-failure mode. May need fallback to message-substring matching for codes that postgres.js doesn't surface.
Context: Section 2/B1 from the v0.22.1 plan-eng-review. After D3 dropped the per-call retry, isConnectionError is no longer in the hot path — only the supervisor watchdog cares about classifying connection errors, and it currently catches anything. This TODO is a cleanup pass when someone next touches that surface.
Effort estimate: S (human: ~2 hr / CC: ~10 min). Priority: P3. Depends on: The above caller-opt-in retry (#1) is the natural co-lander since both touch the same error-classification surface.
remote MCP / HTTP transport (v0.22.7 follow-ups)
Audit-log write amplification on rejected /mcp traffic
What: src/mcp/http-transport.ts writes a row to mcp_request_log for every
incoming /mcp request, including rate-limited (429), oversized (413), and
auth-failed (401) traffic. Under sustained attack the IP rate limit caps audit
writes per IP at 30/min, but at scale (10K distinct IPs) that's still 300K
inserts/min. Two follow-ups: (1) instrument the audit-write rate so we can see
the actual production volume; (2) consider a separate "rejected" table or
sampling for failed-auth rows so the success-path audit table doesn't get
swamped.
Why: Codex flagged this during the v0.22.7 ship adversarial review. We kept the full audit on purpose — forensic data of an attack is valuable — but want to revisit once we have real volume numbers.
Pros: Bounds DB write volume under attack. Keeps the success-path audit table small enough for fast queries.
Cons: Adds a second table or a sampling rule. Not free complexity. Probably not worth it until production hits a real attack pattern.
Context: src/mcp/http-transport.ts:222,235,245 (the three audit-on-reject
call sites) + src/schema.sql:342 (the unbounded table).
Effort estimate: M (human: ~half day / CC: ~30 min once we have volume data).
Priority: P3 — wait for evidence.
Depends on: Production telemetry on mcp_request_log insert rate.
validateParams doesn't check enum values or array item types
What: src/mcp/dispatch.ts:27 (extracted from src/mcp/server.ts in
v0.22.7) only checks top-level JS types. Operations declare enum constraints
(e.g. direction: 'in' | 'out' | 'both') and array items: { type: ... }
schemas in src/core/operations.ts, but validateParams ignores both. Bad
inputs still reach handlers — concretely, an invalid direction falls through
the engine's else branch at src/core/postgres-engine.ts:954, widening
traversal unexpectedly; malformed pages_updated arrays could be written as
garbage JSONB.
Why: Codex flagged this during the v0.22.7 ship adversarial review. The validator was lifted verbatim from the pre-existing stdio path during the dispatch.ts extraction — same gap exists on the stdio MCP server today, so this isn't a v0.22.7 regression. Still worth tightening, since "shared validation" is now the architectural guarantee both transports rely on.
Pros: Better defense-in-depth at the MCP boundary. Catches malformed agent inputs before the engine layer has to.
Cons: Need to walk every operation's param schema and decide which enum violations are user-facing errors vs internal bugs. May need a typed Zod-style schema layer to do this cleanly.
Context: src/mcp/dispatch.ts:27 + src/core/operations.ts (param defs).
Same gap pre-existed on stdio MCP path.
Effort estimate: M (human: ~half day / CC: ~30 min if we use the existing ParamDef shape; XL if a Zod migration is the chosen direction). Priority: P2. Depends on: Whether we want to keep the lightweight ParamDef shape or migrate to typed schemas.
Streaming MCP tool support (re-add SSE based on Accept header)
What: v0.22.7 dropped SSE entirely from gbrain serve --http because no
current MCP tool streams. When the first streaming tool ships (long-running
agent delegation as an MCP tool, resources/subscribe, sampling/createMessage),
re-add SSE in /mcp based on the Accept header per the Streamable HTTP
transport spec. ~30 lines + spec compliance test.
Why: Removing SSE simplified the v0.22.7 transport (one response path, fewer test cases). Adding it back when actually needed is cheap and keeps the code lean in the meantime.
Effort estimate: S (human: ~2 hr / CC: ~15 min). Priority: P3 — wait for the first streaming tool. Depends on: A streaming MCP tool actually existing.
access_tokens.scopes enforcement
What: The access_tokens schema has had a scopes TEXT[] column since
migration v4 (src/core/migrate.ts:84), but nothing enforces it. v0.22.7's
gbrain auth create doesn't accept a --scopes flag, and dispatchToolCall
doesn't gate on scopes. Adding per-tool scope enforcement would let
"claude-desktop-readonly" and "ingest-only" tokens exist.
Effort estimate: M (human: ~1 day / CC: ~30 min for the schema-aware gate). Priority: P3. Depends on: Nothing.