gbrain/TODOS.md

TODOS

P1 (BrainBench v1.1 — categories deferred from PR #188)

BrainBench Cat 5: Source Attribution / Provenance

What: Eval that gbrain correctly cites the right page when claiming fact F, and resolves source-conflict cases (3 sources disagree on $5M raise — which wins?). 200 queries across citation/provenance/conflict sub-categories on a 300-entity dataset with deliberately-conflicting sources.

Why deferred from PR #188: Needs ~$100-200 of Opus tokens to generate the conflict-graph dataset. v1 scope was procedural-only.

Threshold: citation_recall > 90%, citation_precision > 85%, conflict_resolution > 70%.

Depends on: Identity Resolution (Cat 3) shipped — uses same world generator pattern.

BrainBench Cat 6: Auto-link Precision under Prose (at scale)

What: Cat 10 (Robustness/Adversarial) covered code-fence leak and false-positive substrings on 22 hand-crafted cases. v1.1 extends this to 500+ prose-heavy pages with realistic narrative noise. Tests link precision in the wild, not just edge cases.

Why deferred from PR #188: Needs prose-heavy generated corpus (~$100-150 Opus). Existing 22-case eval already caught + fixed the code-fence leak bug.

Threshold: link_precision > 95% on prose, type_accuracy > 80% on varied phrasing.

BrainBench Cat 8: Skill Behavior Compliance

What: Replays 100 inbound signals through a real LLM agent loop with gbrain skills loaded. Measures: brain-first lookup compliance, back-link iron-law adherence, citation format compliance, tier escalation correctness.

Why deferred: Needs real LLM API loop (~$2K total — most expensive single category).

Threshold: brain_first_compliance > 95%, back_link_compliance > 90%, citation_format > 95%.

BrainBench Cat 9: End-to-End Workflows

What: 50 end-to-end scenarios across meeting ingestion, email-to-brain, daily-task-prep, briefing generation, sync cycle. Rubric-graded (10-15 criteria each).

Why deferred: Needs LLM agent loop (~$1K). Plus 50 hand-built rubrics.

Threshold: 80% scenario pass rate per workflow.

BrainBench Cat 11: Multi-modal Ingestion

What: PDF/image/audio/video ingestion accuracy. 50 PDFs, 30 images, 20 audio files, 10 videos, 30 HTML pages. Per-modality recall and fidelity metrics.

Why deferred: Needs licensed real datasets (Common Voice for audio etc.). Dataset curation is the bulk of the work.

Threshold: PDF text fidelity > 95% (text-based) / > 80% (scanned), audio WER < 15%, entity_recall > 80% post-ingestion.

BrainBench Cat 1+2 at full scale

What: Existing benchmark-search-quality.ts (29 pages, 20 queries) and benchmark-graph-quality.ts (80 pages, 5 queries) currently pass at small scale. v1.1 extends both to 2-3K rich-prose pages generated via Opus to surface scale-dependent failures (tied keyword clusters, hub-node fan-out, prose-noise extraction precision).

Why deferred from PR #188: Needs ~$200-300 of Opus tokens for the rich corpus. The 80-page version already proves algorithmic correctness; scale-up proves it survives real-world load.

Threshold: maintain v1 metrics at 30x scale.

v0.10.4: inferLinkType prose precision fix

Shipped in PR #188. BrainBench Cat 2 rich-corpus type accuracy went from 70.7% → 88.5%. Fix: widened verb regexes (added "led the seed/Series A", "early investor", "invests in", "portfolio company", etc.), tightened ADVISES_RE to require explicit advisor rooting (generic "board member" matches investors too), widened context window 80→240 chars, added person-page role prior (partner-bio language → invested_in for outbound company refs only). Per-type after fix: invested_in 91.7% (was 0%), mentions 100%, attended 100%. works_at 58% and advises 41% are next iteration's residuals.

v0.10.5: inferLinkType residuals (works_at, advises)

What: After the v0.10.4 fix, two link types still under-perform on rich prose. Drive these to >85% type accuracy in next iteration.

works_at: 58% type accuracy. Engineer/employee pages use varied phrasings the regex doesn't catch ("spent some time at", "joined the team", narrative "is currently at" without a verb). Approach: extend WORKS_AT_RE; consider employee-role page prior similar to partner prior.

advises: 41% type accuracy. Advisor pages often describe board roles without using the word "advisor" explicitly ("on Beta Health's board", "joined Beta as a board member"). The v0.10.4 fix tightened ADVISES_RE to require "advisor" rooting to avoid false positives from investors. Need a tighter signal that distinguishes "advisor on board" from "investor on board" — likely an advisor-role page prior plus verb-pattern combinations.

Threshold: Cat 2 rich-prose type accuracy > 92% (currently 88.5%).

v0.10.4: gbrain alias resolution feature (driven by Cat 3)

What: Add an alias table to gbrain so "Sarah Chen" / "S. Chen" / "@schen" / "sarah.chen@example.com" resolve to one canonical entity. Schema: aliases (id, slug, alias_text) with a unique index. Search blends alias matches into hybrid scoring.

Why: BrainBench Cat 3 measured 31% recall on undocumented aliases — that's the v0.10.x baseline. With alias table, should jump to 80%+.

Depends on: Cat 3 baseline (shipped in PR #188).

P1

Batch the DB-source extract read path (deferred from v0.12.1)

What: extractLinksFromDB and extractTimelineFromDB at src/commands/extract.ts:447, 504 issue one engine.getPage(slug) per slug after engine.getAllSlugs(). On a 47K-page brain that's still 47K serial reads over the Supabase pooler.

Why: v0.12.1 fixed the write-side N+1 with batched INSERTs (~100x fewer round-trips). The read side still does serial getPage() calls — each fetches compiled_truth + timeline + frontmatter (tens of KB per page). On a 47K-page Supabase brain that's ~10-20 minutes of read latency before any work happens. The v0.12.0 orchestrator's backfill uses --source db, so this stays slow until fixed.

Pros: Mirrors the write-side fix on the read path. Combined with batched writes, full re-extract on a 47K-page brain should drop from "minutes" to "seconds" end-to-end. Eliminates the implicit listPages-pagination-mutation learning risk by giving you a snapshot read.

Cons: New engine method (getPagesBatch(slugs: string[]) → Promise<Page[]> or a streaming cursor) needs to land on both PGLite and Postgres. Memory budget — a 47K-page brain with ~30KB/page is ~1.4GB if loaded all at once; needs chunked iteration (e.g., 500 slugs/query, stream-process).

Context: Codex's plan-time review and the testing/performance specialists at ship time both flagged this. Filed during v0.12.1 to ship the bug fix without scope creep. Approach: add getPagesBatch(slugs) returning chunked results, then update the 4 DB-source extract paths to consume it.

Depends on: v0.12.1 ships first.

Batch embedding queue across files

What: Shared embedding queue that collects chunks from all parallel import workers and flushes to OpenAI in batches of 100, instead of each worker batching independently.

Why: With 4 workers importing files that average 5 chunks each, you get 4 concurrent OpenAI API calls with small batches (5-10 chunks). A shared queue would batch 100 chunks across workers into one API call, cutting embedding cost and latency roughly in half.

Pros: Fewer API calls (500 chunks = 5 calls instead of ~100), lower cost, faster embedding.

Cons: Adds coordination complexity: backpressure when queue is full, error attribution back to source file, worker pausing. Medium implementation effort.

Context: Deferred during eng review because per-worker embedding is simpler and the parallel workers themselves are the bigger speed win (network round-trips). Revisit after profiling real import workloads to confirm embedding is actually the bottleneck. If most imports use --no-embed, this matters less.

Implementation sketch: src/core/embedding-queue.ts with a Promise-based semaphore. Workers await queue.submit(chunks) which resolves when the queue has room. Queue flushes to OpenAI in batches of 100 with max 2-3 concurrent API calls. Track source file per chunk for error propagation.

Depends on: Part 5 (parallel import with per-worker engines) -- already shipped.

P0

Fix bun build --compile WASM embedding for PGLite

What: Submit PR to oven-sh/bun fixing WASM file embedding in bun build --compile (issue oven-sh/bun#15032).

Why: PGLite's WASM files (~3MB) can't be embedded in the compiled binary. Users who install via bun install -g gbrain are fine (WASM resolves from node_modules), but the compiled binary can't use PGLite. Jarred Sumner (Bun founder, YC W22) would likely be receptive.

Pros: Single-binary distribution includes PGLite. No sidecar files needed.

Cons: Requires understanding Bun's bundler internals. May be a large PR.

Context: Issue has been open since Nov 2024. The root cause is that bun build --compile generates virtual filesystem paths (/$bunfs/root/...) that PGLite can't resolve. Multiple users have reported this. A fix would benefit any WASM-dependent package, not just PGLite.

Depends on: PGLite engine shipping (to have a real use case for the PR).

ChatGPT MCP support (OAuth 2.1)

What: Add OAuth 2.1 with Dynamic Client Registration to the self-hosted MCP server so ChatGPT can connect.

Why: ChatGPT requires OAuth 2.1 for MCP connectors. Bearer token auth is NOT supported. This is the only major AI client that can't use GBrain remotely.

Pros: Completes the "every AI client" promise. ChatGPT has the largest user base.

Cons: OAuth 2.1 is a significant implementation: authorization endpoint, token endpoint, PKCE flow, dynamic client registration. Estimated CC: ~3-4 hours.

Context: Discovered during DX review (2026-04-10). All other clients (Claude Desktop/Code/Cowork, Perplexity) work with bearer tokens. The Edge Function deployment was removed in v0.8.0. OAuth needs to be added to the self-hosted HTTP MCP server (or gbrain serve --http when implemented).

Depends on: gbrain serve --http (not yet implemented).

Runtime MCP access control

What: Add sender identity checking to MCP operations. Brain ops return filtered data based on access tier (Full/Work/Family/None).

Why: ACCESS_POLICY.md is prompt-layer enforcement (agent reads policy before responding). A direct MCP caller can bypass it. Runtime enforcement in the MCP server is the real security boundary for multi-user and remote deployments.

Pros: Real security boundary. ACCESS_POLICY.md becomes enforceable, not advisory.

Cons: Requires adding sender_id or access_tier to OperationContext. Each mutating operation needs a permission check. Medium implementation effort.

Context: From CEO review + Codex outside voice (2026-04-13). Prompt-layer access control works in practice (same model as Wintermute) but is not sufficient for remote MCP where direct tool calls bypass the agent's prompt.

Depends on: v0.10.0 GStackBrain skill layer (shipped).

P1 (new from v0.7.0)

Constrained health_check DSL for third-party recipes

Completed: v0.9.3 (2026-04-12). Typed DSL with 4 check types (http, env_exists, command, any_of). All 7 first-party recipes migrated. String health checks accepted with deprecation warning + metachar validation for non-embedded recipes.

P1 (new from v0.11.0 — Minions)

Per-queue rate limiting for Minions

What: Token-bucket rate limiting per queue via a new minion_rate_limits table (queue, capacity, refill_rate, tokens, updated_at), with acquire/release in claim().

Why: The #1 daily OpenClaw pain is spawn storms hitting OpenAI/Anthropic rate limits. max_children caps fan-out per parent, but a queue with 50 ready jobs will still slam the API. Every Minions consumer currently reinvents token-bucket in user code.

Pros: First-class rate limiting means no consumer has to roll their own. Composes with max_children (which is per-parent) to give two orthogonal throttles.

Cons: Adds a write hotspot on the rate-limit row. Mitigate by keeping it a simple UPDATE ... WHERE tokens > 0 RETURNING that fails fast and puts the claim back in the pool.

Effort: ~2 hours. Deferred from v0.11.0 to keep the parity PR at a reviewable size.

Depends on: Minions (shipped in v0.11.0).

Minions repeat/cron scheduler

What: BullMQ-style repeatable jobs. queue.add(name, data, { repeat: { cron: '0 * * * *' } }).

Why: Idempotency keys (shipped in v0.11.0) are the foundation. Consumers currently use launchd/cron to fire gbrain jobs submit, but a native scheduler inside the worker would be cleaner and portable across deployments.

Pros: One mental model for both immediate and scheduled work. Idempotency prevents double-fire.

Cons: Every cron library has edge cases (DST, missed intervals on worker restart). Use a battle-tested parser.

Effort: ~1 day.

Depends on: Idempotency keys (shipped in v0.11.0).

Minions worker event emitter

What: worker.on('job:completed', handler) / worker.on('job:failed', ...) instead of polling.

Why: Consumers currently poll getJob(id) to watch state changes. An event API is the ergonomic BullMQ has and Minions doesn't.

Effort: ~4 hours.

waitForChildren(parent_id, n) / collectResults(parent_id) helpers

What: Convenience wrappers over readChildCompletions for common fan-in patterns.

Why: The child_done inbox primitive shipped in v0.11.0. Now add the ergonomic API on top so orchestrators don't have to write the polling loop.

Effort: ~2 hours.

Depends on: child_done inbox primitive (shipped in v0.11.0).

P2

Security hardening follow-ups (deferred from security-wave-3)

What: Close remaining security gaps identified during the v0.9.4 Codex outside-voice review that didn't make the wave's in-scope cut.

Why: Wave 3 closed 5 blockers + 4 mediums. These are the known residuals. Each is an independent hardening item that becomes trivial as Runtime MCP access control (P0 above) lands.

Items (each a separate small task):

• DNS rebinding protection for HTTP health_checks. Current isInternalUrl validates the hostname string; DNS resolution happens later inside fetch. A malicious DNS server can return a public IP on first lookup and an internal IP on the actual request. Fix: resolve hostname via dns.lookup before fetch, pin the IP with a custom http.Agent lookup override, re-validate post-resolution. Alternative: use ssrf-req-filter library.
• Extended IPv6 private-range coverage. Block fc00::/7 (Unique Local Addresses), fe80::/10 (link-local), 2002::/16 (6to4), 2001::/32 (Teredo), ::/128. Current code covers ::1, ::, and IPv4-mapped (::ffff:*) via hex hextet parsing.
• IPv4 shorthand parsing. 127.1 (legacy 2-octet form = 127.0.0.1), 127.0.1 (3-octet), mixed-radix with trailing dots. Current code handles hex/octal/decimal integer-form IPs but not these shorthand variants.
• Broader operation-layer limit caps. traverse_graph depth param, plus get_chunks, get_links, get_backlinks, get_timeline, get_versions, get_raw_data, resolve_slugs — all currently accept unbounded limit/depth. Wave 3 only clamped list_pages and get_ingest_log.
• sync_brain repo path validation. The repo parameter accepts an arbitrary filesystem path. Same threat model as file_upload before wave 3. Add validateUploadPath (strict) for remote callers.
• file_upload size limit. readFileSync loads the entire file into memory. Trivial memory-DoS from MCP. Add ~100MB cap (matches CLI's TUS routing threshold) and stream for larger files.
• file_upload regular-file check. Reject directories, devices, FIFOs, Unix sockets via stat.isFile() before readFileSync.
• Explicit confinement root (H2). file_upload strict mode currently uses process.cwd(). Move to ctx.config.upload_root (or derive from where the brain's schema lives) so MCP server cwd can't be the wrong anchor.

Effort: M total (human: ~1 day / CC: ~1-2 hrs).

Priority: P2 — deferred consciously. Wave 3 closed the easily-exploitable paths. These are the defense-in-depth follow-ups.

Depends on: Security wave 3 shipped. None are blockers for Runtime MCP access control, but all three security workstreams (this, that P0, and the health-check DSL) converge on the same zero-trust MCP goal.

Community recipe submission (gbrain integrations submit)

What: Package a user's custom integration recipe as a PR to the GBrain repo. Validates frontmatter, checks constrained DSL health_checks, creates PR with template.

Why: Turns GBrain from a single-author integration set into a community ecosystem. The recipe format IS the contribution format.

Pros: Community-driven integration library. Users build Slack-to-brain, RSS-to-brain, Discord-to-brain.

Cons: Support burden. Need constrained DSL (P1) before accepting third-party recipes. Need review process for recipe quality.

Context: From CEO review (2026-04-11). User explicitly deferred due to bandwidth constraints. Target v0.9.0.

Depends on: Constrained health_check DSL (P1) — SHIPPED in v0.9.3.

Always-on deployment recipes (Fly.io, Railway)

What: Alternative deployment recipes for voice-to-brain and future integrations that run on cloud servers instead of local + ngrok.

Why: ngrok free URLs are ephemeral (change on restart). Always-on deployment eliminates the watchdog complexity and gives a stable webhook URL.

Pros: Stable URLs, no ngrok dependency, production-grade uptime.

Cons: Costs $5-10/mo per integration. Requires cloud account.

Context: From DX review (2026-04-11). v0.7.0 ships local+ngrok as v1 deployment path.

Depends on: v0.7.0 recipe format (shipped).

gbrain serve --http + Fly.io/Railway deployment

What: Add gbrain serve --http as a thin HTTP wrapper around the stdio MCP server. Include a Dockerfile/fly.toml for cloud deployment.

Why: The Edge Function deployment was removed in v0.8.0. Remote MCP now requires a custom HTTP wrapper around gbrain serve. A built-in --http flag would make this zero-effort. Bun runs natively, no bundling seam, no 60s timeout, no cold start.

Pros: Simpler remote MCP setup. Users run gbrain serve --http behind ngrok instead of building a custom server. Supports all 30 operations remotely (including sync_brain and file_upload).

Cons: Users need ngrok ($8/mo) or a cloud host (Fly.io $5/mo, Railway $5/mo). Not zero-infra.

Context: Production deployments use a custom Hono server wrapping gbrain serve. This TODO would formalize that pattern into the CLI. ChatGPT OAuth 2.1 support depends on this.

Depends on: v0.8.0 (Edge Function removal shipped).

P2 (knowledge graph follow-ups)

Auto-link skipped writes generate redundant SQL

What: When gbrain put is called with identical content (status=skipped), runAutoLink still does a full getLinks + per-candidate addLink loop. On N identical writes of a 50-entity page that's 50N round trips.

Why: Defensive reconciliation catches drift between page text and links table, but on truly idempotent writes it's wasted work.

Pros: Lower DB load on cron-style re-syncs. Keeps put_page latency tight under bulk MCP usage.

Cons: Need to track whether links could have drifted independent of content (e.g., a target page was deleted). Conservative approach: only skip auto-link reconciliation if status=skipped AND existing links match desired set (which still requires the getLinks call).

Context: Caught in /ship adversarial review (2026-04-18). Acceptable for v0.10.3 because auto-link runs in a transaction with row locks, so amplification cost is bounded.

Effort estimate: S (CC: ~10min) Priority: P2 Depends on: Nothing.

Audit extract --source db against auto_link config flag

What: gbrain extract links --source db writes to the same links table that auto_link=false is supposed to opt out of. The two are conceptually distinct (extract is intentional batch op, auto_link is implicit on write), but a user who turned off auto_link expecting "no automatic link writes" might be surprised.

Why: Either the behavior should match (extract checks auto_link too) or the docs should explicitly state extract is a superset.

Pros: Less surprise for users who treat auto_link as a master switch.

Cons: Some users want extract to work even when auto_link is off (e.g. one-time backfill).

Context: Caught in /ship adversarial review (2026-04-18). Documenting for now.

Effort estimate: S (CC: ~10min for docs OR ~20min for code change). Priority: P2 Depends on: Nothing.

Completed

Implement AWS Signature V4 for S3 storage backend

Completed: v0.6.0 (2026-04-10) — replaced with @aws-sdk/client-s3 for proper SigV4 signing.