10816cba38 v0.41.18.0: gbrain onboard — the activation surface gbrain didn't have before (#1521)
* feat(schema): migrations v98/v99/v100 for onboard wave (A6 A10 A11 A13 A25, codex #1 #9 #10 #11 #12)

Three schema additions supporting the gbrain onboard wave:

v98 — links.link_kind nullable column (A10, codex finding #12).
The NER extraction was originally going to add a new link_source='ner'
provenance, but that would have forced every existing link_source='mentions'
query (backlink-count filter, orphan-ratio, doctor checks) to update or
metrics would drift across the cutover. Instead: keep link_source='mentions'
for the storage layer AND add a nullable link_kind column. Three kinds:
'plain', 'typed_ner', NULL (legacy/unknown — semantically 'plain'). NOT in
the links UNIQUE constraint so the storage shape stays compatible.

v99 — timeline_entries dedup widening (A11, codex finding #11).
Pre-v99 dedup key was (page_id, date, summary). The new --from-meetings
extraction writes timeline entries with source='extract-timeline-from-
meetings:<meeting-slug>', and codex caught that two meetings with the same
date+summary on the same entity page would silently DO NOTHING — the
second meeting's provenance is lost. Widened to (page_id, date, summary,
source). Legacy rows (source='') preserve current dedup behavior.

v100 — migration_impact_log table + content_chunks_stale_idx partial
(A6 + A25 + A13 + codex findings #10 + #9). Bundled because both are
consumed by the onboard pipeline and ship together. Impact log captures
before/after metric stats so gbrain onboard --history shows real deltas;
attribution columns (job_id, source_id, brain_id, started_at,
idempotency_key) prevent concurrent runs misattributing to wrong
migrations. content_chunks_stale_idx partial WHERE embedding IS NULL
supports gbrain embed --stale + --priority recent (outer ORDER BY
p.updated_at DESC uses existing idx_pages_updated_at_desc via JOIN).
Plain NUMERIC columns; delta computed at read time (NOT a stored
GENERATED column per eng-review D2 — zero PGLite parity risk).

Slot history note: plan originally proposed v97/v98/v99 but master had
already used v95 (links 'mentions' CHECK widening), v96 (facts conversation
session index), and v97 (pages_dedup_partial_index) by ship time. Codex
caught the collision; renumbered to v98/v99/v100.

Test pin: test/schema-bootstrap-coverage.test.ts (100/100 migrations
apply clean on PGLite), test/migrate.test.ts (152 cases pass).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(remediation): extract doctor remediation library (A1, codex finding #2)

Pre-fix: src/commands/doctor.ts contained two CLI-shaped functions
(runRemediationPlan + runRemediate) with hardcoded argv parsing,
process.exit calls, and console.log emission. Onboard CLI shell and the
upcoming MCP run_onboard op couldn't compose against them — the plan
file's "100-LOC thin wrapper" assumption didn't survive codex's review
of the actual source.

Post-fix: src/core/remediation/ exports a library shape that all three
consumers (doctor CLI, onboard CLI, MCP run_onboard) wrap.

  src/core/remediation/types.ts
    RemediationPlanOpts, RemediationPlan, RemediationOpts,
    RemediationResult, StepResult, RemediationHooks (the observability
    seam — library never calls console.* itself).

  src/core/remediation/context.ts
    loadRecommendationContext moved verbatim from doctor.ts. Re-exports
    RecommendationContext from brain-score-recommendations.ts since
    that's still the canonical home for the type (consumed by
    computeRecommendations).

  src/core/remediation/plan.ts
    computeRemediationPlan(engine, opts): Promise<RemediationPlan>.
    Pure read; produces the stable JSON envelope downstream agents
    bind to. Pulls in computeRecommendations + classifyChecks +
    maxReachableScore behind one library entry point.

  src/core/remediation/run.ts
    runRemediation(engine, opts, hooks): Promise<RemediationResult>.
    Orchestrator with BudgetTracker, checkpoint resume, D5 dep
    cascade, D7 per-step recheck. Returns a result object instead
    of process.exit calls; the CLI shell maps result.budget_exhausted
    / .target_unreachable / .submitted to exit codes.

  src/core/remediation/index.ts
    Barrel for the three modules above.

doctor.ts is now a thin wrapper:
  runRemediationPlan: parse argv → computeRemediationPlan → human/JSON render
  runRemediate: parse argv → TTY confirm gate → runRemediation(hooks: console.*)
The TTY confirmation step deliberately stays in the CLI shell — the library
never asks for confirmation; that's a CLI concern.

Net: ~340 LOC removed from doctor.ts; ~470 LOC added across the library
module (with full JSDoc + per-A-decision rationale comments). Functional
behavior preserved bit-for-bit: 67 tests pass across doctor.test.ts +
v0_37_gap_fill.serial.test.ts.

The Lane E.4 source-text test (test/v0_37_gap_fill.serial.test.ts:329)
followed loadRecommendationContext to its new home at
src/core/remediation/context.ts — assertions otherwise unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(remediation): generalize computeRecommendations to accept extras (A2, codex finding #3)

Pre-fix: computeRecommendations at brain-score-recommendations.ts:170 was a
hardcoded planner for 5 synthetic check categories. Adding a Check.remediation
field to a new doctor check would NOT auto-wire into --remediation-plan —
the planner simply ignored it. Codex caught this when reviewing the plan's
"checks ARE specs" framing.

Post-fix: optional third arg `extraRemediations: RemediationStep[]` lets
callers inject step entries discovered outside the hardcoded planner. The
existing 5-category surface is preserved bit-for-bit; on id collision the
hardcoded entry wins, so an extra accidentally duplicating a hardcoded id
doesn't shadow legacy behavior.

RemediationPlanOpts gains the matching field; computeRemediationPlan in
src/core/remediation/plan.ts threads opts.extraRemediations through. The
4 new doctor checks (T4) will produce per-check helper functions that
return RemediationStep[]; onboard's render layer (T12) aggregates them
into the opts.extraRemediations slot. doctor's existing
--remediation-plan call passes empty (no behavior change for legacy CLI).

84 tests pass across brain-score-recommendations + doctor suites.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(doctor): 4 new onboard checks (embed_staleness, link_coverage, timeline_coverage, takes_count) (A16, T4)

Adds src/core/onboard/checks.ts: 4 check helpers + a runAllOnboardChecks
aggregator. Each helper returns {check, remediations}, so doctor pushes
the Check entry (for human/JSON rendering) AND onboard's plan path
collects the RemediationStep[] (via T3's new extraRemediations seam in
computeRecommendations).

embed_staleness: COUNT(*) on content_chunks WHERE embedding IS NULL.
  Cheap thanks to content_chunks_stale_idx partial (v100).
  warn at 1+ stale, fail at 1000+; remediation points at embed-catch-up
  handler (built in T6).

entity_link_coverage: fraction of entity pages with inbound links.
  Per A21 + codex #15: TABLESAMPLE BERNOULLI on PG when total_pages > 50K
  with pinned sample formula (LEAST 100, GREATEST 2, target ~5000 rows)
  AND ±sqrt(p(1-p)/n) confidence interval embedded in message
  ("coverage: 31% ± 1.3%") so warn/fail decisions show their margin of error.
  PGLite path: full scan (rare >50K).
  warn <70%, fail <40%; remediation points at extract-ner handler.

timeline_coverage: same TABLESAMPLE policy. warn <90%, fail <70%;
  remediation points at extract-timeline-from-meetings handler.

takes_count: COUNT(*) on takes table. Per A12 two-gate consent: the
  remediation only emits when `takes.bootstrap_enabled` config is true.
  Otherwise the check shows "0 takes (takes.bootstrap_enabled is false;
  opt in to enable)" without an autopilot-eligible remediation. Prevents
  unattended LLM-bearing extractions on brains that haven't opted in.

runDoctor wires runAllOnboardChecks at the end of the DB-checks block
(after stale_locks); fast-mode skipped to preserve --fast UX.

Thin-client parity (A16 spec) deferred to T16 — the MCP run_onboard op
will run these helpers server-side where engine.executeRaw works,
which is the real federated path. Adding them to doctor-remote.ts
would duplicate the logic without functional benefit since the helpers
are server-side queries.

55 doctor tests pass; typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(engine): listStaleChunks --priority recent + executeRaw AbortSignal (A13/A20, codex #7 #9)

Two interface extensions on BrainEngine, with parity across postgres-engine
and pglite-engine. Plus a follow-on fix for v99's timeline_entries dedup
widening.

listStaleChunks gains:
  - orderBy?: 'page_id' | 'updated_desc' (default 'page_id' = legacy)
  - afterUpdatedAt?: string | null (composite cursor for updated_desc)

When orderBy === 'updated_desc' the query JOINs pages and orders by
  p.updated_at DESC NULLS LAST, p.id ASC, cc.chunk_index ASC
backed by idx_pages_updated_at_desc + content_chunks_stale_idx partial
(both indexes added in v100). The cursor "next row" semantic with DESC
NULLS LAST + ASC tiebreakers is:
  (updated_at < prev) OR
  (updated_at = prev AND page_id > prev_page_id) OR
  (updated_at = prev AND page_id = prev_page_id AND chunk_index > prev_chunk_index)
First page (afterUpdatedAt undefined AND afterPageId 0) bypasses the
cursor predicate. Both engines parity-tested via 100/100 pglite-engine
tests; Postgres path mirrors the same WHERE clause structure.

executeRaw gains:
  - opts?: {signal?: AbortSignal}

Postgres impl: real cancellation via postgres.js's .cancel() on the
pending query. Pre-aborted signal short-circuits before the network
round-trip; mid-flight abort fires .cancel(). The query throws on
abort which the caller catches.

PGLite impl: in-process WASM has no kernel-level cancellation.
Best-effort: pre-check, then race the query against a signal-rejection
promise. The query keeps running in WASM but the awaited result is
discarded (DOMException AbortError thrown). Documented gap.

ReservedConnection.executeRaw extends the signature for type
compatibility but doesn't wire the signal (its only callers are
migrations + cycle-lock writes that explicitly don't want cancellation).

V99 timeline dedup follow-on: the dedup widening in migration v99
changed the unique index from (page_id, date, summary) to
(page_id, date, summary, source). The ON CONFLICT clauses in both
engines' addTimelineEntriesBatch + addTimelineEntry impls were still
using the old 3-tuple, causing 12 PGLite tests to fail with SQLSTATE
42P10 "no unique constraint matching ON CONFLICT specification".
Updated all 4 sites (2 per engine) to the 4-tuple.

Typecheck clean, 100/100 PGLite engine tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(embed): --batch-size + --priority recent + --catch-up + embed-catch-up handler (A13)

CLI surface on gbrain embed gains 3 flags:
  --batch-size N       Override hardcoded PAGE_SIZE=2000 (clamped 1..10000)
  --priority recent    Walk stale chunks newest-first (page.updated_at DESC)
                       backed by content_chunks_stale_idx + idx_pages_updated_at_desc
                       via T5's listStaleChunks(orderBy='updated_desc') extension.
                       Composite cursor (updated_at, page_id, chunk_index).
  --catch-up           Removes the GBRAIN_EMBED_TIME_BUDGET_MS wall-clock cap;
                       loops until countStaleChunks() returns 0.

EmbedOpts gains matching fields; embedAll + embedAllStale plumb them through.
The cursor tracking in embedAllStale now advances (afterUpdatedAt, afterPageId,
afterChunkIndex) instead of just (afterPageId, afterChunkIndex) when in
'updated_desc' mode. The engine returns p.updated_at as Date|string; the
caller normalizes to ISO string for the next page's cursor.

New Minion handler `embed-catch-up` registered in jobs.ts. Wraps runEmbedCore
with stale=true + catchUp=true + the priority/batchSize the caller supplies.
NOT in PROTECTED_JOB_NAMES (embedding spend only — same posture as the
existing embed-backfill handler). Consumed by the gbrain onboard remediation
pipeline (T11) when embed_staleness check fires.

63 embed tests pass; typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(extract): NER link extraction via schema-pack inference.regex (A10, T7, codex #12)

NEW src/core/extract-ner.ts: extractNerLinks(engine, opts). Walks pages,
reuses the by-mention gazetteer, applies the active schema-pack's
link_types[].inference.regex patterns to assign a typed verb to each
mention ("CEO of Acme" + Acme is a company → 'works_at' linking the
source page to Acme).

Codex finding #12 design: do NOT split link_source='ner' as a new
provenance. NER is still mention-derived; splitting would break every
existing link_source='mentions' query (backlink-count, orphan-ratio,
doctor checks). Instead: keep link_source='mentions' AND set
link_kind='typed_ner' (v98 column).

LinkBatchInput type gains link_kind field. Both engines'
addLinksBatch impls add the column to the INSERT projection + unnest()
tuple (column #11). The links UNIQUE constraint excludes link_kind so
an existing plain mention row + a typed_ner row for the same (from, to,
type, source, origin) collide DO NOTHING; the typed link goes in as a
separate row with a DIFFERENT link_type (the inferred verb), so they
don't collide on the typical case.

CLI: `gbrain extract links --ner` (DB source only). Combined
`--by-mention --ner` walk shares ONE gazetteer build across both passes
— saves a full walk on big brains. Either flag alone runs its pass
solo. Each gets its own --source-id filter inheritance.

Minion handler: `extract-ner` (NOT in PROTECTED_JOB_NAMES — regex-only,
no LLM spend). Consumed by onboard's entity_link_coverage remediation
when coverage <70%.

Target-type lookup: one round-trip SELECT slug, source_id, type FROM
pages WHERE type IN ('person', 'company', 'organization', 'entity')
AND deleted_at IS NULL — built once at extraction start, consulted
per-mention. Avoids the N+1 getPage cost.

Pack best-effort: when no active pack OR no link_types declared OR
no inference.regex on any link_type, returns pack_unavailable=true and
0 created. CLI prints a one-line note; handler returns silently.

122 tests pass (pglite-engine + by-mention); typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(extract): timeline from meetings — gbrain extract timeline --from-meetings (A11, T8, codex #11)

NEW src/core/extract-timeline-from-meetings.ts:
extractTimelineFromMeetings(engine, opts). Walks meeting pages, finds
discussed entities via two sources, writes a timeline entry on each
entity page.

Discussed-entity sources merged:
  1. Existing 'attended' links from the meeting (canonical attendees).
     One round-trip SELECT pulls all attended edges for the loaded
     meeting set; in-memory Map<meetingSlug → attendees[]> for O(1)
     lookup per meeting.
  2. Body-text mentions via the existing by-mention gazetteer
     (findMentionedEntities + cross-source guard). Catches entities
     discussed in the meeting body even when no explicit 'attended'
     link exists.

De-duped via Map<sourceId::slug → entity> within each meeting so a
person who's both an attendee AND mentioned in the body gets exactly
one timeline row per meeting, not two.

Timeline write uses TimelineBatchInput with:
  source = 'extract-timeline-from-meetings:<meeting-slug>'
  summary = 'Discussed in <meeting-title>'
  date = meeting.effective_date

Per v99 dedup widening (codex #11): the source field is now in the
uniqueness key (page_id, date, summary, source). Two meetings on the
same date with the same summary on the same entity page survive as
distinct rows — the second meeting's provenance is no longer silently
dropped.

CLI: `gbrain extract timeline --from-meetings` (DB source only). Mode
dispatch — runs SOLO (does not combine with --by-mention/--ner; those
are links passes).

Minion handler: `extract-timeline-from-meetings` (NOT in
PROTECTED_JOB_NAMES — pure SQL + string scan). Consumed by onboard's
timeline_coverage remediation when coverage <90%.

Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(takes): takes-bootstrap from concept/atom/lore pages (A12, A24, T9)

NEW src/core/extract-takes-from-pages.ts: Haiku classifier loop. Walks
pages WHERE type IN ('concept','atom','lore','briefing','writing',
'originals') AND deleted_at IS NULL AND length(compiled_truth) > 200,
ordered by updated_at DESC. Each page is truncated to 20K chars and
sent to Haiku with a strict-JSON classifier prompt:
  {"claim", "kind": fact|take|bet|hunch, "weight": 0..1}

Inserts via addTakesBatch with source='cli:takes-bootstrap-from-pages'.

Two-gate consent per A12:
  1. `takes.bootstrap_enabled` config (default false) — even the manual
     CLI refuses without it explicitly set.
  2. --yes flag (CLI) — interactive confirmation that this sends content
     to Haiku.

The handler-side gate also reads takes.bootstrap_enabled, so even a
trusted local Minion submitter (allowProtectedSubmit=true) cannot
fire takes-bootstrap on a brain that hasn't opted in.

CLI: `gbrain takes extract --from-pages [--yes] [--dry-run] [--source-id X]
[--max-pages N] [--holder name]`. Surfaces consent-gate-blocked vs
llm-unavailable distinctly so users see the actual blocker.

Minion handler `extract-takes-from-pages` added to PROTECTED_JOB_NAMES.
Consumed by onboard's takes_count remediation when count=0 AND
takes.bootstrap_enabled=true (handler-side double-check).

Per A24: ships with classifier infrastructure ONLY. Per-prompt eval suite
deferred to v0.42.1 follow-up; autopilot remediation tier for takes-bootstrap
stays manual_only until eval coverage catches up. Manual `gbrain takes
extract --from-pages --yes` is the only path that triggers it in v0.42.0.

parseClaimsJson exported for unit testing — strict JSON parse + ```json
fence strip + kind allowlist filter, returns [] on any parse failure.

Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(minions): recordMinionJobSpend primitive for MCP client_id attribution (A7+A23, codex finding #4)

NEW src/core/minion-spend.ts: small primitive that closes the per-OAuth-
client spend chain gap codex flagged when MCP run_onboard submits child
Minion jobs.

Pre-fix: only subagent loops via budget-meter.ts recorded spend against
the originating OAuth client. Generic Minion handlers (embed-catch-up,
extract-ner, extract-timeline-from-meetings, extract-takes-from-pages)
wrote to the gateway with no per-client attribution — admin-scope tokens
would have unbounded indirect spend via the run_onboard fan-out.

Convention for v0.42.0 (deferred schema column to v0.42.1):
  - run_onboard MCP op sets job.data.client_id when submitting each
    child handler.
  - Handlers that spend LLM/embedding budget call
    recordMinionJobSpend(engine, job, {operation, spendCents, ...})
    which reads job.data.client_id and writes mcp_spend_log with
    the right attribution.
  - Local-submitted jobs (CLI, autopilot tick) pass no client_id;
    the row still lands with client_id=null for global accounting.

Two exports:
  getJobClientId(job): undefined for local jobs; the OAuth client_id
    string for MCP-submitted ones.
  recordMinionJobSpend(engine, job, entry): wraps recordSpend with
    job-aware attribution. Best-effort throughout — spend telemetry
    failures MUST NOT fail the user's call.

A23 full schema column (minion_jobs.client_id + index) deferred to
v0.42.1; today's JSONB-pass-through is sufficient for the MCP
run_onboard chain to land per-client attribution end-to-end. Handlers
adopt the primitive over time; no behavior change for callers that
haven't migrated.

Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(onboard): impact capture module + writeImpactLogRow primitive (A6 + A25 + A17, T11)

NEW src/core/onboard/impact-capture.ts. Three exports:

captureMetric(engine, metric)
  Pure-ish: returns the current numeric value for one of 5 metrics
  (orphan_count, stale_count, entity_link_coverage, timeline_coverage,
  takes_count). Returns null on any throw per A17 best-effort posture
  — a stat-query failure MUST NOT block the extraction itself.

writeImpactLogRow(engine, attribution, metric, before, after, details?)
  Best-effort INSERT into v100's migration_impact_log table. Attribution
  columns (job_id, source_id, brain_id, started_at, idempotency_key,
  applied_by) per A25 + codex finding #10 so concurrent runs can't
  misattribute deltas.

withImpactCapture(engine, attribution, metric, runner, details?)
  Convenience: capture-before → run → capture-after → write log row.
  Per A17 the log row lands even when the runner throws (after-on-fail
  + error in details), so downstream consumers see a "ran but impact
  unknown" entry instead of silent loss.

Designed to be picked up by the 4 new Minion handlers (embed-catch-up,
extract-ner, extract-timeline-from-meetings, extract-takes-from-pages)
when they wrap their main runner. Handlers stay decoupled from the
log-write path — they just call withImpactCapture with the metric they
move. Per-handler integration follows in T12/T13/T15 as those wrappers
land.

Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(onboard): types + render layer (A8, T12)

NEW src/core/onboard/types.ts: OnboardRecommendation (extends
RemediationStep with apply_policy + prompt_text + migration_id),
OnboardReport (stable JSON envelope), OnboardOpts.

NEW src/core/onboard/render.ts:
  toOnboardRecommendation(step): RemediationStep → OnboardRecommendation
    Sets apply_policy per A8 tiered rules:
      - protected + job === extract-takes-from-pages → 'manual_only' (A12/A24)
      - protected + other → 'prompt_required'
      - non-protected → 'auto_apply'
  buildOnboardReport(plan, opts?): assembles the stable JSON envelope.
  renderHuman(report): string. Echoes the "Recommendation + WHY" framing
    the CEO + Eng + Codex reviews settled on; CLI shell prints to stdout.

Stable JSON envelope shape:
  schema_version: 1
  brain_id?: string
  recommendations: OnboardRecommendation[]
  summary: { total, auto_eligible, prompt_required, manual_only,
             est_total_usd }
  history?: Array<{ remediation_id, metric_name, metric_before,
                    metric_after, delta, applied_at }>

Library-shaped — no console.* / process.exit. T13 (onboard CLI shell)
calls these from the wrapping CLI. MCP run_onboard (T16) returns the
JSON envelope unmodified.

Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(onboard): gbrain onboard CLI shell (A1, T13)

NEW src/commands/onboard.ts (~180 LOC). Thin wrapper that composes:
  - T2 library (computeRemediationPlan + runRemediation)
  - T4 onboard checks (runAllOnboardChecks → extraRemediations)
  - T12 render layer (buildOnboardReport + renderHuman)

Three modes:
  --check    (default): print plan, no submission. Computes plan via
             T2 library with T4 check-derived extraRemediations.
             Renders human (default) or JSON envelope (--json).
  --auto:    submit auto_apply tier. Requires --max-usd N (cron-safety
             per A12 + A20 — refuses without explicit cap to avoid
             surprise spend).
  --auto --yes: also submit prompt_required tier.
  --history: dump last 50 migration_impact_log entries.

Library hooks wired into stderr (per CLI/library separation): onStepStart,
onStepEnd, onBudgetRefused, onBudgetExhausted, onNothingToDo,
onTargetUnreachable. Final JSON envelope (--json) or human summary
lands on stdout.

CLI dispatch: registered in src/cli.ts CLI_ONLY set + case dispatch
between 'takes' and 'founder'.

Typecheck clean. Manual smoke-test pending T20 E2E (DATABASE_URL gated).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(onboard): init nudge + upgrade banner (A4, A18, A20, T14)

NEW src/core/onboard/init-nudge.ts exports two fail-open hooks:

runInitNudge(engine):
  Post-initSchema 5-query AbortSignal-bound parallel check against a
  3-second wallclock budget. Per A20: uses REAL cancellation via the
  T5 executeRaw signal extension — Promise.race against a timer was
  codex's #7 wrong shape. Postgres queries actually .cancel(); PGLite
  documented gap.
  Partial-results path: if some checks complete and the budget fires
  on others, prints what landed + a fallthrough hint pointing at
  `gbrain onboard --check` for the full picture.
  Per A18: fail-open — ANY throw is caught, logged to stderr, and
  suppressed so init returns successfully.
  Bypass: GBRAIN_NO_ONBOARD_NUDGE=1 short-circuits. Non-TTY default
  short-circuits too (CI/scripted callers see nothing).
  Nudge format: one-line summary of opportunities ("Brain has
  opportunities: 23000 stale chunks, link coverage 32%, 0 takes")
  + a 'gbrain onboard --check' nudge.

runUpgradeBanner(_engine):
  Lighter post-upgrade banner. Doesn't engine-query — just prints a
  one-line nudge that upgrades may surface new opportunities. Same
  fail-open posture.

Wired into:
  src/commands/init.ts:initPGLite (end-of-function, after reportModStatus)
  src/commands/init.ts:initPostgres (same)
  src/commands/upgrade.ts:runPostUpgrade (end-of-function, after
  postUpgradeReferenceSweep)

Each wire site uses dynamic import + try/catch so even an import
failure can't crash init/upgrade.

Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(autopilot): tick consults onboard recommendations (A5, A19, A22, T15)

Pre-fix: autopilot tick's per-source recommendation walk called
computeRecommendations(health, ctx) — doctor's hardcoded 5-category
planner. The 4 new onboard checks (embed_staleness,
entity_link_coverage, timeline_coverage, takes_count) had nowhere to
hook in, so even with takes.bootstrap_enabled flipped on, autopilot
never noticed 0 takes and never proposed bootstrap.

Post-fix: tick body now ALSO calls runAllOnboardChecks(engine) and
threads the result's RemediationStep[] into the T3-generalized third
arg of computeRecommendations. The planner merges onboard's extras
with the legacy hardcoded entries (hardcoded wins on id collision).

Per A19 fail-open: any throw in the onboard-checks path is caught,
logged to stderr, and suppressed. The legacy plan (without extras)
runs as before — autopilot can't crash from an onboard-check failure.

A22 (idempotency-key dedupe across concurrent manual + autopilot
runs): inherits from the existing computeRecommendations →
remediation.idempotency_key chain. T7-T9 handlers each get their
content-hash key from the makeRemediationStep factory; an autopilot
tick + a manual `gbrain onboard --auto` submitting the same step
in the same brain produce the SAME key, so queue.add(...) dedupes.

No behavior change for brains where all 4 onboard metrics already
look healthy (extras=[]; legacy plan unchanged).

Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(mcp): run_onboard op with run_protected_onboard scope binding (A7, T16, codex finding #5)

NEW MCP op `run_onboard`. Admin scope (NOT localOnly) so federated /
thin-client brain installs can probe brain health + submit auto-eligible
remediation handlers over OAuth-authenticated MCP.

Two-tier authorization per A7 + codex #5:
  - Admin scope: sufficient for mode='check' (read-only OnboardReport JSON)
    AND for submitting non-protected handlers in mode='auto'/'auto-with-prompt'.
  - run_protected_onboard scope (NEW, additive): MUST be granted in
    addition to admin for any PROTECTED_JOB_NAMES handler to fire
    (synthesize, patterns, consolidate, extract-takes-from-pages,
    contextual_reindex_per_chunk).

Without the new scope tier, an admin-scoped OAuth token would silently
bypass the same protected-name gate `submit_job` enforces at
operations.ts:2288. The codex finding #5 caught this: admin scope alone
was insufficient guard. Now the run_onboard op explicitly FILTERS
protected extras from the recommendation plan when the caller lacks
run_protected_onboard; filtered items appear in the response as
skipped_missing_scope[] so the caller knows what would have been
available with the right grants.

Modes:
  check               — read-only OnboardReport JSON envelope.
  auto                — submits auto_apply tier (plus prompt_required
                        when --yes/auto-with-prompt).
  auto-with-prompt    — adds prompt_required tier.

Both auto modes REQUIRE max_usd per A12 + A20 cron-safety (rejects
with invalid_params if missing).

Per A26 source-scope: future extension will scope plans by ctx.sourceId
/ ctx.auth.allowedSources. Today the recommendation planner is
brain-wide; the source-scope thread doesn't change correctness, just
optimization.

Per A19 fail-open: any error in runAllOnboardChecks during plan-build
caught + suppressed; the plan still returns with extras=[] rather than
crashing the op.

Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(verify): add check-source-scope-onboard lint (A26, T17)

NEW scripts/check-source-scope-onboard.sh. Grep guard for SQL sites in
onboard surfaces (src/core/onboard/, src/commands/onboard.ts) that
touch source_id-bearing tables (pages, content_chunks, takes, links,
timeline_entries) WITHOUT either:
  (a) source_id / sourceIds in the WHERE clause, OR
  (b) the opt-out marker `sourcescope:brain-wide` within 4 lines above
      the SQL.

File-level opt-out: `sourcescope:file-brain-wide` in the file header
(first 30 lines) treats every SQL site in that file as intentionally
brain-wide. Used by onboard/checks.ts, onboard/impact-capture.ts, and
commands/onboard.ts because the onboard CHECKS are explicitly brain-wide
aggregates (orphan_count, stale_count, link_coverage are reported
across all sources by design).

Wired into bun run verify (23 checks total now, all green).

Without this gate, any future onboard SQL touching per-source data
without source-scoping would silently leak rows across sources —
exactly the class of bug v0.34.1's P0 seal closed at the engine layer.
The lint adds an explicit forcing function for new code in the onboard
surface.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(install): onboard surface agent prescription (D13, T18)

Adds a v0.42.0+ section to INSTALL_FOR_AGENTS.md describing:
  - First-connect probe: gbrain onboard --check --json
  - Post-upgrade re-probe (after gbrain upgrade)
  - Unattended remediation: gbrain onboard --auto --max-usd 5
  - MCP run_onboard op for federated/thin-client installs
  - run_protected_onboard scope requirement for LLM-bearing handlers
  - Two-gate consent for takes-bootstrap (takes.bootstrap_enabled + --yes)
  - GBRAIN_NO_ONBOARD_NUDGE=1 bypass for CI

Per D13: agents should run --check on first connect AND after every
upgrade as a hygiene step. The autopilot path makes this auto-improve
on a 24h cycle; the explicit agent probe surfaces opportunities
immediately on connect rather than waiting for the next autopilot tick.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(e2e): hermetic onboard surface contracts (T20)

NEW test/e2e/onboard-full-flow.test.ts. 13 hermetic PGLite cases
(no DATABASE_URL needed) covering the key onboard contracts:

  captureMetric — all 5 metrics return expected values on empty brain
    (0 for counts; 1 for coverage = vacuous truth).

  runAllOnboardChecks — returns exactly 4 results with correct names;
    empty brain shows stale/link/timeline ok BUT takes_count warns
    (0 takes); 0 remediations emitted because takes.bootstrap_enabled
    defaults to false per A12 two-gate consent.

  computeRemediationPlan — extras (T3 generalization) thread through to
    plan.plan output; stable schema_version: 2 envelope.

  buildOnboardReport — stable schema_version: 1 envelope with the right
    summary fields populated.

  toOnboardRecommendation tier policy (A8):
    - non-protected job → auto_apply
    - extract-takes-from-pages → manual_only (A12 + A24)
    - other protected jobs (synthesize, patterns, ...) → prompt_required

Full DATABASE_URL-gated end-to-end (real Postgres, actual extractions
through Minion handlers) deferred to v0.42.1 once the per-handler test
seam lands; the hermetic suite covers the data-shape contracts that
matter for downstream consumers binding to the JSON envelopes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.42.0.0 gbrain onboard mega PR — activation surface (closes #1383, completes #1409)

VERSION + package.json bumped to 0.42.0.0. CHANGELOG with full ELI10 lead
+ "What you can do that you couldn't before" itemized list + "To take
advantage of v0.42.0.0" upgrade steps per CLAUDE.md voice rules.

TODOS.md: 9 follow-up items filed (TODO-A through TODO-I) for the
v0.42.1+ wave: pack-aware linkable types, LLM-disambiguation NER,
onboard --explain, live-brain impact measurement, 100+-case takes
classifier eval, admin SPA UI, full DATABASE_URL E2E, minion_jobs
client_id schema column, thin-client doctor-remote parity.

llms-full.txt regenerated per CLAUDE.md rule (every CHANGELOG edit
followed by bun run build:llms in the same commit).

23/23 verify checks pass.

Full implementation across 21 commits on this branch (T0-T21):
  T0  merge master
  T1  schema migrations v98/v99/v100
  T2  extract doctor remediation library
  T3  generalize computeRecommendations
  T4  4 new doctor checks
  T5  engine API: listStaleChunks orderBy + executeRaw AbortSignal
  T6  embed --batch-size / --priority recent / --catch-up
  T7  NER extraction + extract-ner handler
  T8  timeline-from-meetings + extract-timeline-from-meetings handler
  T9  takes-bootstrap + extract-takes-from-pages handler
  T10 recordMinionJobSpend primitive
  T11 impact capture module + writeImpactLogRow
  T12 onboard render layer (types + render)
  T13 gbrain onboard CLI shell
  T14 init nudge + upgrade banner
  T15 autopilot tick consults onboard
  T16 MCP run_onboard + run_protected_onboard scope
  T17 check-source-scope-onboard lint
  T18 INSTALL_FOR_AGENTS.md agent prescription
  T20 hermetic PGLite E2E (13 cases)
  T21 ship (this commit)

Reviews: CEO + Eng + Codex on plan
~/.claude/plans/system-instruction-you-are-working-lively-hollerith.md.
27 A-decisions locked; 18 codex findings absorbed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ci): connection-resilience regex + doctor warn-not-fail + v0.41.18.0

Two CI fixes from PR #1521 + version renumber per user request.

Why fix #1 (connection-resilience.test.ts): T5/A20 extended
PostgresEngine.executeRaw signature to accept an optional
`opts?: { signal?: AbortSignal }` 3rd arg and rewrote the body as
multi-line. The regression test's regex was anchored to the legacy
single-line `(sql: string, params?: unknown[])` shape and the
assertions banned `try {` / `catch` (which T5 legitimately added for
AbortSignal cancellation swallow, NOT for retry). Updated regex to
tolerate both shapes; replaced the wrong `not.toContain('conn.unsafe(
sql, params')` assertion (which incorrectly flagged the legitimate
single call) with a count assertion: `conn.unsafe(` must appear
exactly ONCE in the body. Preserves the original D3 intent (no
per-call retry — recovery is supervisor-driven via reconnect()) while
accepting the new try/catch shape that swallows AbortSignal aborts.

Why fix #2 (src/core/onboard/checks.ts): Three of the four new
onboard doctor checks (entity_link_coverage, timeline_coverage,
embed_staleness) emitted `status = 'fail'` on healthy DBs that simply
hadn't run extractions yet. This flipped `gbrain doctor`'s exit code
to non-zero on freshly initialized brains, breaking
test/e2e/mechanical.test.ts:1280 ("gbrain doctor exits 0 on healthy
DB"). Downgraded all three to `status = 'warn'` — these are
remediation opportunities, not assertion failures. Doctor exit
codes are reserved for actual failures; remediation surfaces use
warn-level signaling so they can be picked up by `--remediate`
without polluting the exit code.

Why fix #3 (version renumber 0.42.0.0 → 0.41.18.0): Per user
directive, this wave ships as v0.41.18.0 rather than v0.42.0.0.
Master is at 0.41.16.0; 0.41.17.0 is reserved for an in-flight
wave. Renamed every reference my branch added (54 files touched):
VERSION, package.json, CHANGELOG.md header, TODOS.md, plus inline
version-stamp comments across src/, test/, and scripts/. Preserved
13 files with PRE-EXISTING `v0.42.0.0` references on master (from
earlier waves originally planned for v0.42 that landed at v0.41.x —
those stay as historical record). Verified via per-file diff against
origin/master: every renamed reference is one I added in this branch.

Audit trio aligned: VERSION=0.41.18.0, package.json=0.41.18.0,
CHANGELOG topmost entry=[0.41.18.0]. llms-full.txt regenerated to
match CLAUDE.md updates.

Bisect contract: this commit fixes CI test failures from PR #1521's
landing. Typecheck clean; connection-resilience suite 26/26 pass.

Refs A20 (executeRaw AbortSignal), A16 (4 new onboard checks),
codex #1 (master collision avoidance via renumber).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 18:59:17 -07:00

GBrain

Search gives you raw pages. GBrain gives you the answer. It's the brain layer your AI agent has been missing — the only one that does synthesis, graph traversal, and gap analysis in one box.

I'm Garry Tan, President and CEO of Y Combinator. I built GBrain to run my own AI agents. It's the production brain behind my OpenClaw and Hermes deployments: 146,646 pages, 24,585 people, 5,339 companies, 66 cron jobs running autonomously. My agent ingests meetings, emails, tweets, voice calls, and original ideas while I sleep. It enriches every person and company it encounters. It fixes its own citations and consolidates memory overnight. I wake up smarter than when I went to bed — and so will you.

And now it works as a company brain too. Each person on the team gets their own slice of the brain, scoped by login. When you query, you only see what you're allowed to see — never another person's notes, never another team's data. We fuzz-tested this across every way you can read the brain (search, list, lookup, multi-source reads) and got zero leaks. Drop GBrain in as your team's shared institutional memory — the company-brain shape YC just put on its Request for Startups. If you're building in that space, you might as well build on this. Tutorial: set up GBrain as your company brain →

Lots of personal-knowledge systems give you keyword matching and grep in a box. GBrain does that, and adds two things nobody else ships together:

  • A synthesis layer that gives you the actual answer. Synthesized, well-cited prose across people, companies, deals, and ideas. Not "here are 10 chunks that mention your query"; an actual answer with citations and an explicit note on what the brain doesn't know yet. The gap analysis is the part that changes how you use the brain.
  • A self-wiring knowledge graph. Every page write extracts entity refs and creates typed edges (attended, works_at, invested_in, founded, advises) with zero LLM calls. Ask "who works at Acme AI?" or "what did Bob invest in this quarter?" and get answers vector search alone can't reach. Benchmarked: P@5 49.1%, R@5 97.9% on a 240-page Opus-generated rich-prose corpus, +31.4 points P@5 over its graph-disabled variant and over ripgrep-BM25 + vector-only RAG by a similar margin. Full BrainBench scorecards live in the sibling gbrain-evals repo.

The point of building a 100K-page brain is to use it as a strategic moat. To never lose context. To query what's in your own head without re-reading it. The brain layer is what makes the moat usable. The 24/7 dream cycle is what keeps it sharp. Both run on your hardware, your DB, your keys.

It's easier to ship a daemon that runs 24/7 to ingest, enrich, and consolidate than it is to keep an agent in chat working hard. GBrain is that daemon, generalized. Install in 30 minutes. Your agent does the work. As my personal agent gets smarter, so does yours.

~30 minutes to a fully working brain. Database ready in 2 seconds (PGLite, no server). You just answer questions about API keys.

LLMs: fetch llms.txt for the documentation map, or llms-full.txt for the same map with core docs inlined in one fetch. Agents: start with AGENTS.md (or CLAUDE.md if you're Claude Code).

What this looks like

Say you have a meeting with Alice tomorrow. You want to walk in remembering what she works on, when you last talked, and what's still open between you. Here's what you'd type, and what you'd get back.

You ask:

"What do I need to know before my meeting with Alice tomorrow?"

Most personal-knowledge tools give you back a list of pages. Something like:

1. people/alice — Alice runs engineering at Acme...
2. meetings/2026-03-15-alice-q1 — Q1 product review with Alice...
3. meetings/2026-01-08-acme-kickoff — Kickoff meeting with Acme team...
4. customers/acme — Acme is a series-B fintech we work with...
5. notes/2026-04-22 — Quick chat with Alice about pricing...

Five pages you now have to open and read yourself to actually prepare. The tool found the right material, but it didn't do the work.

GBrain gives you back the answer, with sources:

Alice runs engineering at Acme (a series-B fintech). You last spoke
on April 22 in a quick pricing chat. Three things are still open
from that conversation:

1. She owes you the security review for the new tier
   (deadline was May 1; no update since).
2. You committed to pricing for a 500-seat tier
   (you sent it April 25; no response yet).
3. She mentioned they're hiring a CISO; you said you'd intro
   someone from your network.

Heads up: nothing's been added to the brain about Alice or Acme
since April 22, six weeks ago. She may have replied through email
or Slack DM, channels the brain doesn't see. Worth asking her to
catch up before assuming any of this is still current.

Every claim has a source page behind it. The "heads up" at the end tells you what the brain doesn't know yet, so you can ask Alice about it directly instead of being surprised. The brain just did your meeting prep.

This is the difference between a search engine and a brain. Search finds the pages. The brain reads them for you and writes the answer.

Install

GBrain is designed to be installed and operated by an AI agent. The fastest path is to have your agent do it for you. The CLI and MCP paths below are for people who want to wire it up themselves.

If you don't already have an AI agent platform running, start with one of these. Both are designed to read GBrain's install protocol and execute it:

Then paste this into your agent:

Retrieve and follow the instructions at:
https://raw.githubusercontent.com/garrytan/gbrain/master/INSTALL_FOR_AGENTS.md

The agent installs GBrain, creates the brain, asks for your API keys, loads 43 skills, configures the dream cycle, and verifies the install end-to-end. ~30 minutes. You answer questions, it does the work.

Never set up an AI agent platform before? The personal-brain tutorial walks the whole path end-to-end — picking OpenClaw vs Hermes, deploying it, pointing it at INSTALL_FOR_AGENTS.md, getting the API keys, and verifying the first query. Start there if any of the above is new.

Install it into your existing agent

Already running Codex, Claude Code, Cursor, or another coding agent? Paste the same instruction in:

Retrieve and follow the instructions at:
https://raw.githubusercontent.com/garrytan/gbrain/master/INSTALL_FOR_AGENTS.md

This works in any agent that can read files over HTTPS and execute shell commands. Tested with Codex, Claude Code, Claude Cowork, Cursor, and AlphaClaw.

CLI standalone (no agent)

bun install -g github:garrytan/gbrain
gbrain init --pglite     # 2 seconds; no server, no Docker
gbrain doctor            # verify health
gbrain import ~/notes/   # index your markdown
gbrain query "what themes show up across my notes?"

Postgres-at-scale, Supabase, and thin-client setup paths live in docs/INSTALL.md.

Connect GBrain to your AI client (MCP)

GBrain exposes 30+ tools over MCP (stdio and HTTP). The specific snippet depends on which client you use:

  • Claude Code — one command: claude mcp add gbrain -- gbrain serve. Zero server, zero tunnel.
  • Cursor / Windsurf / any stdio MCP client — same shape, add {"command": "gbrain", "args": ["serve"]} to your MCP config.
  • Claude Desktop (Cowork) — Settings → Integrations → add the URL of your HTTP server. Remote only; the local claude_desktop_config.json does not work for remote servers.
  • Claude Cowork (team plan) — org Owner adds the connector under Organization Settings → Connectors.
  • Perplexity Computer — Settings → Connectors → add the URL + bearer token. Pro subscription required.
  • ChatGPT — uses OAuth 2.1 with PKCE (the hard requirement). Register a chatgpt client from the admin dashboard with grant type authorization_code.

For the HTTP server itself:

gbrain serve              # stdio MCP (local subprocess; for Claude Code, Cursor, Windsurf)
gbrain serve --http       # HTTP MCP with OAuth 2.1 + admin dashboard at /admin
                          # (required for Claude Desktop, Cowork, Perplexity, ChatGPT)

The HTTP server includes DCR-style client registration, scope-gated access (read / write / admin), and rate limiting. Deployment guides (ngrok, Railway, Fly.io) live under docs/mcp/.

Two ways to query your brain

Raw retrieval (what most personal-knowledge tools ship) and a synthesis layer that gives you an actual answer. They serve different jobs.

# raw retrieval: top pages by hybrid score, fast, no LLM cost
gbrain search "who's working on AI agents at portfolio companies?"

# brain layer: synthesized answer with citations and gap analysis
gbrain think "who's working on AI agents at portfolio companies?"

gbrain search returns the top retrieved pages, ranked by hybrid scoring (vector + keyword + RRF + source-tier boost + reranker). Use it when you want raw material to skim: agent context windows, citation lookups, finding a specific quote.

gbrain think runs the same retrieval, then composes a synthesized answer across the results with explicit citations to the source pages AND an honest note on what the brain doesn't know yet. The gap analysis is the differentiator: the answer tells you when a page is stale, when a claim is uncited, when two pages contradict each other, when there's a hole you should fill.

Why it compounds. Pair the brain layer with find_trajectory and you get answers like "how have the company's metrics changed AND what does the team look like right now AND what did they promise / share AND when did we last meet AND what's the value-add I can offer here": well-scored, well-cited, in one shot. That's the strategic moat. That's why building a 100K-page brain is worth the effort.

gbrain agent run "..." exposes the same surface to a sub-agent through the Minions queue, with crash-safe two-phase persistence. Same answers, durable.

How to get data in

One command, local or hosted, synchronous receipt:

gbrain capture "the thought I want to remember"
gbrain capture --file ./notes/today.md
echo "from a pipe" | gbrain capture --stdin
SLUG=$(gbrain capture "..." --quiet)

The page lands in the database and on disk in one move. Default slug inbox/YYYY-MM-DD-<hash8> so captures cluster in a predictable triage location. On thin-client installs the verb routes through MCP to the server: same command, same UX.

For webhook ingestion (Zapier / IFTTT / Apple Shortcuts):

curl -X POST https://your-brain/ingest \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: text/markdown" \
  -d "# a thought from a Shortcut"

For mobile capture, the inbox folder source picks up anything dropped into ~/.gbrain/inbox/ from iOS Shortcuts / AirDrop / Drafts / Finder.

Third-party skillpacks can ship custom ingestion sources (Granola, Linear, voice, OCR) against the versioned IngestionSource contract at gbrain/ingestion. See docs/skillpack-anatomy.md.

Your brain's shape (schema packs)

Most personal-knowledge tools force one fixed layout: their idea of "notes" + "people" + "tags." Drop a Notion export or your own years-old Obsidian vault on top, and the agent doesn't know what a Projects/ folder means or whether Reading/ is people or sources.

gbrain doesn't have a fixed layout. It ships with two bundled schema packs and lets you author your own when neither fits:

  • gbrain-base (default) — the layout my production brain uses: people/, companies/, concepts/, meetings/, deal/, daily/, originals/, writing/, etc. Zero config. Drop a brain that fits this shape and everything works.
  • gbrain-recommended — extends gbrain-base with the 13 additional directories from docs/GBRAIN_RECOMMENDED_SCHEMA.md (source, place, trip, conversation, personal, civic, project, etc.). Activate with gbrain schema use gbrain-recommended.
  • Your own packgbrain schema detect clusters your actual filesystem into proposed types, gbrain schema suggest runs an LLM pass over them, and gbrain schema review-candidates --apply promotes the ones you like. Three commands and the brain knows your shape.
gbrain schema active                # which pack is running, which tier set it
gbrain schema list                  # bundled + installed packs
gbrain schema detect                # propose types matching your filesystem
gbrain schema suggest               # LLM-refined proposals on top of detect
gbrain schema review-candidates     # human gate: promote / rename / ignore
gbrain schema use my-pack           # activate

The active pack threads through every read + write path: parseMarkdown infers page type from the pack's path prefixes; whoknows scopes expert routing to types declared expert_routing: true; extract_facts runs only on extractable: true types; the search cache folds the pack name + version into its key so cross-pack contamination is structurally impossible. Switch packs and the brain re-interprets itself; switch back and nothing's lost.

Seven-tier resolution chain (per-call flag → env var → per-source DB key → brain-wide DB key → gbrain.yml~/.gbrain/config.jsongbrain-base default). Full reference + authoring guide: docs/architecture/schema-packs.md.

Tutorials

Step-by-step walkthroughs for getting the most out of GBrain. Each one takes you from zero to a working outcome, with concrete commands and real numbers.

More walkthroughs in progress: connecting an existing agent (Claude Code, Cursor, OpenClaw, Hermes) to a GBrain memory layer; setting up GBrain for VC dealflow with founder scorecards and meeting prep; migrating an existing Notion or Obsidian vault; indexing a codebase as a queryable code brain. Full tutorial index: docs/tutorials/.

Want to see a tutorial that isn't here yet? Open an issue describing the workflow you want documented.

What it does (the loop)

  signal   →   search   →   respond   →   write   →   auto-link   →   sync
  (every    (brain-first  (informed     (page +    (typed edges     (cron
  message)  retrieval)    by context)   timeline)  + backlinks)     keeps fresh)
  • Signal detector runs on every message your agent receives. Captures ideas, entity mentions, time-sensitive todos, names, links.
  • Brain-first lookup before any external API call. The cheapest, fastest, most personal information source you have.
  • Auto-link fires on every page write. No LLM calls; pure pattern matching on [[wiki/people/bob]] style references. New entity → new page stub → graph grows.
  • Cron-driven enrichment runs while you sleep: dedup people pages, fix citations, score salience, find contradictions, prep tomorrow's tasks.

The whole loop is described in docs/architecture/topologies.md with diagrams.

Capabilities

Hybrid search. Vector (HNSW on pgvector) + BM25 keyword + reciprocal-rank fusion + source-tier boost + intent-aware query rewriting. Three named search modes (conservative, balanced, tokenmax) bundle the cost/quality knobs into a single config key. Live cost/recall comparisons in docs/eval/SEARCH_MODE_METHODOLOGY.md. Default: balanced with ZeroEntropy reranker on. Per-query graph signals notice when a top result is a hub for THAT query (adjacency boost), is corroborated across team brains (cross-source boost), or is being crowded out by weak chunks from a chatty session (session demote). Run gbrain search "<query>" --explain to see per-stage attribution: base score, every boost that fired, what it multiplied. gbrain doctor ships a graph_signals_coverage check; gbrain search stats shows fire counts and failure breakdowns.

Self-wiring knowledge graph. Every put_page extracts entity refs from markdown/wikilinks/typed-link syntax and writes edges with zero LLM calls. Typed edges (attended, works_at, invested_in, founded, advises, mentions, …). Multi-hop traversal via gbrain graph-query. The graph is what produces the +31.4 P@5 lift over vector-only RAG.

Job queue (Minions). BullMQ-shaped, Postgres-native job queue. Durable subagents (LLM tool loops that survive crashes via two-phase pending→done persistence), shell jobs with audit, child jobs with cascading timeouts, rate leases for outbound providers, attachments via S3/Supabase storage. Replaces "spawn subagent as fire-and-forget Promise" with something that recovers from anything.

43 curated skills. Routing lives in skills/RESOLVER.md. Covers signal capture, ingest (idea / media / meeting), enrichment, querying, brain ops, citation fixing, daily task management, cron scheduling, reports, voice, soul audit, skill creation, eval framework, and migrations. Skills are markdown files (tool-agnostic), packaged as a single skillpack the installer drops into your agent workspace.

Eval framework. gbrain eval longmemeval runs the public LongMemEval benchmark against your hybrid retrieval. gbrain eval export + gbrain eval replay capture real queries and replay them against code changes (set GBRAIN_CONTRIBUTOR_MODE=1). gbrain eval cross-modal cross-checks an output against the task using three different-provider frontier models. Full methodology in docs/eval/SEARCH_MODE_METHODOLOGY.md.

Brain consistency. gbrain eval suspected-contradictions samples retrieval pairs, layered date pre-filter, query-conditioned LLM judge, persistent cache. Surfaces conflicts between takes + facts the agent has written. Wired into the daily dream cycle.

Agent-authored schema (v0.40.7.0). Your brain has a shape — what page types exist (person, meeting, paper, case, lab-result), what they link to (attended, authored, prescribed-by), what facts get extracted automatically. The default ships with 22 universal types, but your brain's actual shape is not the default shape. Agents can now evolve that shape on your behalf via 14 gbrain schema CLI verbs + a batched MCP op (schema_apply_mutations, admin scope, NOT localOnly so remote agents reach it over HTTPS). Atomic file locks, audit log with the agent's identity, chunked UPDATE backfill in 1000-row batches that never wedge concurrent writers. The brain stops being a pile of notes and becomes something with structure. Why it matters: docs/what-schemas-unlock.md — 7 killer use cases (4000 invisible meetings, founder ops brain, research brain, legal brain, team brain, agent-as-co-curator). 5-minute walkthrough: docs/schema-author-tutorial.md. Agent skill: skills/schema-author/SKILL.md.

Integrations

Data flowing into the brain. Each integration is a recipe — markdown + setup hints — that ships in recipes/ and is discoverable via gbrain integrations list.

  • Voice: Phone calls create brain pages via Twilio + OpenAI Realtime (or DIY STT+LLM+TTS). Setup recipe: recipes/twilio-voice-brain.md.
  • Email + calendar: webhook handlers that route to brain signals. docs/integrations/meeting-webhooks.md.
  • Embedding providers: 16 recipes covering OpenAI (default fallback), OpenRouter, Voyage, ZeroEntropy (default), Google Gemini, Azure OpenAI, MiniMax, Alibaba DashScope, Zhipu, Ollama (local), llama.cpp llama-server (local), LiteLLM proxy. Pricing matrix + decision tree in docs/integrations/embedding-providers.md.
  • Rerankers: ZeroEntropy zerank-2 hosted (default in tokenmax mode) plus the v0.40.6.1 llama-server-reranker recipe for fully-local cross-encoder rerank via llama.cpp — runs Qwen3-Reranker or self-hosted ZeroEntropy weights against the same gateway.rerank() seam. Setup walkthrough in docs/ai-providers/llama-server-reranker.md.
  • Credential gateway: vault-aware secret distribution. docs/integrations/credential-gateway.md.
  • MCP clients: every major MCP client is supported. docs/mcp/ per-client setup.

Architecture

Two engines, one contract. PGLite (Postgres 17 via WASM, zero-config, default) for personal brains up to ~50K pages. Postgres + pgvector (Supabase or self-hosted) for shared / large / multi-machine deployments. The contract-first BrainEngine interface in src/core/engine.ts defines ~47 operations both engines implement; CLI and MCP server are generated from one source.

Brain repo is the system of record. Your knowledge lives in a regular git repo (your "brain repo") as markdown files. GBrain syncs the repo into Postgres for retrieval; deletes in git become soft-deletes in DB. You can publish public subsets, share team mounts, run thin-client setups pointing at a colleague's brain server. Topologies in docs/architecture/topologies.md.

Two organizational axes (brain ⊥ source). A brain is a database (your personal brain, a team mount you joined). A source is a repo inside that brain (wiki, gstack, an essay, a knowledge base). Routing lives in .gbrain-source dotfiles and resolves via a documented 6-tier precedence chain. Full diagrams in docs/architecture/brains-and-sources.md.

Why the graph matters. Vector search returns chunks that are semantically close. The graph returns chunks that are factually connected. Hybrid search pulls from both; auto-linking on every write keeps the graph fresh. Deep dive: docs/architecture/RETRIEVAL.md.

Troubleshooting

gbrain import fails with expected N dimensions, not M? Run gbrain doctor. It will print the exact gbrain config set ... or gbrain retrieval-upgrade command to repair the mismatch. You should not need to delete ~/.gbrain. Fresh gbrain init --pglite auto-detects your embedding provider from API keys in your environment: set OPENAI_API_KEY (or ZEROENTROPY_API_KEY / VOYAGE_API_KEY) before running init, or pass --embedding-model <provider>:<model> explicitly. With multiple keys set, init fires an interactive picker. In non-TTY contexts (CI, Docker) with no keys, init exits 1 with a paste-ready setup hint; pass --no-embedding to defer setup until runtime. See docs/integrations/embedding-providers.md for the full provider matrix and docs/operations/headless-install.md for Docker/CI sequencing.

Hourly cron sync keeps timing out on a federated brain? v0.41.13.0 ships two flags + a recommended pattern. Switch your cron to a per-source loop with shell timeout(1) doing the OS-level kill and gbrain self-terminating gracefully half-a-minute earlier:

gbrain sync --break-lock --all --max-age 1800
for src in $(gbrain sources list --json | jq -r '.[].id'); do
  timeout 600 gbrain sync --source "$src" --timeout 540 || true
done

When --timeout fires mid-import, gbrain sync exits 0 with status partial and last_commit UNCHANGED — the next run re-walks the same diff and content_hash short-circuits already-imported files. The --max-age 1800 first command self-heals any wedged-but-alive locks left by a hung previous run, using the v98 last_refreshed_at semantic (NOT acquired_at) so healthy long-running holders are safe by construction. See the v0.41.13.0 entry in CHANGELOG.md for the honest scope notes (extract + embed phases run to completion; 30-min rollout window for --max-age post-migration v98; full-sync triggers deferred to v0.42+).

Docs

  • docs/INSTALL.md — every install path, end to end
  • docs/what-schemas-unlock.md — why schemas matter: 7 killer use cases, the structural argument for typed page kinds, the agent-co-curates pattern (v0.40.7.0)
  • docs/schema-author-tutorial.md — 5-minute walkthrough: fork the bundled pack, add a custom type, backfill existing pages, prove the wiring via gbrain whoknows
  • docs/architecture/ — system design, topologies, retrieval theory
  • docs/guides/ — how-to runbooks (sub-agent routing, minion deployment, skill development, brain-first lookup, idea capture, diligence ingestion)
  • docs/integrations/ — connecting external data sources (voice, email, calendar, embedding providers)
  • docs/mcp/ — per-client MCP setup (Claude Desktop, Code, Cursor, ChatGPT, Perplexity, Cowork)
  • docs/eval/ — eval framework, metric glossary, methodology
  • docs/ethos/ — philosophy (thin harness, fat skills, markdown as recipes, origin story)
  • AGENTS.md — entry point for non-Claude agents
  • CLAUDE.md — entry point for Claude Code (deep operating context)
  • CONTRIBUTING.md — contributor guide, test discipline, eval-capture mode
  • SECURITY.md — OAuth threat model, hardening defaults

Contributing

Run bun run test for the fast loop, bun run verify for the pre-push gate, bun run ci:local to run the full Docker-backed CI stack locally. Detailed test discipline in CONTRIBUTING.md.

Community PRs are batched into release waves rather than merged one-by-one — see the "PR wave workflow" section in CLAUDE.md. Contributor attribution stays attached via Co-Authored-By: trailers. We credit every accepted contribution in CHANGELOG.md.

If you find a bug or want a feature: open an issue first. Quick fixes (typo, doc bug, obvious regression) can go straight to a PR. Anything touching schema, retrieval ranking, MCP protocol, or the security boundary needs a design discussion in the issue first.

License + credit

MIT. I built GBrain to run my OpenClaw and Hermes deployments — the production brain behind my AI agents.

Origin story: docs/ethos/ORIGIN.md.

Community PR contributors are credited in CHANGELOG.md per release. ZeroEntropy (@zeroentropy) for the embedding + reranker stack that ships as the default. Voyage AI for the asymmetric-encoding recipe template. Ramp Labs for the search quality improvements lineage.

S
Description
No description provided
Readme MIT
160 MiB
Languages
TypeScript 97.4%
Shell 1.2%
JavaScript 0.9%
PLpgSQL 0.4%