* feat(skillopt): foundation modules — types, lr-schedule, benchmark, score, audit, lock
* feat(skillopt): edit primitives — apply-edits (D5+D9), rejected-buffer LRU, version-store (D8 history-intent-first)
* feat(skillopt): rollout (D2 gateway.toolLoop + D13 read-only allowlist), reflect (D7 two calls), validate-gate (D12 median+epsilon, D4 parallel), preflight (D3), bundled-skill-gate (D16)
* feat(skillopt): orchestrator (D6 slow-update, D10 ASCII diagrams, D11 caching), checkpoint, bootstrap (D15 sentinel), CLI dispatch + help
* feat(skillopt): cycle phase (F1 dream-loop wiring), PROTECTED_JOB_NAMES + MCP op (F6 admin scope + allowlist) + Minion handler (F7 --background)
* feat(skillopt): full cathedral — --all batch (F4), --target-models fleet (F5), write-capture (F10), held-out scaffold (F11), adversarial suite 41 cases (F2), E2E PGLite (F3), meta-skill bundle (T7), reflect+judge evals (F8+F9), docs (T10)
* chore: bump version to v0.42.0.0 (MINOR — significant new feature)
* fix(skillopt): wire trajectories from forward gate to reflect + fix parseEditsResponse parser misuse
Two related v0.42.0.0 bugs that conspired to make `runSkillOpt` structurally
unable to accept any candidate edit. Either alone would have killed self-evolution;
together they made the loop a no-op for every input.
**Bug 1 (orchestrator gap):** `runOptimizationLoop` in orchestrator.ts called
`runReflect({successes: [], failures: []})` with hardcoded empty arrays. The
forward gate's `scoredRollouts` were computed then voided. `runReflect`
short-circuits both modes when their batches are empty, so the optimizer was
never asked to propose an edit. Every step hit the no_edits_applied branch.
Fix: add `scoredRollouts: ScoredRollout[]` to `GateResult` and
`runsPerTask?: number` to `ValidateGateOpts`. Forward pass uses
`runsPerTask: 1`; orchestrator partitions returned rollouts by `score >= 0.5`
and threads real successes + failures into `runReflect`.
**Bug 2 (parser misuse):** `parseEditsResponse` in reflect.ts routed every
optimizer response through `parseJudgeJson` first. `parseJudgeJson` looks for
a `score` key (it's a judge-output parser, not an edits parser) and returns
null for any JSON without one — including the well-formed `{"edits": [...]}`
the optimizer is contractually required to emit. The function then early-
returned `[]` and the actual `tryExtractEdits` path on the next line was
unreachable dead code.
Fix: drop the wrong-typed guard. `parseEditsResponse` now calls
`tryExtractEdits` directly. Export it so `reflect.test.ts` can pin the
contract independently of the chat transport.
**Why this slipped through 152 prior skillopt tests:** zero unit coverage
of `parseEditsResponse` or `runReflect`. The existing E2E `all-reject` case
asserted no_improvement (which was true for the wrong reason — empty edits,
not gate rejection). Both bugs were structurally invisible to the existing
test surface.
**New coverage:**
- `test/skillopt/reflect.test.ts` (15 cases):
- 8 `parseEditsResponse` cases including the IRON-RULE regression pin
for the v0.42.0.1 fix (`{"edits": [...]}` JSON must survive the parser).
- 7 `runReflect` D7 contract cases: both modes fire, empty-batch skips,
additive token usage, one-mode-throws-other-still-works, rejected-buffer
flows into anti-bias prompt.
- Documents the trailing-comma limitation as an explicit out-of-scope pin
(so a future tightening of `tryExtractEdits` lights this test up
intentionally).
- `test/e2e/skillopt-loop.serial.test.ts` (7 cases):
- HAPPY PATH: stubbed `gateway.chat` acts as both target agent (emits
sections based on skill content) and optimizer (proposes a real
add-Citations edit). Drives `runSkillOpt` end-to-end against PGLite.
Asserts outcome=accepted, SKILL.md mutated with new section,
frontmatter preserved (D5), history has one committed row,
best.md mirrors disk, delta > epsilon, receipt fields populated.
- 5 broken cases (each isolates a distinct orchestrator-visible failure):
1. Below-baseline regression: optimizer proposes a destructive edit;
gate rejects with reason=below_baseline; SKILL.md unchanged;
rejected-buffer captures the bad edit for anti-bias context.
2. Malformed reflect JSON: orchestrator degrades gracefully to
no_improvement without crashing.
3. Anchor-not-found: applyEditBatch rejects all; sel gate skipped;
rejected-buffer captures with reason=apply_failed.
4. Budget exhausted mid-step: outcome=aborted, no pending rows survive.
5. Converged-skill re-run: starting from already-perfect skill →
no_improvement (no thrash on a well-tuned starting point).
- IDEMPOTENT RE-RUN: drive runSkillOpt twice in sequence. Run 1 accepts.
Run 2 sees improved baseline, no failures, returns no_improvement.
SKILL.md byte-identical to post-run-1; history still has exactly 1
committed row. Proves stability at the fixed point.
All hermetic (no DATABASE_URL, no API keys). PGLite in-memory engine,
tempdir SKILL.md + benchmark, stubbed gateway.chat via
`__setChatTransportForTests`. `.serial.test.ts` because the stub installs
module state and the loop walks shared disk state across epochs.
Test counts after fix: 174 skillopt-surface tests pass (149 pre-existing
unit + 15 new reflect unit + 3 existing E2E + 7 new E2E). Typecheck clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cycle): align ALL_PHASES skillopt position with actual dispatch order
v0.42.0.0 added skillopt to ALL_PHASES right after `patterns` (line 127), but
the dispatch block in runCycle (line ~1912) actually runs skillopt between
`conversation_facts_backfill` and `embed`. The two were inconsistent, and the
serial test `report.phases.map(p => p.phase)).toEqual(ALL_PHASES)` was failing
on master because of it.
A second pre-existing failure: the two phase-count assertions in
`test/core/cycle.serial.test.ts` still said `toBe(20)` even though
ALL_PHASES grew to 21 when skillopt was added. The author bumped the array
but forgot the test.
Two fixes, one commit:
1. Move `'skillopt'` in ALL_PHASES from after `patterns` to between
`conversation_facts_backfill` and `embed`, matching where runCycle
actually dispatches it. Runtime behavior is unchanged — only the
declaration order moves. Updated the surrounding comment to call out
the position invariant and reference the test that pins it.
2. Update both `toBe(20)` assertions in cycle.serial.test.ts to `toBe(21)`
with a v0.42.0.0 history line in the running comments.
Why declaration follows runtime (not the other way around): the comment
intent ("Runs AFTER patterns — graph-fresh") is still satisfied because
"after the entire main graph-mutating cluster" is strictly fresher than
"right after patterns". No design intent is lost.
Test result: cycle.serial.test.ts is now 28/28 (was 27/28 on master + my
prior commit). Skillopt suite still 174/174.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ci): bump PHASE_SCOPE assertion to 21 + fix skill-optimizer Anti-Patterns case
Two CI failures pre-existing on this branch since the v0.42.0.0 skillopt
cathedral landed; master is green because skillopt didn't exist there yet.
1. test/phase-scope-coverage.test.ts asserted ALL_PHASES.length === 20.
skillopt is the 21st phase. Bumped to 21 with v0.42.0.0 history line
in the comment chain. Sibling fix to the cycle.serial.test.ts bump
in commit 08ad2468.
2. skills/skill-optimizer/SKILL.md had `## Anti-patterns` (lowercase p).
skills-conformance.test.ts asserts `## Anti-Patterns` (capital P) as
the required section header. Single-character rename.
Local: 174 skillopt-surface tests + 6 phase-scope tests + 249 skills-
conformance tests all green. Typecheck clean.
Remaining CI delta: 5 put_page facts backstop failures in shard 10 that
reproduce only on Linux CI, not locally even with empty env / cleared
HOME / max-concurrency=1. The error surface is `r.isError === true` with
no further detail captured in the bun:test output. Pushing these 2 fixes
first to narrow the CI signal; will instrument if the 5 persist.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(e2e): align dream-cycle-phase-order + onboard-full-flow with v0.41/v0.42 reality
Two stale E2E assertion files surfaced by a full local E2E run against
real Postgres (the gbrain-test-pg container on port 5434). Neither file
is in the CI E2E job (CI only runs mechanical.test.ts + mcp.test.ts +
skills.test.ts + zeroentropy-live.test.ts), so the drift has been latent.
1. `test/e2e/dream-cycle-phase-order-pglite.test.ts`
EXPECTED_PHASES was missing 4 phases that landed in master since the
list was last revised:
- extract_atoms (v0.41 T9 — atom extraction, after extract_facts)
- synthesize_concepts (v0.41 T9 — concept synthesis, after patterns)
- conversation_facts_backfill (v0.41.11.0, after calibration_profile)
- skillopt (v0.42.0.0 — self-evolving skills, between
conversation_facts_backfill and embed)
Updated to 21 entries in the actual runtime dispatch order (matches
ALL_PHASES exactly). 5/5 tests in the file pass after.
2. `test/e2e/onboard-full-flow.test.ts`
`runAllOnboardChecks` shape test asserted exactly 4 checks; v0.42's
type-unification cathedral (PR #1542, T13-T15) added 3 more
(`pack_upgrade_available`, `type_proliferation`, `dangling_aliases`)
for a total of 7. And `empty brain returns 0 remediations` regressed
because `pack_upgrade_available` can emit a manual_only remediation
on brains where gbrain-base@1.x is active and gbrain-base-v2 is
registered as a successor. Tightened that assertion to `total <= 1`
AND kept a per-check guard asserting takes_count remediations stay 0
(the original test's load-bearing claim — A12 two-gate consent).
13/13 tests in the file pass after.
Honest scope: 4 other E2E files still fail locally after this commit
(cycle.test.ts, dream.test.ts, phantom-redirect.test.ts,
sync-lock-recovery.test.ts), each for a distinct pre-existing master
bug unrelated to v0.42 skillopt work:
- cycle.test.ts (5 fails): PostgresEngine.getConfig falls back to
db.getConnection() singleton via the `get sql()` getter when no
poolSize is set; the new conversation_facts_backfill phase chain
hits this fallback even though the test's setupDB() connects both
the singleton AND the engine. Race condition between the test's
singleton lifecycle and the phase's getConfig call. Deeper fix
needed in PostgresEngine.getConfig (use this._sql directly with
explicit fallback only on user-driven CLI paths).
- dream.test.ts (1 fail): expects "concepts/testing" slug to appear
in dream cycle output, gets empty array. Related to v0.42 concept
type-unification semantics.
- phantom-redirect.test.ts (2 fails): concurrent-sync race +
postgres-js text-string embedding survival. Master-level data-path
bug; would need its own fix wave.
- sync-lock-recovery.test.ts (1 fail): `gbrain sync --break-lock
--all` exits 0 but test expects 1 with a shell-loop hint. CLI
behavior changed in a master commit; need to either restore the
refusal behavior or update the assertion.
None of these 4 block CI (E2E job doesn't run them). Filed as a
TODOS.md entry for a follow-up wave; the 2 in this commit are the
ones that mirror v0.42 work landing.
Local: 130/136 E2E files green, 927/940 tests pass (was 925/940
before these fixes; the 2 files this commit fixes added 7 newly-
passing tests).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ci): quarantine query-cache-knobs-hash.test.ts to serial runner
CI shard 10 (commit 4d721077) failed 5 tests in the
`SemanticQueryCache cross-mode isolation (CDX-4 hotfix)` describe block,
all ~7-34ms each, all expecting writes/reads to round-trip through one
shared PGLite engine + a `beforeEach DELETE FROM query_cache`. Passes
9/9 locally; fails 5/9 on Linux CI under bun's default in-file
max-concurrency=4.
Classic intra-file concurrency race shape: test A's `beforeEach`
clears the table → test A's `store` writes a row → test B's
`beforeEach` (concurrent with A's `store`) clears the table → test A's
follow-up COUNT query returns 0. Same root cause that quarantined
`embed-stale.test.ts`, `brain-allowlist.test.ts`, and
`schema-pack-find-pack-successors.test.ts` to the serial runner in
prior fix waves (documented in v0.41.22.0 CI fix wave).
Fix: rename to `query-cache-knobs-hash.serial.test.ts` so the v0.26.7
serial-tests runner picks it up at `max-concurrency=1`. Tests still
exercise the actual cache logic — no test deleted, no production code
changed. The describe block's `beforeAll` engine + `beforeEach`
TRUNCATE pattern works correctly at serial concurrency.
Local: 12/12 in this file + 52/52 in the serial runner. Production
SemanticQueryCache code is untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(heavy): frontmatter_scan_wallclock — opt into --no-embedding so CI runners work
Heavy tests workflow run 26542447602 (commit 483a5577) failed on the
first heavy script:
[fm_wallclock] FAIL: gbrain init exited non-zero
No embedding provider configured. Set one of:
OPENAI_API_KEY / ZEROENTROPY_API_KEY / VOYAGE_API_KEY
Or defer setup: gbrain init --pglite --no-embedding
The v0.37 D9 hard-require landed in init.ts: `gbrain init --pglite` now
refuses to proceed without an embedding provider configured. The
heavy-tests GitHub workflow doesn't pipe any embedding API keys
(deliberate — the heavy tests measure ops shape, not LLM behavior), so
every CI invocation now blocks at step 2 of this script.
The script's whole purpose is measuring `gbrain doctor`'s
frontmatter-scan wallclock — it never embeds, never calls
`gbrain embed`, never queries vectors. The right fix is to opt out of
the provider requirement via the same `--no-embedding` flag init.ts
already exposes for this exact "deferred setup" case.
Verified locally:
TMP=$(mktemp -d); GBRAIN_HOME="$TMP" \
bun run src/cli.ts init --pglite --yes --no-embedding
# exit 0, brain initialized.
No production code change. One-line + comment in the script.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(heavy): sync_lock_regression — pass --no-embed so CI runs measure lock contention, not key absence
Heavy tests workflow run 26542545802 (commit 7962d312, after the
previous fm_wallclock fix) failed at the next heavy script in the chain:
[sync_lock_regression] outcomes: winners=0 losers=0 unknown=4
[sync_lock_regression] FAIL: expected 1 winner, got 0
[sync_lock_regression] FAIL: expected 3 lock-busy losers, got 0
Each of the 4 parallel `gbrain sync` invocations failed for the same
reason — none of them ever even got to the lock-acquire step:
Embedding model "zeroentropyai:zembed-1" requires ZEROENTROPY_API_KEY.
Re-run with --no-embed to import-only and embed later once the key is set.
The CI runner doesn't pipe any embedding-provider API keys (deliberate —
heavy tests measure ops shape, not LLM behavior), and sync now hard-fails
when its embed step can't reach a configured provider.
This script measures the writer-lock race shape — `gbrain-sync` row in
`gbrain_cycle_locks`, exactly-one-winner semantics, N-1 fail-fast losers
with "Another sync is in progress", zero leaked rows post-run. It never
needed embeddings; the original write predates the hard-require landing.
Fix: pass `--no-embed` to the sync invocation. Same kind of fix as
fm_wallclock (commit 7962d312) but on the sync side rather than init.
No production code touched. One-line change in the bash script.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(heavy): sync_lock_regression — register source via psql + use --repo + tolerate doctor warns
Heavy tests run 26542638471 (commit 60145eee, after the --no-embed
fix) failed at the same script but at a downstream step:
> Source "default" has no local_path. Run: gbrain sources add default --path <path>
Three independent bugs in the script that all surfaced at once after
v0.41's source-registry landed:
1. `gbrain config set sync.repo_path` is the legacy way; sync now
reads `sources.local_path` first. Replaced with an upsert into the
sources table via psql:
INSERT INTO sources (id, name, local_path)
VALUES ('default', 'default', $BRAIN_DIR)
ON CONFLICT (id) DO UPDATE SET local_path = EXCLUDED.local_path
Kept the legacy `config set sync.repo_path` line too as
belt-and-suspenders for any downstream caller that still reads it.
2. `gbrain sync --dir <path>` is silently ignored; sync's CLI parser
recognizes `--repo`, not `--dir`. Switched to `--repo`.
3. `bun run src/cli.ts doctor --json` at the top (used to apply
migrations as a side effect) exits non-zero whenever ANY check
warns — including the new "no embedding provider configured"
warning on a fresh CI runner. The script's `set -e` aborted at
line 53 before reaching any of the sync invocations. Added `|| true`
since the migration runs regardless of doctor's exit verdict.
Verified locally — `DATABASE_URL=... bash tests/heavy/sync_lock_regression.sh`
output:
[sync 1] rc= (lock-busy: 'Another sync is in progress')
[sync 2] rc=0 (winner)
[sync 3] rc= (lock-busy: 'Another sync is in progress')
[sync 4] rc= (lock-busy: 'Another sync is in progress')
outcomes: winners=1 losers=3 unknown=0
post-run gbrain_cycle_locks(gbrain-sync) row count: 0
OK — 1 winner, 3 lock-busy losers, no leaked lock rows.
Production code untouched. All three fixes are in the bash script.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(skillopt): hands-on tutorial for auto-improving a skill + discoverability
There was no tutorial for skillopt — only a reference guide
(docs/guides/skillopt.md) that opens at --bootstrap-from-routing and
assumes you already understand benchmarks, and an agent-facing SKILL.md.
README had ZERO skillopt mention. The one thing a user must hand-author
(the benchmark JSONL) was taught nowhere with a worked example.
New: docs/tutorials/improving-skills-with-skillopt.md — Diataxis tutorial
(learning-oriented), copy-pasteable end to end:
1. mental model in two sentences (SKILL.md is the trainable param, the
agent is frozen)
2. write your first benchmark from scratch — a complete 15-task rule-judge
starter you paste and run, with the full check-op table
(contains/regex/section_present/max_chars/min_citations/tool_called/
tool_not_called)
3. --dry-run cost preview (and that it exits 2 by convention, not failure)
4. real run + reading accepted(0)/no_improvement(1)/aborted(2) with the
actual stderr output shape
5. where output lands (best.md, versions/, history.json, rejected.json,
audit jsonl)
6. accept/reject — bundled vs user skills, --no-mutate vs
--allow-mutate-bundled
7. iterate by sharpening the benchmark
The load-bearing fix the tutorial makes that the reference guide got wrong:
the DEFAULT --split 4:1:5 needs ~50 tasks before it runs (sel = N/10, floor
5). A first-time author writing 10-15 tasks hits `D_sel has N task(s)
(need >=5)` and bounces. The tutorial ships 15 tasks + `--split 1:1:1`
(clean 5/5/5) so the copy-paste path actually works. Verified against the
real loadBenchmark + splitBench: the exact shipped block parses 15 unique
tasks and splits 5/5/5 with sel>=5; the system's own error message confirms
"need ~50 total for 4:1:5".
Discoverability (Diataxis cross-linking):
- README.md tutorials section: new entry (was zero skillopt mention)
- docs/tutorials/README.md: added under ## Shipped
- docs/guides/skillopt.md: "New to this? Start with the tutorial" callout
Every claim devex-verified against source: exit-code map from
skillopt.ts (accepted:0/no_improvement:1/aborted:2/errored:2), stderr
format from skillopt.ts:286-292, check ops from score.ts, output paths
from SKILL.md, split math from benchmark.ts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: regenerate llms-full.txt after skillopt tutorial + README edit
Refreshes the inlined doc bundle so the committed llms-full.txt matches
fresh `bun run build:llms` output (test/build-llms.test.ts drift guard).
Picks up the README tutorials-section edit from c39dbdb1. The new tutorial
file itself isn't curated into scripts/llms-config.ts (the bundle curates
a fixed doc set, not every tutorial) — this is purely the README delta.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ci): stop embed-preflight leaking gateway config into facts-backstop shard
CI shard 10 failed 5 `put_page facts backstop` tests with:
[embed(openai:text-embedding-3-small)] Incorrect API key provided: sk-test
(captured by the diagnostic stderr added in a prior commit). Root cause is
a cross-file module-state leak, not a logic bug:
- `embed-preflight.test.ts` calls `configureGateway({env:{OPENAI_API_KEY:
'sk-test'}})` to drive credential-validation scenarios. It resets the
gateway `beforeEach` but never AFTER its last test, so it leaves the
gateway configured with `sk-test`.
- bun runs every file in a shard inside ONE process. The residual config
bleeds into the next file. When `facts-backstop-gating.test.ts` lands in
the same shard, its put_page calls see `isAvailable('embedding') === true`
(the key is *present*, just invalid), so put_page attempts a real embed
and 401s before the backstop gating even runs.
- It's intermittent across master merges because shard bin-packing changes
which files co-locate. (It "resolved" after the v107 merge earlier for
exactly this reason, then came back.)
R1/R2 test-isolation lint doesn't catch this — it's `configureGateway`
module state, not `process.env` or `mock.module`.
Two fixes, both using the gateway's own `resetGateway()` seam (no
process.env, R-compliant):
1. embed-preflight.test.ts — `afterAll(() => resetGateway())` so the leaker
cleans up after the whole file. Primary fix; also protects any OTHER
shard-mate that reads gateway state.
2. facts-backstop-gating.test.ts — `beforeEach(() => resetGateway())` so the
suite is deterministic regardless of ambient gateway config. Defense in
depth: isAvailable('embedding') is now reliably false → put_page uses
noEmbed → the import never embeds → only the backstop gating (the suite's
actual subject) is exercised.
Verified: running leaker+victim in one process (the shard repro) goes
16/16; full shard 10 goes 1208/1208 (was 5 fail in CI). Typecheck clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(skillopt): make benchmark authoring an agent job, not a human chore
The prior tutorial taught a human to hand-write a 15-task benchmark — but
nobody does that. The real workflow is: user says "make skill X better,"
the AGENT authors the benchmark and runs the optimizer. The agent-facing
dispatcher didn't actually cover that.
Gap found: skill-optimizer/SKILL.md documented exactly one authoring path,
`--bootstrap-from-routing`, which (a) requires a pre-existing
routing-eval.jsonl (bootstrap-benchmark.ts:57-63 refuses without it) and
(b) generates tasks from ROUTING fixtures — which test dispatch ("does
this phrasing pick this skill"), not output quality. So an agent told to
improve a skill with no benchmark had no documented way to author a
*quality* benchmark; it'd have to reinvent the JSONL format the human
tutorial teaches.
Two fixes:
1. skills/skill-optimizer/SKILL.md — new "Authoring the benchmark yourself
(the common case)" section: read the target SKILL.md, generate ~15
realistic tasks, attach rule judges (contains/max_chars/min_citations/
section_present/regex/tool_called), write the JSONL, run with
`--split 1:1:1` (the default 4:1:5 needs ~50 tasks). Decision-tree row
"New skill, no benchmark" now says "Author one" instead of pointing at
bootstrap-from-routing; the bootstrap row is reframed as a head-start
that only applies when routing fixtures exist and notes routing tasks
test dispatch, not quality.
2. docs/tutorials/improving-skills-with-skillopt.md — new "The easiest
path: ask your agent" section up top. Tells humans to just tell their
agent "improve my X skill — write a benchmark first," and frames the
manual walkthrough as "read this when you want to understand or
hand-curate what the agent is doing."
Verified: conformance 249/0, resolver 99/0, build-llms drift guard 7/0,
cross-link resolves.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(skillopt): --bootstrap-from-skill starter benchmark generator
Generate a quality benchmark from a skill's SKILL.md directly, no
routing-eval.jsonl required. One LLM call emits JSONL tasks (each with rule
judges) that the agent reviews + strengthens before optimizing.
- runBootstrapFromSkill: JSONL output parsed line-by-line with skip-bad-line
salvage (a truncated final line drops, the rest survive); a task is kept only
when >=2 valid rule checks survive; provider errors propagate instead of
collapsing to bootstrap_empty.
- --bootstrap-tasks N (default 15, cap 50); maxTokens scales with the count.
- Extracted assertBenchmarkAbsent + readSkillBodyOrThrow shared with the routing
bootstrap; hardened runBootstrap's routing-eval parse to skip malformed lines.
- CLI: --bootstrap-from-skill short-circuit + 6-way mutual exclusion; parseFlags
exported for unit tests. The benchmark-not-found hint + --help now point here.
- The generator's REVIEW line prints the paste-ready
`--bootstrap-reviewed --split 1:1:1` next command (the default 4:1:5 split
refuses a 15-task starter at D_sel >= 5).
- 20 hermetic cases incl. round-trip into loadBenchmark + splitBench(1:1:1).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(skillopt): make --bootstrap-from-skill the primary no-benchmark path
The agent runs --bootstrap-from-skill, strengthens the generated judges (they
are weak drafts), deletes the sentinel, then runs --bootstrap-reviewed
--split 1:1:1. Freehand authoring is demoted to the fallback for the rare skill
the generator can't draft well. Updates the Iron Law, decision tree, and
anti-patterns to cover both bootstrap modes and the 15-task / --split 1:1:1
gotcha.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(release): v0.42.1.0 --bootstrap-from-skill
VERSION + package.json -> 0.42.1.0, CHANGELOG entry, CLAUDE.md skillopt
annotation, regenerated llms-full.txt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: surface --bootstrap-from-skill in README + skillopt reference
- docs/guides/skillopt.md: 30-second pitch leads with --bootstrap-from-skill;
flag table adds --bootstrap-from-skill + --bootstrap-tasks rows.
- README.md: skillopt tutorial pointer mentions generating a starter benchmark.
- Regenerated llms-full.txt (README is in the bundle).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ci): bump FULL_SIZE_BUDGET 700KB→750KB for legitimate CLAUDE.md growth
The skillopt wave annotations + merged v0.41.34-36 master releases pushed
llms-full.txt to 700,423 bytes — 423 over the 700KB cap — failing the
build-llms size-budget test on CI shard 6. CLAUDE.md is ~540KB (77% of the
bundle) and is the whole point of the one-fetch artifact, so it stays inlined;
the budget tracks its per-release growth. 750KB still fits 200k+ context models.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GBrain
Search gives you raw pages. GBrain gives you the answer. It's the brain layer your AI agent has been missing — the only one that does synthesis, graph traversal, and gap analysis in one box.
I'm Garry Tan, President and CEO of Y Combinator. I built GBrain to run my own AI agents. It's the production brain behind my OpenClaw and Hermes deployments: 146,646 pages, 24,585 people, 5,339 companies, 66 cron jobs running autonomously. My agent ingests meetings, emails, tweets, voice calls, and original ideas while I sleep. It enriches every person and company it encounters. It fixes its own citations and consolidates memory overnight. I wake up smarter than when I went to bed — and so will you.
And now it works as a company brain too. Each person on the team gets their own slice of the brain, scoped by login. When you query, you only see what you're allowed to see — never another person's notes, never another team's data. We fuzz-tested this across every way you can read the brain (search, list, lookup, multi-source reads) and got zero leaks. Drop GBrain in as your team's shared institutional memory — the company-brain shape YC just put on its Request for Startups. If you're building in that space, you might as well build on this. Tutorial: set up GBrain as your company brain →
Lots of personal-knowledge systems give you keyword matching and grep in a box. GBrain does that, and adds two things nobody else ships together:
- A synthesis layer that gives you the actual answer. Synthesized, well-cited prose across people, companies, deals, and ideas. Not "here are 10 chunks that mention your query"; an actual answer with citations and an explicit note on what the brain doesn't know yet. The gap analysis is the part that changes how you use the brain.
- A self-wiring knowledge graph. Every page write extracts entity refs and creates typed edges (
attended,works_at,invested_in,founded,advises) with zero LLM calls. Ask "who works at Acme AI?" or "what did Bob invest in this quarter?" and get answers vector search alone can't reach. Benchmarked: P@5 49.1%, R@5 97.9% on a 240-page Opus-generated rich-prose corpus, +31.4 points P@5 over its graph-disabled variant and over ripgrep-BM25 + vector-only RAG by a similar margin. Full BrainBench scorecards live in the sibling gbrain-evals repo.
The point of building a 100K-page brain is to use it as a strategic moat. To never lose context. To query what's in your own head without re-reading it. The brain layer is what makes the moat usable. The 24/7 dream cycle is what keeps it sharp. Both run on your hardware, your DB, your keys.
It's easier to ship a daemon that runs 24/7 to ingest, enrich, and consolidate than it is to keep an agent in chat working hard. GBrain is that daemon, generalized. Install in 30 minutes. Your agent does the work. As my personal agent gets smarter, so does yours.
~30 minutes to a fully working brain. Database ready in 2 seconds (PGLite, no server). You just answer questions about API keys.
LLMs: fetch
llms.txtfor the documentation map, orllms-full.txtfor the same map with core docs inlined in one fetch. Agents: start withAGENTS.md(orCLAUDE.mdif you're Claude Code).
What this looks like
Say you have a meeting with Alice tomorrow. You want to walk in remembering what she works on, when you last talked, and what's still open between you. Here's what you'd type, and what you'd get back.
You ask:
"What do I need to know before my meeting with Alice tomorrow?"
Most personal-knowledge tools give you back a list of pages. Something like:
1. people/alice — Alice runs engineering at Acme...
2. meetings/2026-03-15-alice-q1 — Q1 product review with Alice...
3. meetings/2026-01-08-acme-kickoff — Kickoff meeting with Acme team...
4. customers/acme — Acme is a series-B fintech we work with...
5. notes/2026-04-22 — Quick chat with Alice about pricing...
Five pages you now have to open and read yourself to actually prepare. The tool found the right material, but it didn't do the work.
GBrain gives you back the answer, with sources:
Alice runs engineering at Acme (a series-B fintech). You last spoke
on April 22 in a quick pricing chat. Three things are still open
from that conversation:
1. She owes you the security review for the new tier
(deadline was May 1; no update since).
2. You committed to pricing for a 500-seat tier
(you sent it April 25; no response yet).
3. She mentioned they're hiring a CISO; you said you'd intro
someone from your network.
Heads up: nothing's been added to the brain about Alice or Acme
since April 22, six weeks ago. She may have replied through email
or Slack DM, channels the brain doesn't see. Worth asking her to
catch up before assuming any of this is still current.
Every claim has a source page behind it. The "heads up" at the end tells you what the brain doesn't know yet, so you can ask Alice about it directly instead of being surprised. The brain just did your meeting prep.
This is the difference between a search engine and a brain. Search finds the pages. The brain reads them for you and writes the answer.
Install
GBrain is designed to be installed and operated by an AI agent. The fastest path is to have your agent do it for you. The CLI and MCP paths below are for people who want to wire it up themselves.
Have your agent install it (recommended)
If you don't already have an AI agent platform running, start with one of these. Both are designed to read GBrain's install protocol and execute it:
- OpenClaw — deploy AlphaClaw on Render (one click, 8GB+ RAM)
- Hermes — deploy on Railway (one click)
Then paste this into your agent:
Retrieve and follow the instructions at:
https://raw.githubusercontent.com/garrytan/gbrain/master/INSTALL_FOR_AGENTS.md
The agent installs GBrain, creates the brain, asks for your API keys, loads 43 skills, configures the dream cycle, and verifies the install end-to-end. ~30 minutes. You answer questions, it does the work.
Never set up an AI agent platform before? The personal-brain tutorial walks the whole path end-to-end — picking OpenClaw vs Hermes, deploying it, pointing it at INSTALL_FOR_AGENTS.md, getting the API keys, and verifying the first query. Start there if any of the above is new.
Install it into your existing agent
Already running Codex, Claude Code, Cursor, or another coding agent? Paste the same instruction in:
Retrieve and follow the instructions at:
https://raw.githubusercontent.com/garrytan/gbrain/master/INSTALL_FOR_AGENTS.md
This works in any agent that can read files over HTTPS and execute shell commands. Tested with Codex, Claude Code, Claude Cowork, Cursor, and AlphaClaw.
CLI standalone (no agent)
bun install -g github:garrytan/gbrain
gbrain init --pglite # 2 seconds; no server, no Docker
gbrain doctor # verify health
gbrain import ~/notes/ # index your markdown
gbrain query "what themes show up across my notes?"
Postgres-at-scale, Supabase, and thin-client setup paths live in docs/INSTALL.md.
Connect GBrain to your AI client (MCP)
GBrain exposes 30+ tools over MCP (stdio and HTTP). The specific snippet depends on which client you use:
- Claude Code — one command:
claude mcp add gbrain -- gbrain serve. Zero server, zero tunnel. - Cursor / Windsurf / any stdio MCP client — same shape, add
{"command": "gbrain", "args": ["serve"]}to your MCP config. - Claude Desktop (Cowork) — Settings → Integrations → add the URL of your HTTP server. Remote only; the local
claude_desktop_config.jsondoes not work for remote servers. - Claude Cowork (team plan) — org Owner adds the connector under Organization Settings → Connectors.
- Perplexity Computer — Settings → Connectors → add the URL + bearer token. Pro subscription required.
- ChatGPT — uses OAuth 2.1 with PKCE (the hard requirement). Register a
chatgptclient from the admin dashboard with grant typeauthorization_code.
For the HTTP server itself:
gbrain serve # stdio MCP (local subprocess; for Claude Code, Cursor, Windsurf)
gbrain serve --http # HTTP MCP with OAuth 2.1 + admin dashboard at /admin
# (required for Claude Desktop, Cowork, Perplexity, ChatGPT)
The HTTP server includes DCR-style client registration, scope-gated access (read / write / admin), and rate limiting. Deployment guides (ngrok, Railway, Fly.io) live under docs/mcp/.
Two ways to query your brain
Raw retrieval (what most personal-knowledge tools ship) and a synthesis layer that gives you an actual answer. They serve different jobs.
# raw retrieval: top pages by hybrid score, fast, no LLM cost
gbrain search "who's working on AI agents at portfolio companies?"
# brain layer: synthesized answer with citations and gap analysis
gbrain think "who's working on AI agents at portfolio companies?"
gbrain search returns the top retrieved pages, ranked by hybrid scoring (vector + keyword + RRF + source-tier boost + reranker). Use it when you want raw material to skim: agent context windows, citation lookups, finding a specific quote.
gbrain think runs the same retrieval, then composes a synthesized answer across the results with explicit citations to the source pages AND an honest note on what the brain doesn't know yet. The gap analysis is the differentiator: the answer tells you when a page is stale, when a claim is uncited, when two pages contradict each other, when there's a hole you should fill.
Why it compounds. Pair the brain layer with find_trajectory and you get answers like "how have the company's metrics changed AND what does the team look like right now AND what did they promise / share AND when did we last meet AND what's the value-add I can offer here": well-scored, well-cited, in one shot. That's the strategic moat. That's why building a 100K-page brain is worth the effort.
gbrain agent run "..." exposes the same surface to a sub-agent through the Minions queue, with crash-safe two-phase persistence. Same answers, durable.
How to get data in
One command, local or hosted, synchronous receipt:
gbrain capture "the thought I want to remember"
gbrain capture --file ./notes/today.md
echo "from a pipe" | gbrain capture --stdin
SLUG=$(gbrain capture "..." --quiet)
The page lands in the database and on disk in one move. Default slug inbox/YYYY-MM-DD-<hash8> so captures cluster in a predictable triage location. On thin-client installs the verb routes through MCP to the server: same command, same UX.
For webhook ingestion (Zapier / IFTTT / Apple Shortcuts):
curl -X POST https://your-brain/ingest \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: text/markdown" \
-d "# a thought from a Shortcut"
For mobile capture, the inbox folder source picks up anything dropped into
~/.gbrain/inbox/ from iOS Shortcuts / AirDrop / Drafts / Finder.
Third-party skillpacks can ship custom ingestion sources (Granola, Linear,
voice, OCR) against the versioned IngestionSource contract at
gbrain/ingestion. See docs/skillpack-anatomy.md.
Your brain's shape (schema packs)
Most personal-knowledge tools force one fixed layout: their idea of "notes" + "people" + "tags." Drop a Notion export or your own years-old Obsidian vault on top, and the agent doesn't know what a Projects/ folder means or whether Reading/ is people or sources.
gbrain doesn't have a fixed layout. It ships with bundled schema packs and lets you author your own when none fit:
gbrain-base-v2(default as of v0.41.22) — 15-type DRY/MECE canonical taxonomy (14 canonical +notecatch-all):person,company,media,tweet,social-digest,analysis,atom,concept,source,deal,email,slack,writing,project,note. Subtypes/format/origin pushed to frontmatter. The taxonomy that responds to issue #1479.gbrain-base(legacy, v0.41 and earlier brains) — the original 24-type layout. Stays bundled for back-compat; brains on it can upgrade viagbrain onboard --check --explain→gbrain jobs submit unify-types --allow-protected --params '{"target_pack":"gbrain-base-v2"}'.gbrain-recommended— extendsgbrain-basewith the 13 additional directories fromdocs/GBRAIN_RECOMMENDED_SCHEMA.md(source, place, trip, conversation, personal, civic, project, etc.). Activate withgbrain schema use gbrain-recommended.- Your own pack —
gbrain schema detectclusters your actual filesystem into proposed types,gbrain schema suggestruns an LLM pass over them, andgbrain schema review-candidates --applypromotes the ones you like. Three commands and the brain knows your shape. Authoring a successor pack (declaresmigration_from:so existing brains can opt in): seedocs/architecture/pack-upgrade-mechanism.md.
gbrain schema active # which pack is running, which tier set it
gbrain schema list # bundled + installed packs
gbrain schema detect # propose types matching your filesystem
gbrain schema suggest # LLM-refined proposals on top of detect
gbrain schema review-candidates # human gate: promote / rename / ignore
gbrain schema use my-pack # activate
The active pack threads through every read + write path: parseMarkdown infers page type from the pack's path prefixes; whoknows scopes expert routing to types declared expert_routing: true; extract_facts runs only on extractable: true types; the search cache folds the pack name + version into its key so cross-pack contamination is structurally impossible. Switch packs and the brain re-interprets itself; switch back and nothing's lost.
Seven-tier resolution chain (per-call flag → env var → per-source DB key → brain-wide DB key → gbrain.yml → ~/.gbrain/config.json → gbrain-base default). Full reference + authoring guide: docs/architecture/schema-packs.md.
Tutorials
Step-by-step walkthroughs for getting the most out of GBrain. Each one takes you from zero to a working outcome, with concrete commands and real numbers.
- Set up your personal AI agent + brain from zero — the canonical full-stack install. Two GitHub repos, a Telegram bot, AlphaClaw on Render, OpenClaw + GBrain + Supabase. End-to-end in about 2 hours.
- Set up GBrain as your company brain — federated, multi-user, OAuth-scoped institutional memory for a 10-50 person team. About 90 minutes end-to-end.
- Auto-improve a skill with
gbrain skillopt— treat aSKILL.mdas a trainable parameter. Generate a starter benchmark straight from the skill with--bootstrap-from-skill(or write your own), strengthen the judges, then watch the optimizer propose edits and keep only the ones that measurably score higher. ~20 minutes, ~$1 in API calls. Flag + cost + safety reference:docs/guides/skillopt.md.
More walkthroughs in progress: connecting an existing agent (Claude Code, Cursor, OpenClaw, Hermes) to a GBrain memory layer; setting up GBrain for VC dealflow with founder scorecards and meeting prep; migrating an existing Notion or Obsidian vault; indexing a codebase as a queryable code brain. Full tutorial index: docs/tutorials/.
Want to see a tutorial that isn't here yet? Open an issue describing the workflow you want documented.
What it does (the loop)
signal → search → respond → write → auto-link → sync
(every (brain-first (informed (page + (typed edges (cron
message) retrieval) by context) timeline) + backlinks) keeps fresh)
- Signal detector runs on every message your agent receives. Captures ideas, entity mentions, time-sensitive todos, names, links.
- Brain-first lookup before any external API call. The cheapest, fastest, most personal information source you have.
- Auto-link fires on every page write. No LLM calls; pure pattern matching on
[[wiki/people/bob]]style references. New entity → new page stub → graph grows. - Cron-driven enrichment runs while you sleep: dedup people pages, fix citations, score salience, find contradictions, prep tomorrow's tasks.
The whole loop is described in docs/architecture/topologies.md with diagrams.
Capabilities
Hybrid search. Vector (HNSW on pgvector) + BM25 keyword + reciprocal-rank fusion + source-tier boost + intent-aware query rewriting. Three named search modes (conservative, balanced, tokenmax) bundle the cost/quality knobs into a single config key. Live cost/recall comparisons in docs/eval/SEARCH_MODE_METHODOLOGY.md. Default: balanced with ZeroEntropy reranker on. Per-query graph signals notice when a top result is a hub for THAT query (adjacency boost), is corroborated across team brains (cross-source boost), or is being crowded out by weak chunks from a chatty session (session demote). Run gbrain search "<query>" --explain to see per-stage attribution: base score, every boost that fired, what it multiplied. gbrain doctor ships a graph_signals_coverage check; gbrain search stats shows fire counts and failure breakdowns. Vector retrieval pools the best chunk per page, so a page surfaces on its strongest evidence instead of losing to a neighbor on one weak chunk. Queries that match a page's title phrase or a declared free-text alias (gbrain reindex --aliases backfills existing pages) get boosted to the page they name. Every result carries an evidence tag (why it matched) and a create_safety hint (exists / probable / unknown) so an agent decides whether a page already exists instead of guessing from a raw score. gbrain search diagnose "<query>" --target <slug> traces which retrieval layer surfaces (or misses) a page.
Self-wiring knowledge graph. Every put_page extracts entity refs from markdown/wikilinks/typed-link syntax and writes edges with zero LLM calls. Typed edges (attended, works_at, invested_in, founded, advises, mentions, …). Multi-hop traversal via gbrain graph-query. The graph is what produces the +31.4 P@5 lift over vector-only RAG.
Job queue (Minions). BullMQ-shaped, Postgres-native job queue. Durable subagents (LLM tool loops that survive crashes via two-phase pending→done persistence), shell jobs with audit, child jobs with cascading timeouts, rate leases for outbound providers, attachments via S3/Supabase storage. Replaces "spawn subagent as fire-and-forget Promise" with something that recovers from anything.
43 curated skills. Routing lives in skills/RESOLVER.md. Covers signal capture, ingest (idea / media / meeting), enrichment, querying, brain ops, citation fixing, daily task management, cron scheduling, reports, voice, soul audit, skill creation, eval framework, and migrations. Skills are markdown files (tool-agnostic), packaged as a single skillpack the installer drops into your agent workspace.
Eval framework. gbrain eval longmemeval runs the public LongMemEval benchmark against your hybrid retrieval. gbrain eval export + gbrain eval replay capture real queries and replay them against code changes (set GBRAIN_CONTRIBUTOR_MODE=1). gbrain eval cross-modal cross-checks an output against the task using three different-provider frontier models. gbrain eval retrieval-quality runs NamedThingBench, which hard-gates the named-thing retrieval families (title-substring, alias-synonym, generic-to-named, multi-chunk-dilution) so a regression in "find the page this query names" fails CI loudly. Full methodology in docs/eval/SEARCH_MODE_METHODOLOGY.md.
Brain consistency. gbrain eval suspected-contradictions samples retrieval pairs, layered date pre-filter, query-conditioned LLM judge, persistent cache. Surfaces conflicts between takes + facts the agent has written. Wired into the daily dream cycle.
Agent-authored schema (v0.40.7.0). Your brain has a shape — what page types exist (person, meeting, paper, case, lab-result), what they link to (attended, authored, prescribed-by), what facts get extracted automatically. The default ships with 22 universal types, but your brain's actual shape is not the default shape. Agents can now evolve that shape on your behalf via 14 gbrain schema CLI verbs + a batched MCP op (schema_apply_mutations, admin scope, NOT localOnly so remote agents reach it over HTTPS). Atomic file locks, audit log with the agent's identity, chunked UPDATE backfill in 1000-row batches that never wedge concurrent writers. The brain stops being a pile of notes and becomes something with structure. Why it matters: docs/what-schemas-unlock.md — 7 killer use cases (4000 invisible meetings, founder ops brain, research brain, legal brain, team brain, agent-as-co-curator). 5-minute walkthrough: docs/schema-author-tutorial.md. Agent skill: skills/schema-author/SKILL.md.
Integrations
Data flowing into the brain. Each integration is a recipe — markdown + setup hints — that ships in recipes/ and is discoverable via gbrain integrations list.
- Voice: Phone calls create brain pages via Twilio + OpenAI Realtime (or DIY STT+LLM+TTS). Setup recipe:
recipes/twilio-voice-brain.md. - Email + calendar: webhook handlers that route to brain signals.
docs/integrations/meeting-webhooks.md. - Embedding providers: 16 recipes covering OpenAI (default fallback), OpenRouter, Voyage, ZeroEntropy (default), Google Gemini, Azure OpenAI, MiniMax, Alibaba DashScope, Zhipu, Ollama (local), llama.cpp llama-server (local), LiteLLM proxy. Pricing matrix + decision tree in
docs/integrations/embedding-providers.md. - Rerankers: ZeroEntropy
zerank-2hosted (default intokenmaxmode) plus the v0.40.6.1llama-server-rerankerrecipe for fully-local cross-encoder rerank via llama.cpp — runs Qwen3-Reranker or self-hosted ZeroEntropy weights against the samegateway.rerank()seam. Setup walkthrough indocs/ai-providers/llama-server-reranker.md. - Credential gateway: vault-aware secret distribution.
docs/integrations/credential-gateway.md. - MCP clients: every major MCP client is supported.
docs/mcp/per-client setup.
Architecture
Two engines, one contract. PGLite (Postgres 17 via WASM, zero-config, default) for personal brains up to ~50K pages. Postgres + pgvector (Supabase or self-hosted) for shared / large / multi-machine deployments. The contract-first BrainEngine interface in src/core/engine.ts defines ~47 operations both engines implement; CLI and MCP server are generated from one source.
Brain repo is the system of record. Your knowledge lives in a regular git repo (your "brain repo") as markdown files. GBrain syncs the repo into Postgres for retrieval; deletes in git become soft-deletes in DB. You can publish public subsets, share team mounts, run thin-client setups pointing at a colleague's brain server. Topologies in docs/architecture/topologies.md.
Two organizational axes (brain ⊥ source). A brain is a database (your personal brain, a team mount you joined). A source is a repo inside that brain (wiki, gstack, an essay, a knowledge base). Routing lives in .gbrain-source dotfiles and resolves via a documented 6-tier precedence chain. Full diagrams in docs/architecture/brains-and-sources.md.
Why the graph matters. Vector search returns chunks that are semantically close. The graph returns chunks that are factually connected. Hybrid search pulls from both; auto-linking on every write keeps the graph fresh. Deep dive: docs/architecture/RETRIEVAL.md.
Troubleshooting
gbrain import fails with expected N dimensions, not M? Run gbrain doctor. It will print the exact gbrain config set ... or gbrain retrieval-upgrade command to repair the mismatch. You should not need to delete ~/.gbrain. Fresh gbrain init --pglite auto-detects your embedding provider from API keys in your environment: set OPENAI_API_KEY (or ZEROENTROPY_API_KEY / VOYAGE_API_KEY) before running init, or pass --embedding-model <provider>:<model> explicitly. With multiple keys set, init fires an interactive picker. In non-TTY contexts (CI, Docker) with no keys, init exits 1 with a paste-ready setup hint; pass --no-embedding to defer setup until runtime. See docs/integrations/embedding-providers.md for the full provider matrix and docs/operations/headless-install.md for Docker/CI sequencing.
Hourly cron sync keeps timing out on a federated brain? v0.41.13.0 ships
two flags + a recommended pattern. Switch your cron to a per-source loop
with shell timeout(1) doing the OS-level kill and gbrain self-terminating
gracefully half-a-minute earlier:
gbrain sync --break-lock --all --max-age 1800
for src in $(gbrain sources list --json | jq -r '.[].id'); do
timeout 600 gbrain sync --source "$src" --timeout 540 || true
done
When --timeout fires mid-import, gbrain sync exits 0 with status
partial and last_commit UNCHANGED — the next run re-walks the same
diff and content_hash short-circuits already-imported files. The
--max-age 1800 first command self-heals any wedged-but-alive locks
left by a hung previous run, using the v98 last_refreshed_at semantic
(NOT acquired_at) so healthy long-running holders are safe by
construction. See the v0.41.13.0 entry in CHANGELOG.md
for the honest scope notes (extract + embed phases run to completion;
30-min rollout window for --max-age post-migration v98; full-sync
triggers deferred to v0.42+).
Dream cycle silently losing wiki links on Supabase? v0.41.19.0 fixes
the bug class structurally. The engine now self-retries every bulk batch
write (addLinksBatch / addTimelineEntriesBatch / upsertChunks) on
Supavisor pooler blips, with a 12s worst-case wait that covers the full
5-10s circuit-breaker recovery window. gbrain doctor surfaces incidents
via the new batch_retry_health check (reads the last 24h of
~/.gbrain/audit/batch-retry-YYYY-Www.jsonl). To tune for an unusually
slow pooler:
# Defaults: 3 retries, base 1s, max 10s, decorrelated jitter.
# Override per operator without a release:
export GBRAIN_BULK_MAX_RETRIES=5 # int >= 0; 0 disables retries
export GBRAIN_BULK_RETRY_BASE_MS=2000 # int > 0
export GBRAIN_BULK_RETRY_MAX_MS=15000 # int >= base
Bad values surface at gbrain doctor startup with a paste-ready fix
(not at first-retry mid-cycle). PGLite-only installs pay zero cost — the
retry wrap is engine-level, but PGLite has no pooler so retries never
fire in practice.
Dream cycle losing ~150 link rows per run with 'No database connection: connect() has not been called' errors in the log? v0.41.27.0
makes the retry layer self-heal on a nulled-out database singleton. A
new reconnect callback on withRetry rebuilds the connection between
attempts; PostgresEngine.batchRetry injects () => this.reconnect()
so engine-level batch writes survive a mid-cycle disconnect by something
else in the same process. Same release: gbrain capture no longer trails
a 'No database connection' stderr line from a background facts:absorb
worker firing after CLI exit — the op-dispatch finally block awaits
getFactsQueue().drainPending({timeout: 1000}) before
engine.disconnect(). To find which code path is still calling
disconnect mid-process, run gbrain doctor --json | jq '.checks[] | select(.id=="batch_retry_health")'; the extended check now surfaces
24h disconnect-call count and the most-recent caller frame from a new
~/.gbrain/audit/db-disconnect-YYYY-Www.jsonl audit. (Closes #1570.)
gbrain brainstorm returning judge_failed: true with 0 scored
ideas? v0.41.21.0 closes the two bugs that caused it. The judge
hard-coded a 4K-token output cap; for any run past ~40 ideas the call
truncated mid-JSON and the parser threw. Same release closes a slash-
form pricing miss: gbrain brainstorm --judge-model anthropic/claude-sonnet-4-6 --max-cost 5 failed with
BudgetExhausted reason=no_pricing because every pricing site only
matched the colon form. Both shapes work now. No config change, no
schema migration — gbrain upgrade is the whole fix.
gbrain reindex --markdown wiped your auto/dream/signal-detector
tags? v0.41.37.0 makes tag reconciliation add-only. Re-import and
reindex --markdown now ADD current frontmatter tags and never delete,
so enrichment tags written to the DB (auto-tag, dream synthesize,
signal-detector) survive a re-chunk. The reindex DB-only fallback also
reconstructs the full markdown (frontmatter + body + timeline) before
re-chunking, so a page with no on-disk source keeps its frontmatter,
title, and timeline instead of getting overwritten with empty
frontmatter. Trade-off: removing a tag from a page's frontmatter no
longer removes it from the DB on the next sync (frontmatter-tag removal
needs a provenance column, deferred). (Closes #1621.)
gbrain sync wedges on a large brain (no progress, high CPU)?
v0.41.37.0 ships three things. First, name the stalling file:
GBRAIN_SYNC_TRACE=1 gbrain sync --no-pull --no-embed --yes
The last [sync] begin import: <path> line with no following completion
is the file being processed when the hang hit. Second, if you suspect a
schema-pack inference.regex with catastrophic backtracking, complete
the sync with the pack disabled and re-run extraction later:
gbrain sync --no-schema-pack --no-pull --no-embed --yes
gbrain schema lint now warns on the classic nested-quantifier ReDoS
shapes ((a+)+, (a*)*, …) in pack regexes, and the runtime caps
inference-regex input length (override via GBRAIN_MAX_REGEX_INPUT_CHARS).
Third, on a PGLite brain, stop gbrain serve before a large sync —
PGLite is single-writer and a live MCP server contends for the write
lock. See docs/architecture/serve-sync-concurrency.md
for the full triage. (Closes #1569.)
gbrain init --migrate-only / a schema migration fails on Windows
with getaddrinfo ENOTFOUND? v0.41.37.0 runs the 9 schema-bring-up
phases in-process instead of spawning a child gbrain init --migrate-only per phase. The spawned child died on
Windows + bun + Supabase pooler with a DNS-resolution failure even
though the parent connected fine; running in-process removes the spawn
entirely. The v0.13.1 grandfather migration that hung 70+ minutes on an
82K-page PGLite brain is also fixed — it now runs as a chunked bulk SQL
pass (keyed on the page PK, soft-delete-filtered, source-safe) that
completes in ~1-2 seconds. (Closes #1605, #1581.)
Docs
docs/INSTALL.md— every install path, end to enddocs/what-schemas-unlock.md— why schemas matter: 7 killer use cases, the structural argument for typed page kinds, the agent-co-curates pattern (v0.40.7.0)docs/schema-author-tutorial.md— 5-minute walkthrough: fork the bundled pack, add a custom type, backfill existing pages, prove the wiring viagbrain whoknowsdocs/architecture/— system design, topologies, retrieval theorydocs/guides/— how-to runbooks (sub-agent routing, minion deployment, skill development, brain-first lookup, idea capture, diligence ingestion)docs/integrations/— connecting external data sources (voice, email, calendar, embedding providers)docs/mcp/— per-client MCP setup (Claude Desktop, Code, Cursor, ChatGPT, Perplexity, Cowork)docs/eval/— eval framework, metric glossary, methodologydocs/ethos/— philosophy (thin harness, fat skills, markdown as recipes, origin story)AGENTS.md— entry point for non-Claude agentsCLAUDE.md— entry point for Claude Code (deep operating context)CONTRIBUTING.md— contributor guide, test discipline, eval-capture modeSECURITY.md— OAuth threat model, hardening defaults
Contributing
Run bun run test for the fast loop, bun run verify for the pre-push gate, bun run ci:local to run the full Docker-backed CI stack locally. Detailed test discipline in CONTRIBUTING.md.
Community PRs are batched into release waves rather than merged one-by-one — see the "PR wave workflow" section in CLAUDE.md. Contributor attribution stays attached via Co-Authored-By: trailers. We credit every accepted contribution in CHANGELOG.md.
If you find a bug or want a feature: open an issue first. Quick fixes (typo, doc bug, obvious regression) can go straight to a PR. Anything touching schema, retrieval ranking, MCP protocol, or the security boundary needs a design discussion in the issue first.
License + credit
MIT. I built GBrain to run my OpenClaw and Hermes deployments — the production brain behind my AI agents.
Origin story: docs/ethos/ORIGIN.md.
Community PR contributors are credited in CHANGELOG.md per release. ZeroEntropy (@zeroentropy) for the embedding + reranker stack that ships as the default. Voyage AI for the asymmetric-encoding recipe template. Ramp Labs for the search quality improvements lineage.