Add provider_chat_options alongside provider_base_urls and thread it through
the gateway config path into chat(). The chat request now deep-merges
provider-scoped options and model-scoped overrides into providerOptions keyed
by recipe id, preserving existing gateway-built options such as Anthropic
cacheControl and leaving absent-config behavior unchanged.
This lets operators disable thinking for small-budget hybrid-reasoning utility
calls without hardcoding that behavior for every use of those models.
* feat(skillopt): foundation modules — types, lr-schedule, benchmark, score, audit, lock
* feat(skillopt): edit primitives — apply-edits (D5+D9), rejected-buffer LRU, version-store (D8 history-intent-first)
* feat(skillopt): rollout (D2 gateway.toolLoop + D13 read-only allowlist), reflect (D7 two calls), validate-gate (D12 median+epsilon, D4 parallel), preflight (D3), bundled-skill-gate (D16)
* feat(skillopt): orchestrator (D6 slow-update, D10 ASCII diagrams, D11 caching), checkpoint, bootstrap (D15 sentinel), CLI dispatch + help
* feat(skillopt): cycle phase (F1 dream-loop wiring), PROTECTED_JOB_NAMES + MCP op (F6 admin scope + allowlist) + Minion handler (F7 --background)
* feat(skillopt): full cathedral — --all batch (F4), --target-models fleet (F5), write-capture (F10), held-out scaffold (F11), adversarial suite 41 cases (F2), E2E PGLite (F3), meta-skill bundle (T7), reflect+judge evals (F8+F9), docs (T10)
* chore: bump version to v0.42.0.0 (MINOR — significant new feature)
* fix(skillopt): wire trajectories from forward gate to reflect + fix parseEditsResponse parser misuse
Two related v0.42.0.0 bugs that conspired to make `runSkillOpt` structurally
unable to accept any candidate edit. Either alone would have killed self-evolution;
together they made the loop a no-op for every input.
**Bug 1 (orchestrator gap):** `runOptimizationLoop` in orchestrator.ts called
`runReflect({successes: [], failures: []})` with hardcoded empty arrays. The
forward gate's `scoredRollouts` were computed then voided. `runReflect`
short-circuits both modes when their batches are empty, so the optimizer was
never asked to propose an edit. Every step hit the no_edits_applied branch.
Fix: add `scoredRollouts: ScoredRollout[]` to `GateResult` and
`runsPerTask?: number` to `ValidateGateOpts`. Forward pass uses
`runsPerTask: 1`; orchestrator partitions returned rollouts by `score >= 0.5`
and threads real successes + failures into `runReflect`.
**Bug 2 (parser misuse):** `parseEditsResponse` in reflect.ts routed every
optimizer response through `parseJudgeJson` first. `parseJudgeJson` looks for
a `score` key (it's a judge-output parser, not an edits parser) and returns
null for any JSON without one — including the well-formed `{"edits": [...]}`
the optimizer is contractually required to emit. The function then early-
returned `[]` and the actual `tryExtractEdits` path on the next line was
unreachable dead code.
Fix: drop the wrong-typed guard. `parseEditsResponse` now calls
`tryExtractEdits` directly. Export it so `reflect.test.ts` can pin the
contract independently of the chat transport.
**Why this slipped through 152 prior skillopt tests:** zero unit coverage
of `parseEditsResponse` or `runReflect`. The existing E2E `all-reject` case
asserted no_improvement (which was true for the wrong reason — empty edits,
not gate rejection). Both bugs were structurally invisible to the existing
test surface.
**New coverage:**
- `test/skillopt/reflect.test.ts` (15 cases):
- 8 `parseEditsResponse` cases including the IRON-RULE regression pin
for the v0.42.0.1 fix (`{"edits": [...]}` JSON must survive the parser).
- 7 `runReflect` D7 contract cases: both modes fire, empty-batch skips,
additive token usage, one-mode-throws-other-still-works, rejected-buffer
flows into anti-bias prompt.
- Documents the trailing-comma limitation as an explicit out-of-scope pin
(so a future tightening of `tryExtractEdits` lights this test up
intentionally).
- `test/e2e/skillopt-loop.serial.test.ts` (7 cases):
- HAPPY PATH: stubbed `gateway.chat` acts as both target agent (emits
sections based on skill content) and optimizer (proposes a real
add-Citations edit). Drives `runSkillOpt` end-to-end against PGLite.
Asserts outcome=accepted, SKILL.md mutated with new section,
frontmatter preserved (D5), history has one committed row,
best.md mirrors disk, delta > epsilon, receipt fields populated.
- 5 broken cases (each isolates a distinct orchestrator-visible failure):
1. Below-baseline regression: optimizer proposes a destructive edit;
gate rejects with reason=below_baseline; SKILL.md unchanged;
rejected-buffer captures the bad edit for anti-bias context.
2. Malformed reflect JSON: orchestrator degrades gracefully to
no_improvement without crashing.
3. Anchor-not-found: applyEditBatch rejects all; sel gate skipped;
rejected-buffer captures with reason=apply_failed.
4. Budget exhausted mid-step: outcome=aborted, no pending rows survive.
5. Converged-skill re-run: starting from already-perfect skill →
no_improvement (no thrash on a well-tuned starting point).
- IDEMPOTENT RE-RUN: drive runSkillOpt twice in sequence. Run 1 accepts.
Run 2 sees improved baseline, no failures, returns no_improvement.
SKILL.md byte-identical to post-run-1; history still has exactly 1
committed row. Proves stability at the fixed point.
All hermetic (no DATABASE_URL, no API keys). PGLite in-memory engine,
tempdir SKILL.md + benchmark, stubbed gateway.chat via
`__setChatTransportForTests`. `.serial.test.ts` because the stub installs
module state and the loop walks shared disk state across epochs.
Test counts after fix: 174 skillopt-surface tests pass (149 pre-existing
unit + 15 new reflect unit + 3 existing E2E + 7 new E2E). Typecheck clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cycle): align ALL_PHASES skillopt position with actual dispatch order
v0.42.0.0 added skillopt to ALL_PHASES right after `patterns` (line 127), but
the dispatch block in runCycle (line ~1912) actually runs skillopt between
`conversation_facts_backfill` and `embed`. The two were inconsistent, and the
serial test `report.phases.map(p => p.phase)).toEqual(ALL_PHASES)` was failing
on master because of it.
A second pre-existing failure: the two phase-count assertions in
`test/core/cycle.serial.test.ts` still said `toBe(20)` even though
ALL_PHASES grew to 21 when skillopt was added. The author bumped the array
but forgot the test.
Two fixes, one commit:
1. Move `'skillopt'` in ALL_PHASES from after `patterns` to between
`conversation_facts_backfill` and `embed`, matching where runCycle
actually dispatches it. Runtime behavior is unchanged — only the
declaration order moves. Updated the surrounding comment to call out
the position invariant and reference the test that pins it.
2. Update both `toBe(20)` assertions in cycle.serial.test.ts to `toBe(21)`
with a v0.42.0.0 history line in the running comments.
Why declaration follows runtime (not the other way around): the comment
intent ("Runs AFTER patterns — graph-fresh") is still satisfied because
"after the entire main graph-mutating cluster" is strictly fresher than
"right after patterns". No design intent is lost.
Test result: cycle.serial.test.ts is now 28/28 (was 27/28 on master + my
prior commit). Skillopt suite still 174/174.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ci): bump PHASE_SCOPE assertion to 21 + fix skill-optimizer Anti-Patterns case
Two CI failures pre-existing on this branch since the v0.42.0.0 skillopt
cathedral landed; master is green because skillopt didn't exist there yet.
1. test/phase-scope-coverage.test.ts asserted ALL_PHASES.length === 20.
skillopt is the 21st phase. Bumped to 21 with v0.42.0.0 history line
in the comment chain. Sibling fix to the cycle.serial.test.ts bump
in commit 08ad2468.
2. skills/skill-optimizer/SKILL.md had `## Anti-patterns` (lowercase p).
skills-conformance.test.ts asserts `## Anti-Patterns` (capital P) as
the required section header. Single-character rename.
Local: 174 skillopt-surface tests + 6 phase-scope tests + 249 skills-
conformance tests all green. Typecheck clean.
Remaining CI delta: 5 put_page facts backstop failures in shard 10 that
reproduce only on Linux CI, not locally even with empty env / cleared
HOME / max-concurrency=1. The error surface is `r.isError === true` with
no further detail captured in the bun:test output. Pushing these 2 fixes
first to narrow the CI signal; will instrument if the 5 persist.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(e2e): align dream-cycle-phase-order + onboard-full-flow with v0.41/v0.42 reality
Two stale E2E assertion files surfaced by a full local E2E run against
real Postgres (the gbrain-test-pg container on port 5434). Neither file
is in the CI E2E job (CI only runs mechanical.test.ts + mcp.test.ts +
skills.test.ts + zeroentropy-live.test.ts), so the drift has been latent.
1. `test/e2e/dream-cycle-phase-order-pglite.test.ts`
EXPECTED_PHASES was missing 4 phases that landed in master since the
list was last revised:
- extract_atoms (v0.41 T9 — atom extraction, after extract_facts)
- synthesize_concepts (v0.41 T9 — concept synthesis, after patterns)
- conversation_facts_backfill (v0.41.11.0, after calibration_profile)
- skillopt (v0.42.0.0 — self-evolving skills, between
conversation_facts_backfill and embed)
Updated to 21 entries in the actual runtime dispatch order (matches
ALL_PHASES exactly). 5/5 tests in the file pass after.
2. `test/e2e/onboard-full-flow.test.ts`
`runAllOnboardChecks` shape test asserted exactly 4 checks; v0.42's
type-unification cathedral (PR #1542, T13-T15) added 3 more
(`pack_upgrade_available`, `type_proliferation`, `dangling_aliases`)
for a total of 7. And `empty brain returns 0 remediations` regressed
because `pack_upgrade_available` can emit a manual_only remediation
on brains where gbrain-base@1.x is active and gbrain-base-v2 is
registered as a successor. Tightened that assertion to `total <= 1`
AND kept a per-check guard asserting takes_count remediations stay 0
(the original test's load-bearing claim — A12 two-gate consent).
13/13 tests in the file pass after.
Honest scope: 4 other E2E files still fail locally after this commit
(cycle.test.ts, dream.test.ts, phantom-redirect.test.ts,
sync-lock-recovery.test.ts), each for a distinct pre-existing master
bug unrelated to v0.42 skillopt work:
- cycle.test.ts (5 fails): PostgresEngine.getConfig falls back to
db.getConnection() singleton via the `get sql()` getter when no
poolSize is set; the new conversation_facts_backfill phase chain
hits this fallback even though the test's setupDB() connects both
the singleton AND the engine. Race condition between the test's
singleton lifecycle and the phase's getConfig call. Deeper fix
needed in PostgresEngine.getConfig (use this._sql directly with
explicit fallback only on user-driven CLI paths).
- dream.test.ts (1 fail): expects "concepts/testing" slug to appear
in dream cycle output, gets empty array. Related to v0.42 concept
type-unification semantics.
- phantom-redirect.test.ts (2 fails): concurrent-sync race +
postgres-js text-string embedding survival. Master-level data-path
bug; would need its own fix wave.
- sync-lock-recovery.test.ts (1 fail): `gbrain sync --break-lock
--all` exits 0 but test expects 1 with a shell-loop hint. CLI
behavior changed in a master commit; need to either restore the
refusal behavior or update the assertion.
None of these 4 block CI (E2E job doesn't run them). Filed as a
TODOS.md entry for a follow-up wave; the 2 in this commit are the
ones that mirror v0.42 work landing.
Local: 130/136 E2E files green, 927/940 tests pass (was 925/940
before these fixes; the 2 files this commit fixes added 7 newly-
passing tests).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ci): quarantine query-cache-knobs-hash.test.ts to serial runner
CI shard 10 (commit 4d721077) failed 5 tests in the
`SemanticQueryCache cross-mode isolation (CDX-4 hotfix)` describe block,
all ~7-34ms each, all expecting writes/reads to round-trip through one
shared PGLite engine + a `beforeEach DELETE FROM query_cache`. Passes
9/9 locally; fails 5/9 on Linux CI under bun's default in-file
max-concurrency=4.
Classic intra-file concurrency race shape: test A's `beforeEach`
clears the table → test A's `store` writes a row → test B's
`beforeEach` (concurrent with A's `store`) clears the table → test A's
follow-up COUNT query returns 0. Same root cause that quarantined
`embed-stale.test.ts`, `brain-allowlist.test.ts`, and
`schema-pack-find-pack-successors.test.ts` to the serial runner in
prior fix waves (documented in v0.41.22.0 CI fix wave).
Fix: rename to `query-cache-knobs-hash.serial.test.ts` so the v0.26.7
serial-tests runner picks it up at `max-concurrency=1`. Tests still
exercise the actual cache logic — no test deleted, no production code
changed. The describe block's `beforeAll` engine + `beforeEach`
TRUNCATE pattern works correctly at serial concurrency.
Local: 12/12 in this file + 52/52 in the serial runner. Production
SemanticQueryCache code is untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(heavy): frontmatter_scan_wallclock — opt into --no-embedding so CI runners work
Heavy tests workflow run 26542447602 (commit 483a5577) failed on the
first heavy script:
[fm_wallclock] FAIL: gbrain init exited non-zero
No embedding provider configured. Set one of:
OPENAI_API_KEY / ZEROENTROPY_API_KEY / VOYAGE_API_KEY
Or defer setup: gbrain init --pglite --no-embedding
The v0.37 D9 hard-require landed in init.ts: `gbrain init --pglite` now
refuses to proceed without an embedding provider configured. The
heavy-tests GitHub workflow doesn't pipe any embedding API keys
(deliberate — the heavy tests measure ops shape, not LLM behavior), so
every CI invocation now blocks at step 2 of this script.
The script's whole purpose is measuring `gbrain doctor`'s
frontmatter-scan wallclock — it never embeds, never calls
`gbrain embed`, never queries vectors. The right fix is to opt out of
the provider requirement via the same `--no-embedding` flag init.ts
already exposes for this exact "deferred setup" case.
Verified locally:
TMP=$(mktemp -d); GBRAIN_HOME="$TMP" \
bun run src/cli.ts init --pglite --yes --no-embedding
# exit 0, brain initialized.
No production code change. One-line + comment in the script.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(heavy): sync_lock_regression — pass --no-embed so CI runs measure lock contention, not key absence
Heavy tests workflow run 26542545802 (commit 7962d312, after the
previous fm_wallclock fix) failed at the next heavy script in the chain:
[sync_lock_regression] outcomes: winners=0 losers=0 unknown=4
[sync_lock_regression] FAIL: expected 1 winner, got 0
[sync_lock_regression] FAIL: expected 3 lock-busy losers, got 0
Each of the 4 parallel `gbrain sync` invocations failed for the same
reason — none of them ever even got to the lock-acquire step:
Embedding model "zeroentropyai:zembed-1" requires ZEROENTROPY_API_KEY.
Re-run with --no-embed to import-only and embed later once the key is set.
The CI runner doesn't pipe any embedding-provider API keys (deliberate —
heavy tests measure ops shape, not LLM behavior), and sync now hard-fails
when its embed step can't reach a configured provider.
This script measures the writer-lock race shape — `gbrain-sync` row in
`gbrain_cycle_locks`, exactly-one-winner semantics, N-1 fail-fast losers
with "Another sync is in progress", zero leaked rows post-run. It never
needed embeddings; the original write predates the hard-require landing.
Fix: pass `--no-embed` to the sync invocation. Same kind of fix as
fm_wallclock (commit 7962d312) but on the sync side rather than init.
No production code touched. One-line change in the bash script.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(heavy): sync_lock_regression — register source via psql + use --repo + tolerate doctor warns
Heavy tests run 26542638471 (commit 60145eee, after the --no-embed
fix) failed at the same script but at a downstream step:
> Source "default" has no local_path. Run: gbrain sources add default --path <path>
Three independent bugs in the script that all surfaced at once after
v0.41's source-registry landed:
1. `gbrain config set sync.repo_path` is the legacy way; sync now
reads `sources.local_path` first. Replaced with an upsert into the
sources table via psql:
INSERT INTO sources (id, name, local_path)
VALUES ('default', 'default', $BRAIN_DIR)
ON CONFLICT (id) DO UPDATE SET local_path = EXCLUDED.local_path
Kept the legacy `config set sync.repo_path` line too as
belt-and-suspenders for any downstream caller that still reads it.
2. `gbrain sync --dir <path>` is silently ignored; sync's CLI parser
recognizes `--repo`, not `--dir`. Switched to `--repo`.
3. `bun run src/cli.ts doctor --json` at the top (used to apply
migrations as a side effect) exits non-zero whenever ANY check
warns — including the new "no embedding provider configured"
warning on a fresh CI runner. The script's `set -e` aborted at
line 53 before reaching any of the sync invocations. Added `|| true`
since the migration runs regardless of doctor's exit verdict.
Verified locally — `DATABASE_URL=... bash tests/heavy/sync_lock_regression.sh`
output:
[sync 1] rc= (lock-busy: 'Another sync is in progress')
[sync 2] rc=0 (winner)
[sync 3] rc= (lock-busy: 'Another sync is in progress')
[sync 4] rc= (lock-busy: 'Another sync is in progress')
outcomes: winners=1 losers=3 unknown=0
post-run gbrain_cycle_locks(gbrain-sync) row count: 0
OK — 1 winner, 3 lock-busy losers, no leaked lock rows.
Production code untouched. All three fixes are in the bash script.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(skillopt): hands-on tutorial for auto-improving a skill + discoverability
There was no tutorial for skillopt — only a reference guide
(docs/guides/skillopt.md) that opens at --bootstrap-from-routing and
assumes you already understand benchmarks, and an agent-facing SKILL.md.
README had ZERO skillopt mention. The one thing a user must hand-author
(the benchmark JSONL) was taught nowhere with a worked example.
New: docs/tutorials/improving-skills-with-skillopt.md — Diataxis tutorial
(learning-oriented), copy-pasteable end to end:
1. mental model in two sentences (SKILL.md is the trainable param, the
agent is frozen)
2. write your first benchmark from scratch — a complete 15-task rule-judge
starter you paste and run, with the full check-op table
(contains/regex/section_present/max_chars/min_citations/tool_called/
tool_not_called)
3. --dry-run cost preview (and that it exits 2 by convention, not failure)
4. real run + reading accepted(0)/no_improvement(1)/aborted(2) with the
actual stderr output shape
5. where output lands (best.md, versions/, history.json, rejected.json,
audit jsonl)
6. accept/reject — bundled vs user skills, --no-mutate vs
--allow-mutate-bundled
7. iterate by sharpening the benchmark
The load-bearing fix the tutorial makes that the reference guide got wrong:
the DEFAULT --split 4:1:5 needs ~50 tasks before it runs (sel = N/10, floor
5). A first-time author writing 10-15 tasks hits `D_sel has N task(s)
(need >=5)` and bounces. The tutorial ships 15 tasks + `--split 1:1:1`
(clean 5/5/5) so the copy-paste path actually works. Verified against the
real loadBenchmark + splitBench: the exact shipped block parses 15 unique
tasks and splits 5/5/5 with sel>=5; the system's own error message confirms
"need ~50 total for 4:1:5".
Discoverability (Diataxis cross-linking):
- README.md tutorials section: new entry (was zero skillopt mention)
- docs/tutorials/README.md: added under ## Shipped
- docs/guides/skillopt.md: "New to this? Start with the tutorial" callout
Every claim devex-verified against source: exit-code map from
skillopt.ts (accepted:0/no_improvement:1/aborted:2/errored:2), stderr
format from skillopt.ts:286-292, check ops from score.ts, output paths
from SKILL.md, split math from benchmark.ts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: regenerate llms-full.txt after skillopt tutorial + README edit
Refreshes the inlined doc bundle so the committed llms-full.txt matches
fresh `bun run build:llms` output (test/build-llms.test.ts drift guard).
Picks up the README tutorials-section edit from c39dbdb1. The new tutorial
file itself isn't curated into scripts/llms-config.ts (the bundle curates
a fixed doc set, not every tutorial) — this is purely the README delta.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ci): stop embed-preflight leaking gateway config into facts-backstop shard
CI shard 10 failed 5 `put_page facts backstop` tests with:
[embed(openai:text-embedding-3-small)] Incorrect API key provided: sk-test
(captured by the diagnostic stderr added in a prior commit). Root cause is
a cross-file module-state leak, not a logic bug:
- `embed-preflight.test.ts` calls `configureGateway({env:{OPENAI_API_KEY:
'sk-test'}})` to drive credential-validation scenarios. It resets the
gateway `beforeEach` but never AFTER its last test, so it leaves the
gateway configured with `sk-test`.
- bun runs every file in a shard inside ONE process. The residual config
bleeds into the next file. When `facts-backstop-gating.test.ts` lands in
the same shard, its put_page calls see `isAvailable('embedding') === true`
(the key is *present*, just invalid), so put_page attempts a real embed
and 401s before the backstop gating even runs.
- It's intermittent across master merges because shard bin-packing changes
which files co-locate. (It "resolved" after the v107 merge earlier for
exactly this reason, then came back.)
R1/R2 test-isolation lint doesn't catch this — it's `configureGateway`
module state, not `process.env` or `mock.module`.
Two fixes, both using the gateway's own `resetGateway()` seam (no
process.env, R-compliant):
1. embed-preflight.test.ts — `afterAll(() => resetGateway())` so the leaker
cleans up after the whole file. Primary fix; also protects any OTHER
shard-mate that reads gateway state.
2. facts-backstop-gating.test.ts — `beforeEach(() => resetGateway())` so the
suite is deterministic regardless of ambient gateway config. Defense in
depth: isAvailable('embedding') is now reliably false → put_page uses
noEmbed → the import never embeds → only the backstop gating (the suite's
actual subject) is exercised.
Verified: running leaker+victim in one process (the shard repro) goes
16/16; full shard 10 goes 1208/1208 (was 5 fail in CI). Typecheck clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(skillopt): make benchmark authoring an agent job, not a human chore
The prior tutorial taught a human to hand-write a 15-task benchmark — but
nobody does that. The real workflow is: user says "make skill X better,"
the AGENT authors the benchmark and runs the optimizer. The agent-facing
dispatcher didn't actually cover that.
Gap found: skill-optimizer/SKILL.md documented exactly one authoring path,
`--bootstrap-from-routing`, which (a) requires a pre-existing
routing-eval.jsonl (bootstrap-benchmark.ts:57-63 refuses without it) and
(b) generates tasks from ROUTING fixtures — which test dispatch ("does
this phrasing pick this skill"), not output quality. So an agent told to
improve a skill with no benchmark had no documented way to author a
*quality* benchmark; it'd have to reinvent the JSONL format the human
tutorial teaches.
Two fixes:
1. skills/skill-optimizer/SKILL.md — new "Authoring the benchmark yourself
(the common case)" section: read the target SKILL.md, generate ~15
realistic tasks, attach rule judges (contains/max_chars/min_citations/
section_present/regex/tool_called), write the JSONL, run with
`--split 1:1:1` (the default 4:1:5 needs ~50 tasks). Decision-tree row
"New skill, no benchmark" now says "Author one" instead of pointing at
bootstrap-from-routing; the bootstrap row is reframed as a head-start
that only applies when routing fixtures exist and notes routing tasks
test dispatch, not quality.
2. docs/tutorials/improving-skills-with-skillopt.md — new "The easiest
path: ask your agent" section up top. Tells humans to just tell their
agent "improve my X skill — write a benchmark first," and frames the
manual walkthrough as "read this when you want to understand or
hand-curate what the agent is doing."
Verified: conformance 249/0, resolver 99/0, build-llms drift guard 7/0,
cross-link resolves.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(skillopt): --bootstrap-from-skill starter benchmark generator
Generate a quality benchmark from a skill's SKILL.md directly, no
routing-eval.jsonl required. One LLM call emits JSONL tasks (each with rule
judges) that the agent reviews + strengthens before optimizing.
- runBootstrapFromSkill: JSONL output parsed line-by-line with skip-bad-line
salvage (a truncated final line drops, the rest survive); a task is kept only
when >=2 valid rule checks survive; provider errors propagate instead of
collapsing to bootstrap_empty.
- --bootstrap-tasks N (default 15, cap 50); maxTokens scales with the count.
- Extracted assertBenchmarkAbsent + readSkillBodyOrThrow shared with the routing
bootstrap; hardened runBootstrap's routing-eval parse to skip malformed lines.
- CLI: --bootstrap-from-skill short-circuit + 6-way mutual exclusion; parseFlags
exported for unit tests. The benchmark-not-found hint + --help now point here.
- The generator's REVIEW line prints the paste-ready
`--bootstrap-reviewed --split 1:1:1` next command (the default 4:1:5 split
refuses a 15-task starter at D_sel >= 5).
- 20 hermetic cases incl. round-trip into loadBenchmark + splitBench(1:1:1).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(skillopt): make --bootstrap-from-skill the primary no-benchmark path
The agent runs --bootstrap-from-skill, strengthens the generated judges (they
are weak drafts), deletes the sentinel, then runs --bootstrap-reviewed
--split 1:1:1. Freehand authoring is demoted to the fallback for the rare skill
the generator can't draft well. Updates the Iron Law, decision tree, and
anti-patterns to cover both bootstrap modes and the 15-task / --split 1:1:1
gotcha.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(release): v0.42.1.0 --bootstrap-from-skill
VERSION + package.json -> 0.42.1.0, CHANGELOG entry, CLAUDE.md skillopt
annotation, regenerated llms-full.txt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: surface --bootstrap-from-skill in README + skillopt reference
- docs/guides/skillopt.md: 30-second pitch leads with --bootstrap-from-skill;
flag table adds --bootstrap-from-skill + --bootstrap-tasks rows.
- README.md: skillopt tutorial pointer mentions generating a starter benchmark.
- Regenerated llms-full.txt (README is in the bundle).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ci): bump FULL_SIZE_BUDGET 700KB→750KB for legitimate CLAUDE.md growth
The skillopt wave annotations + merged v0.41.34-36 master releases pushed
llms-full.txt to 700,423 bytes — 423 over the 700KB cap — failing the
build-llms size-budget test on CI shard 6. CLAUDE.md is ~540KB (77% of the
bundle) and is the whole point of the one-fetch artifact, so it stays inlined;
the budget tracks its per-release growth. 750KB still fits 200k+ context models.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* skill: compress-agents-md — functional-area resolver pattern
Proven via A/B eval: 100% routing accuracy at 48% size reduction.
Converts granular per-skill resolver rows into functional-area dispatchers
with '(dispatcher for: ...)' sub-skill lists.
Includes:
- SKILL.md with full pattern docs, before/after examples, eval results
- routing-eval.jsonl with 5 fixtures
- Anti-patterns (resolver-of-resolvers pipe table = 15% accuracy)
* skill: rename compress-agents-md → functional-area-resolver, cite prior art
The contribution is a pattern (functional-area dispatcher with `(dispatcher
for: ...)` clauses), not a file. Rename describes the contribution; triggers
broaden to cover both AGENTS.md and RESOLVER.md phrasings.
SKILL.md rewrite:
- Three-model A/B table (Opus 4.7 / Sonnet 4.6 / Haiku 4.5) replaces the
original Sonnet-only claim. Functional-areas beats baseline by +13 to +17pp
training (lenient) across all three models at 48% the size.
- Strict + lenient scoring documented side by side. Lenient (predicted shares
dispatcher area with expected) matches production agent behavior.
- Preconditions added: refuse to compress if file <12KB or working tree dirty.
- Multi-file routing precedence section for the v0.31.7 RESOLVER.md/AGENTS.md
merge case.
- Mandatory verification step (≥95% via the harness).
- Daily-doctor.mjs reference scrubbed (didn't exist in gbrain).
- Three prior-art citations: AnyTool (arXiv:2402.04253), RAG-MCP
(arXiv:2505.03275), Anthropic Agent Skills progressive disclosure. The
pattern is the static-prompt analog of runtime hierarchical routing.
routing-eval.jsonl: 8 positive (5 original + 3 broadened triggers) + 4
adversarial negatives targeting skillify, skill-creator, book-mirror,
concept-synthesis to prove broadened triggers don't over-capture adjacent
meta-skills.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* evals: A/B harness for functional-area-resolver (gateway-routed, strict + lenient scoring)
evals/functional-area-resolver/ lives outside skills/ deliberately. The
skillpack bundler walks skills/<skill>/ recursively, so an eval surface in
there would copy harness + variants + fixtures + tests into every downstream
install. The pattern (in SKILL.md) ships everywhere; the eval evidence stays
in the gbrain repo.
What ships:
- Three variant resolvers in variants/ — baseline.md (verbose 25KB) and
functional-areas.md (compressed 13KB) extracted from a real production
AGENTS.md at git commits 93848ff3b^ and 93848ff3b (owner PII scrubbed).
resolver-of-resolvers.md derived mechanically by stripping (dispatcher
for: ...) clauses — the ablation case.
- 20 hand-authored training fixtures + 5 held-out blind fixtures.
- harness-runner.ts — TypeScript runner via gbrain gateway. Flags:
--model {opus|sonnet|haiku|<full-id>}, --variants-dir, --variants for
description-length sweeps, --parallel N (rate-lease bound), --limit N
for smoke runs, --yes for non-TTY.
- Every output row carries BOTH `correct` (strict) and `correct_lenient`
(predicted shares dispatcher area with expected). Lenient matches
production behavior.
- Receipt header binds (model, prompt_template_hash, fixtures_hash,
harness_sha, ts, cmd_args). Re-runs are auditable.
- harness.mjs — thin Node shim that spawns the TS runner via bun.
- rescore.mjs — zero-cost lenient re-score of an existing JSONL.
- harness-runner.test.ts — 45 unit tests (no API key needed) covering
every pure function plus the dispatcher-list parser.
The prompt template is load-bearing: without the "drill into (dispatcher
for: ...) list" instruction, every compression variant collapses to
~30-60%. Documented in SKILL.md and README.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* evals: baseline receipts (Opus 4.7 + Sonnet 4.6 + Haiku 4.5, 2026-05-11)
Three canonical 225-row receipts (3 variants × 25 fixtures × 3 seeds per
model). Each receipt header binds (model, prompt_template_hash,
fixtures_hash, harness_sha, ts) so the published SKILL.md numbers are
reproducible.
Training corpus (n=20, lenient):
baseline | Opus 81.7% | Sonnet 86.7% | Haiku 73.3% | 25KB
functional-areas | Opus 98.3% | Sonnet 100% | Haiku 88.3% | 13KB
resolver-of-resolvers | Opus 63.3% | Sonnet 41.7% | Haiku 65.0% | 10KB
functional-areas beats baseline by +13 to +17pp across all three models at
48% the size. resolver-of-resolvers' Sonnet collapse (41.7%) is the SKILL.md
"compression without dispatcher clause is broken" claim, observed.
Held-out (n=5, lenient) saturates at 100% across most cells (Sonnet ×
resolver-of-resolvers is 73.3% — the same failure mode visible on a smaller
sample).
~$3 API spend across all three runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* skill: wire functional-area-resolver into RESOLVER.md + manifests
skills/RESOLVER.md gets a new row in Operational, adjacent to skillify.
Triggers: "Compress my resolver", "AGENTS.md too large", "RESOLVER.md too
big", "functional area dispatcher", "shrink routing table".
skills/manifest.json adds the new entry and bumps manifest version
0.25.1 → 0.32.3.0 (loadOrDeriveManifest reads this for sync-guard).
openclaw.plugin.json adds functional-area-resolver to the skills array
and bumps version 0.25.1 → 0.32.3.0 so install receipts stop being stale
(src/core/skillpack/installer.ts:307-311 uses manifest version on every
install).
Verified:
- gbrain check-resolvable --json: 42/42 reachable, 0 errors.
- gbrain routing-eval: 70/70 pass (100% structural).
- bun test test/skillpack-sync-guard.test.ts: passes (manifest in sync).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.32.3.0 skill: functional-area-resolver — pattern for compressing routing tables
Headline: compress a 25KB AGENTS.md down to 13KB without losing routing
accuracy. Pattern proven across Opus 4.7, Sonnet 4.6, and Haiku 4.5 — beats
the verbose baseline by +13 to +17pp at 48% the size.
Empirical (training, n=20, 3 seeds, lenient):
baseline 25KB: Opus 81.7% | Sonnet 86.7% | Haiku 73.3%
functional-areas 13KB: Opus 98.3% | Sonnet 100% | Haiku 88.3%
resolver-of-resolvers 10KB: Opus 63.3% | Sonnet 41.7% | Haiku 65.0%
The (dispatcher for: ...) clause is the load-bearing signal. Strip it (the
resolver-of-resolvers variant) and Sonnet collapses to 41.7% — the failure
case the pattern's authors predicted, now observed.
Files in this release:
- VERSION + package.json bumped to 0.32.3.0 (4-segment per CLAUDE.md).
- CHANGELOG.md: full empirical story, cross-model table, three prior-art
citations (AnyTool, RAG-MCP, Anthropic Agent Skills progressive
disclosure).
- TODOS.md: nine v0.33.x follow-ups (dogfood on gbrain's own RESOLVER.md,
CLI promotion to gbrain routing-eval --ab-compare, held-out corpus
growth, cross-vendor Gemini+GPT verification, per-row description
length sweep, structural compression to ~10KB, hierarchical
area-of-areas, embedding pre-router, adversarial fixtures,
prompt-design ablation doc).
- llms-full.txt regenerated.
Bisect-friendly history on this branch:
502d447e skill: rename + content rewrite + routing-eval.jsonl
472cc686 evals: A/B harness + variants + fixtures + tests (no receipts)
243e013e evals: cross-model baseline receipts (Opus + Sonnet + Haiku)
ecab180b skill: wire-up to RESOLVER.md + manifest.json + openclaw.plugin.json
THIS: v0.32.3.0 release marker
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* evals: codex review fixes — accept ASCII -> arrow + provider-aware auth gate
Two P2 findings from /codex review on commit 8870c64e:
P2-2: parseDispatcherLists regex required Unicode `→`, but SKILL.md
Step 4 documents the template with ASCII `->`. Downstream-authored
resolvers following the template silently fell through to strict-only
scoring (correct_lenient == correct always), under-reporting same-area
accuracy with no warning. Regex now accepts both `→` and `->`. Two
new test cases pin the behavior — pure-ASCII variant + mixed-arrow
variant.
P2-3: main() exited with `ANTHROPIC_API_KEY is not set` even when the
user passed `--model openai:gpt-4o` with a valid OPENAI_API_KEY. The
CLI advertises full provider:model support (resolveModel tests cover
openai:* explicitly) and the gateway routes by recipe; the env check
should match the provider that will actually be called. Now extracts
the provider id from the model string and looks up the right env var
from REQUIRED_ENV_BY_PROVIDER (anthropic, openai, google, groq,
voyage, together, deepseek, minimax, dashscope, zhipu). Unknown
providers fall through to the gateway, which raises a clear
recipe-specific error.
47/47 harness unit tests pass after the change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* skill: codex review P2-1 — verification gate now tests the user's edited file
The original SKILL.md Step 6 told users to run `node harness.mjs` from the
gbrain repo as the mandatory ≥95% gate. But that runs the harness against
the COMMITTED sample variants in evals/functional-area-resolver/variants/,
not the file the user just compressed. The gate could pass while the edit
dropped a sub-skill.
Step 6 now:
- Gate 1 stays at `gbrain routing-eval --json` (structural, runs against
the user's actual routing-eval.jsonl fixtures).
- Gate 2 is rewritten: copy the user's edited routing file into a tmp
variants dir, then run `node harness.mjs --variants-dir <tmp>
--variants my-edit --model opus`. This exercises the harness's existing
--variants flag (added in commit 472cc686 / T4) but now points at the
user's actual edit. The harness uses gbrain-bundled fixtures, so this
is a regression check on shared skills, not a full eval of the user's
fixture set — and the SKILL.md says so explicitly.
Also adds a "common false negatives" callout: when the user's routing
file doesn't expose the skills gbrain's bundled fixtures target (e.g.
`gmail`, `enrich`), expect strict-scoring fails on those rows; lenient
scoring remains accurate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* evals: codex review P3 — regenerate Opus baseline with current schema
The prior Opus receipt was generated before commit 472cc686 (T4 added
harness_sha to ReceiptRow and correct_lenient to every RunRow). The
Sonnet and Haiku receipts shipped with the new schema, but Opus was
the outlier.
This run was produced with the current harness (sha ca99fbfeb, after
the P2-1 + P2-2 + P2-3 fixes). The harness_sha in the receipt header
binds the numbers to a specific harness revision so consumers can detect
schema drift.
Numbers (training, lenient, n=20, 3 seeds):
baseline: 81.7% ± 7.2% (unchanged — strict and lenient are equal)
functional-areas: 100% ± 0% (was 98.3% — one nondeterministic seed
is now in-cluster; pattern continues
to beat baseline at 48% the size)
resolver-of-resolvers: 66.7% ± 7.2% (was 63.3% — still in noise; absent
dispatcher clause keeps it ~30pp
behind functional-areas on training)
Held-out (n=5, 3 seeds, lenient): all variants 100% except resolver-of-
resolvers on Sonnet (committed in earlier baseline) — Opus held-out
saturates the small fixture set.
Run cost: ~$1.40 at Opus 4.7 pricing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* post-merge: scrub fork-private paths + add Contract/Output Format sections
Two CI gates landed on master after this branch was cut:
1) scripts/check-privacy.sh (v0.32.2): banned /data/brain/ and /data/.openclaw/
in committed files. The eval variants extracted from a real production
AGENTS.md still contained those fork-private path literals. Rewrote to
/your/brain/path/, /your/agent/.openclaw/, /your/gbrain, /your/gstack,
/your/tmp, /your/git-projects/. Only path strings changed — the routing
structure (skill names, dispatcher clauses, trigger phrases) is byte-for-
byte identical, so harness baseline-runs/ receipts are still valid.
2) test/skills-conformance.test.ts (master): added required sections
`## Contract` and `## Output Format` to every skill. Added both to
skills/functional-area-resolver/SKILL.md following the book-mirror
convention (short body referencing the canonical content above + a
conformance-test footnote). Contract notes the privacy guarantee +
the verification-gate semantics; Output Format documents the area
entry template (with both ASCII -> and Unicode → arrows accepted).
Full unit suite: 5578 pass / 0 fail. bun run verify clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: surface functional-area-resolver in CLAUDE.md + README.md for v0.32.3.0
CLAUDE.md — adds a "Routing-table compression (v0.32.3.0)" entry under Skills,
covering the two-layer dispatch pattern, the load-bearing (dispatcher for: ...)
clause, the eval surface at evals/functional-area-resolver/, the three
cross-model baseline receipts, the 25KB → 13KB compression numbers, and the
nine v0.33.x follow-up TODOs. Cites AnyTool / RAG-MCP / Anthropic Agent Skills
prior art so the pattern's position in the literature is discoverable from the
agent entry point.
README.md — adds a "New in v0.32.3.0" callout in the intro section so users
landing on the repo see the new skill before scrolling to the skills list.
Links the SKILL.md and eval directory; states the cross-model gain (+13 to
+17pp at 48% the size) so the reason to apply the pattern is one click away.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: garrytan-agents <garrytan-agents@users.noreply.github.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(engines): pre-add v0.20 + v0.26.3 forward-reference columns in bootstrap
The forward-reference bootstrap (PostgresEngine + PGLiteEngine
applyForwardReferenceBootstrap) covered v0.18 + v0.19 + v0.26.5 columns
but missed two later groups. Brains upgrading from v0.14-era to current
master crash before the migration ladder runs:
1. v0.20 Cathedral II — content_chunks.search_vector,
parent_symbol_path, doc_comment, symbol_name_qualified.
`CREATE INDEX idx_chunks_search_vector` and
`CREATE INDEX idx_chunks_symbol_qualified` in schema.sql/PGLITE_SCHEMA_SQL
crash with "column search_vector does not exist" / "column
symbol_name_qualified does not exist".
2. v0.26.3 — mcp_request_log.agent_name, params, error_message.
`CREATE INDEX idx_mcp_log_agent_time ON mcp_request_log(agent_name,...)`
crashes with "column agent_name does not exist".
Reproduces deterministically on a v0.13/v0.14 brain upgraded straight
to current master. The user hits the wall before any of v15-v36 can run.
Both engines now probe for these columns and pre-add them via
`ALTER TABLE ADD COLUMN IF NOT EXISTS` before SCHEMA_SQL runs. Migrations
v26, v27, v33 still run later via runMigrations and remain idempotent
(they handle backfill on top of the bootstrap-added columns).
Test coverage extended in test/schema-bootstrap-coverage.test.ts:
REQUIRED_BOOTSTRAP_COVERAGE now lists 6 new forward references; the
strip-and-rebuild block drops the corresponding indexes/triggers so the
test exercises a brain that pre-dates v0.20 + v0.26.3 migrations.
Repro: brain on schema v13/v14 + run `gbrain init --migrate-only` against
current master → fails. With this patch → succeeds; ladder runs to v36.
* fix(engines): pre-add v0.27 subagent_messages.provider_id in bootstrap
PR #682 covered v0.20 (chunks) + v0.26.3 (mcp_request_log) but missed
v0.27's subagent_messages.provider_id. The composite index
`idx_subagent_messages_provider ON subagent_messages (job_id, provider_id)`
in PGLITE_SCHEMA_SQL crashes on brains pinned at v0.18-v0.26 because
provider_id is the SECOND column in the composite — array-extraction
patterns that scan only first-column references miss it entirely.
This is the wedge surfaced by issue #670 (v0.22.0 → v0.27.0 init
--migrate-only crashes with "column 'provider_id' does not exist") and
contributing to #661/#657.
Both engines now probe for subagent_messages.provider_id and pre-add
the column via ALTER TABLE ADD COLUMN IF NOT EXISTS before SCHEMA_SQL
runs. Migration v36 (subagent_provider_neutral_persistence_v0_27) still
runs later via runMigrations and remains idempotent.
Note on the test side: REQUIRED_BOOTSTRAP_COVERAGE is hand-maintained
and just gained a v0.27 entry. v0.28.5's Step 3 replaces this array
with a SQL parser that auto-derives coverage from PGLITE_SCHEMA_SQL,
including composite-index columns. This commit is the targeted
follow-up to PR #682's cherry-pick; A2's parser closes the class
permanently.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cli): conditional schema-init on connect (closes#651)
Adds `hasPendingMigrations(engine)` next to `runMigrations` in migrate.ts:
single getConfig('version') probe, returns true when current < LATEST_VERSION,
defensively returns true on getConfig failure (treats wedged-config as pending).
`connectEngine` in cli.ts now wraps `engine.initSchema()` in a probe gate:
short-lived CLI calls (gbrain stats, query, doctor, etc.) on already-migrated
brains skip the bootstrap-probe + SCHEMA_SQL replay + ledger-check entirely.
Wedged brains still auto-heal — the probe says "yes pending" and initSchema
runs as before.
Building on oyi77's investigation in PR #652. Same correctness as #652's
unconditional initSchema-on-every-connect, but no perf regression on the
hot path. Failure non-fatal: if probe or init throws, log a hint and let
subsequent operations surface the real error in context.
Test coverage in test/migrate.test.ts: 3 cases covering fully-migrated
(false), version-rewound (true), and missing-version-config (defensive
true). Pairs with v0.28.5's X1 (post-upgrade auto-apply) — the upgrade
path runs initSchema explicitly while every other code path that goes
through connectEngine gets the cheap probe.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(upgrade): post-upgrade auto-applies pending schema migrations (X1)
Prior behavior: `gbrain upgrade` → `gbrain post-upgrade` → `apply-migrations`
only WARNs at apply-migrations.ts:296-302 when schema version is behind
LATEST_VERSION, telling the user to run `gbrain init --migrate-only`. 11
wedge incidents over 2 years have proven users don't read that WARN —
they file an issue instead.
This commit makes `runPostUpgrade` explicitly call `engine.initSchema()`
after the orchestrator migration pass, mirroring `init --migrate-only`'s
flow. Side-effect: `gbrain upgrade` now walks away with a healthy brain
in the cluster A wedge case (#670, #661, #657, #651, #625, #615, #609).
Defensive: wrapped in try/catch so a connection or DDL failure falls
back to the existing user-facing WARN. The hint to run
`gbrain init --migrate-only` is preserved as the manual escape hatch.
Pairs with v0.28.5's A1 (hasPendingMigrations probe in connectEngine):
the upgrade path runs initSchema explicitly here, while every other code
path that goes through connectEngine gets the cheap probe.
Codex outside-voice review caught this gap during plan review: "the plan
still does not prove `upgrade` will actually run schema migrations."
This is the load-bearing fix that makes v0.28.5's headline outcome
("run upgrade, brain works") literally true for cluster A.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(bootstrap): auto-derive coverage from PGLITE_SCHEMA_SQL (A2)
Replaces the hand-maintained REQUIRED_BOOTSTRAP_COVERAGE assertion with a
SQL-parser-backed structural check. The new test:
1. parseIndexColumnReferences(PGLITE_SCHEMA_SQL) extracts every column
referenced by every CREATE INDEX — including composite-index second
and third columns. Codex outside-voice review caught that earlier
first-col-only patterns missed v0.27's
`idx_subagent_messages_provider ON subagent_messages (job_id, provider_id)`,
which is exactly how the v0.28.5 wedge happened.
2. parseBaseTableColumns(PGLITE_SCHEMA_SQL) extracts every column declared
in CREATE TABLE bodies (including via ALTER TABLE ADD COLUMN inside
the schema blob).
3. parseAlterAddColumns(pglite-engine.ts source) extracts every column
that applyForwardReferenceBootstrap adds.
4. Static contract: every (table, column) pair from step 1 must appear in
either step 2 or step 3. Otherwise the test fails loud, names every
uncovered pair, and points at the bootstrap function for the fix.
Self-updating: any future CREATE INDEX added to PGLITE_SCHEMA_SQL on a
column that bootstrap doesn't yet provide fails this test at PR time. No
human required to remember to update an array. Closes the 11-incident
wedge class identified in CLAUDE.md (#239, #243, #266, #357, #366, #374,
#375, #378, #395, #396).
Helper parsers also have their own unit tests covering composite-index
second columns, function-wrapped columns (lower(col)), HNSW operator-class
suffixes (vector_cosine_ops), and ALTER TABLE column extraction. Existing
REQUIRED_BOOTSTRAP_COVERAGE-based tests preserved as a coarse-grained
lower bound; the new parser-based test is the load-bearing structural
gate going forward.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: support Voyage 2048d schema setup
* fix: harden Voyage schema templating
* feat: Voyage 4 embedding support + doctor eval
- Add voyage-4-large/4/4-lite/4-nano + domain models to Voyage recipe
- Fix AI SDK compatibility: strip encoding_format (Voyage rejects 'float'),
patch response to add prompt_tokens from total_tokens
- Add embedding_provider doctor check: live smoke test verifying model,
API key, dimensions, and DB column alignment
- Add embedding provider eval qrels for post-migration quality testing
Closes: Voyage AI integration for gbrain embedding pipeline
* fix: adaptive embed batch sizing for Voyage token limits
Voyage's tokenizer is 3-4x denser than OpenAI tiktoken, causing batches
of 50+ texts to exceed the 120K token-per-batch limit even when DB
token counts (from tiktoken) suggest they'd fit.
Changes:
- Add max_batch_tokens to EmbeddingTouchpoint type (provider-declared limit)
- Set Voyage recipe to 120K token limit
- Gateway embed() now auto-splits batches using conservative char-to-token
estimate (1:1 ratio, 80% budget utilization)
- On token-limit errors, embedSubBatch recursively halves and retries
(down to single-text batches before giving up)
- Reduce embedding.ts BATCH_SIZE from 100 to 50 as a secondary guard
- Add tests for batch splitting logic and error pattern matching
Fixes infinite retry loops where the same oversized batch would fail
repeatedly because WHERE embedding IS NULL re-fetches identical rows.
* fix(init): error on existing-brain dim mismatch + embedding-migration recipe
Adds A4 hard-error path: when `gbrain init --embedding-dimensions N` is
run against an existing brain whose `content_chunks.embedding` column is
a different `vector(M)`, init exits 1 with an inline four-step ALTER
recipe and a pointer to docs/embedding-migrations.md.
This kills the silent-corruption pattern surfaced by issue #673: the
v0.27 schema seeded `('embedding_dimensions', '1536')` regardless of the
flag, so users got a config saying 768 but a column at 1536 — first
sync write blew up with "expected 1536, got 768."
A4's contract:
1. Connect to engine BEFORE saveConfig so we can read the live column type
2. If column exists AND dim != requested, exit 1 (loud failure)
3. If column doesn't exist (fresh init) OR dim matches, proceed normally
Recipe in docs/embedding-migrations.md (and inlined in init's error
output) covers all four destructive steps codex's plan-review caught:
1. DROP INDEX IF EXISTS idx_chunks_embedding (HNSW won't survive ALTER)
2. ALTER TABLE content_chunks ALTER COLUMN embedding TYPE vector(N)
3. UPDATE content_chunks SET embedding = NULL, embedded_at = NULL
4. CREATE INDEX HNSW *only if N <= 2000* (pgvector cap)
Step 4 is conditional: dims > 2000 (e.g. Voyage 4 Large 2048d) cannot
be HNSW-indexed in pgvector; the recipe explicitly says "Skip reindex"
in that case so the user doesn't paste a CREATE INDEX that crashes.
Helper `readContentChunksEmbeddingDim` and message builder
`embeddingMismatchMessage` live in src/core/embedding-dim-check.ts so
doctor 8b (next commit) can reuse the same source of truth.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(gateway): correct dim-mismatch error to point at manual ALTER recipe (#672)
Previous error message recommended running `gbrain migrate --embedding-model
… --embedding-dimensions …`, but `gbrain migrate` only handles engine
migration (postgres ↔ pglite), not embedding reconfiguration. Following
that hint produced a different error and confused users further.
New message:
- Names the actual options: change models OR migrate the existing brain
- Inlines a one-line quick recipe (DROP INDEX → ALTER → UPDATE NULL →
config set → embed --stale)
- Points at docs/embedding-migrations.md (added in commit 306fc0e1)
for the full four-step recipe with HNSW conditional handling
Closes#672. Note: #671 (config show hides embedding_model / dimensions)
appears to be already fixed on master — `Object.entries(loadConfig())`
in config.ts:24 correctly enumerates all keys including embedding_*. Will
close#671 with that note when shipping v0.28.5.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(types): doctor 8b uses portable executeRaw + Voyage fetch-shim cast
#665's doctor 8b dim-probe used `engine.sql\`...\`` directly (Postgres
template literal) which doesn't typecheck against the BrainEngine
interface (only PostgresEngine has the .sql getter; PGLite does not).
Refactored to use `readContentChunksEmbeddingDim` from
src/core/embedding-dim-check.ts — same helper init's A4 hard-error
path uses, runs portably on both engines.
#680's Voyage fetch-shim passes a custom fetch handler to
`createOpenAICompatible` for the encoding_format + prompt_tokens
normalization. The SDK accepts the field at runtime but the typed
parameter on the pinned version doesn't expose it. Cast to the
parameter type so the shim ships without a type error.
Both fixes are mechanical cleanup of cherry-picked PRs that didn't
typecheck against current master's stricter shape. No behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cli): mark cli.ts executable so bun-linked installs work
`package.json` declares `"bin": { "gbrain": "src/cli.ts" }`, and bun's
linker creates `~/.bun/bin/gbrain` as a symlink to the file. The shebang
`#!/usr/bin/env bun` works only when the target file is executable —
otherwise bun runs it as a script (because it sees the script via the
shebang interpreter), but executing the symlinked target itself fails:
$ ls -la ~/.bun/bin/gbrain
lrwxrwxrwx ... -> ../install/global/node_modules/gbrain/src/cli.ts
$ ~/.bun/bin/gbrain --version
/opt/homebrew/bin/bash: line 1: /Users/brandon/.bun/bin/gbrain: Permission denied
This bites the postinstall hook that calls `gbrain apply-migrations`
(masked by the `||` fallback) and any subprocess that invokes the
binary by absolute path (e.g., subagent_messages migration v0.16's
`execSync('gbrain init --migrate-only', ...)`).
Setting the mode in-tree to 755 fixes both. No content change.
* test(ci): guard against src/cli.ts mode-bit regression (cluster C)
Cluster C cherry-pick (#683) restored the executable bit on src/cli.ts.
This commit adds scripts/check-cli-executable.sh that asserts the git
index mode is 100755 and wires it into `bun run verify` (and check:all).
Why a CI guard: bun-link installs symlink to src/cli.ts directly. If the
mode bit ever regresses to 100644, the very first `gbrain --version`
fails with `permission denied` — the exact symptom that motivated #683.
This guard runs in <100ms, fast enough for the inner verify loop.
Failure mode: clear instructions on what command to run to fix
(`chmod +x src/cli.ts && git add --chmod=+x src/cli.ts`) plus a pointer
back to issue #683 so future maintainers know why the guard exists.
Note: darwin and linux only. Windows preserves the git-stored mode
regardless of filesystem chmod, so the index-mode check works the same
on every platform CI uses.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(upgrade): detect bun-link, warn on npm squatter (#656, #658)
Rewrites detectInstallMethod() in src/commands/upgrade.ts:247 with three
layered signals per v0.28.5 plan cluster D + codex finding C1:
1. bun-link signal (closes#656): when argv[1] is a symlink, walk up
from realpath(argv[1]) up to 6 levels looking for a .git/config whose
contents include `garrytan/gbrain` (case-insensitive substring).
Returns 'bun-link'. Best-effort: forks, tarballs, and detached source
trees fall through to the existing chain.
2. canonical bun authenticity check (closes#658 detection half): when
the install lives in node_modules, read package.json and verify
repository.url contains `garrytan/gbrain` OR src/cli.ts coexists
(squatter ships compiled binary, not source). On 'suspect' verdict,
print printSquatterRecovery() — names both git-clone AND
release-binary recovery paths so users without a local clone can
still recover.
3. Source-marker fallback inside (2). Codex flagged this is spoofable
by a determined squatter; accepted — best-effort warning, not
assertion. The structural fix is publishing under @garrytan/gbrain
(tracked v0.29 follow-up).
The squatter's `name: gbrain` field doesn't disambiguate (codex caught
this in plan review of my original heuristic). repository.url is the
field a careless squatter is least likely to set correctly; src/cli.ts
presence is the secondary signal.
bun-link installs return 'bun-link' from the switch in runUpgrade, which
prints the source-clone upgrade path (`git pull && bun install && bun
link`) instead of trying `bun update gbrain` which doesn't apply.
README updated with the corresponding "DO NOT use `bun add -g gbrain`"
callout naming both #658 and the v0.29 scoped-name plan.
Tests in test/upgrade.test.ts cover return-type extension, bun-link
signal shape, classifyBunInstall's two-signal check, and the recovery
message contents.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.28.5 release: PGLite upgrade wedge + embedding dim corruption + bun-link foot-gun
Fix wave bundling 9 community PRs to unwedge users stuck since v0.27.
Cluster A — PGLite upgrade wedge (#670, #661, #657, #651, #625, #615, #609):
- Bootstrap now covers v0.20+v0.26.3+v0.27 forward references (both engines)
- hasPendingMigrations() probe gates initSchema() in connectEngine
- Post-upgrade auto-applies pending schema migrations (X1)
- SQL-parser-backed bootstrap coverage replaces hand-maintained array (A2)
Cluster B — Embedding dim corruption (#673, #672, #666, #640):
- Schema templating cascade fixed end-to-end (#641 from @100yenadmin)
- gbrain doctor 8b live embedding-provider probe (#665)
- Voyage adaptive batch sizing for 120K-token cap (#680)
- gbrain init A4 hard-error on existing-brain dim mismatch
- docs/embedding-migrations.md with conditional-HNSW four-step recipe
- #672 misleading migrate-suggestion error replaced with inline recipe
Cluster C — CLI exec bit (#683, dupe of #655):
- src/cli.ts mode 100644 → 100755 (#683 from @brandonlipman)
- scripts/check-cli-executable.sh CI guard against future regression
Cluster D — bun add -g foot-gun (#656, #658):
- 3-signal detectInstallMethod rewrite (bun-link, repo.url, source-marker)
- Loud-red recovery message names source-clone AND release-binary paths
- README "DO NOT use bun add -g gbrain" callout
Contributors: @brandonlipman (#682, #683), @mdcruz88 (#668), @ChenyqThu
(#627), @alan-mathison-enigma (#610), @oyi77 (#652 building block),
@abkrim (#655), @100yenadmin (#641).
VERSION 0.27.0 → 0.28.5
package.json 0.27.0 → 0.28.5
schema-embedded.ts regenerated via bun run build:schema
llms-full.txt regenerated via bun run build:llms
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(e2e): v0.28.5 fix-wave end-to-end coverage
PGLite-only E2E covering the three regression scenarios v0.28.5 was shipped
to fix:
1. cluster A — pre-v0.20 brain (missing v0.20 + v0.26.3 + v0.27 columns)
re-runs initSchema cleanly. Strips the column set v0.28.5's bootstrap
claims to restore (search_vector, parent_symbol_path, doc_comment,
symbol_name_qualified, agent_name, params, error_message, provider_id),
resets the version row to 13, then re-runs initSchema. Asserts every
column comes back AND version reaches LATEST_VERSION.
Closes the gap that pre-v0.28.5 produced 11 wedge incidents.
2. cluster B — fresh init at non-default dims templates the column
correctly (768d AND 2048d cases). The 2048d case explicitly verifies
idx_chunks_embedding is NOT created (codex finding #8 — pgvector's
HNSW cap is 2000).
3. A4 — existing-brain dim mismatch helper produces a recipe that inlines
all four steps (DROP INDEX, ALTER TYPE, NULL, conditional reindex).
Validates the conditional CREATE INDEX HNSW for dims <= 2000 AND its
omission for dims > 2000. The recipe a user copy-pastes won't crash
them on Voyage 4 Large.
Plus a hasPendingMigrations() lifecycle test covering the four states
(fresh / migrated / rewound / re-applied) — pairs with the unit test in
test/migrate.test.ts but exercises the engine end-to-end.
PGLite-only because none of these cases need real Postgres. Postgres-side
bootstrap is covered by test/e2e/postgres-bootstrap.test.ts.
Run: bun test test/e2e/v0_28_5-fix-wave.test.ts (no DATABASE_URL needed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test: refactor embedding-dim-check.test.ts to canonical PGLite pattern
Test-isolation lint (R3+R4) requires PGLiteEngine in beforeAll() context
with afterAll() disconnect. Refactored to single-engine-per-file pattern;
the fresh-brain test uses a one-off engine inside its own try/finally so
the file-level engine stays at LATEST schema for the migrated-brain test.
No behavior change to the assertions.
`bun run verify` now passes clean (privacy + jsonb + progress +
test-isolation + wasm + admin-build + cli-exec + typecheck).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(doctor): make 8b embedding-provider probe non-fatal (CI green)
CI Tier 1 was failing on `gbrain doctor exits 0 on healthy DB` because the
v0.28.5 doctor 8b check (cherry-picked from #665) pushed `status: 'fail'`
in two non-fatal scenarios:
1. No API key configured (`isAvailable('embedding')` returns false)
2. Probe throws (network blip, transient 5xx, DNS, rate limit)
Both are noise in CI and on offline workstations — the brain is healthy,
the provider just isn't reachable from this environment. The v0.28.5 plan
P1 decision called for non-fatal-on-offline behavior:
> Doctor 8b probes live every run (taken as-is). Non-fatal on network
> failure (warns rather than errors); silently skipped when no API key
> configured.
This commit aligns the implementation with that decision:
- !available → status 'ok' with "Skipped (no provider credentials)"
message so the run is visible in --json output without failing exit code
- catch block → status 'warn' (was 'fail') so probe failures surface
informationally without crashing CI / autopilot's periodic doctor runs
The mismatch slipped past plan-time review because #665 was cherry-picked
before P1 was finalized; the type-fix pass in 4c26e484 only adjusted the
DB-column probe shape, not the API-availability gate.
CI Tier 1 (Mechanical) — `test/e2e/mechanical.test.ts:1220` —
"gbrain doctor exits 0 on healthy DB" now passes against a fresh Postgres
without `OPENAI_API_KEY` / `VOYAGE_API_KEY` set.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Brandon Lipman <brandon@offdeck.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Eva <eva@100yen.org>
Co-authored-by: garrytan-agents <garrytan-agents@users.noreply.github.com>
