mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 21:19:18 +00:00
master
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
26c54588fe |
v0.38.0.0 ingestion cathedral — gbrain capture + write-through + IngestionSource contract (#1275)
* feat(ingestion): v0.38 substrate — daemon + IngestionSource contract + 2 sources
The foundation for the ingestion cathedral (CEO+DX+Eng plan-reviewed).
Plan: ~/.claude/plans/system-instruction-you-are-working-ethereal-riddle.md
WHAT YOU CAN NOW DO
The IngestionSource public contract is locked. Skillpack publishers can
build third-party ingestion sources (Granola, Linear, Mail, voice, OCR,
etc.) and ship them through the v0.37 skillpack registry. The locked
surface lives at the new package subpaths:
import { IngestionSource, IngestionEvent } from 'gbrain/ingestion';
import { IngestionTestHarness, expectEvent } from 'gbrain/ingestion/test-harness';
Both subpaths are pinned by test/public-exports.test.ts — breaking either
is a major-version change.
WHAT THIS COMMIT BUILDS
Foundation:
- src/core/ingestion/types.ts (IngestionSource, IngestionEvent,
IngestionSourceContext, validateIngestionEvent, computeContentHash,
INGESTION_SOURCE_API_VERSION, INGESTION_CONTENT_TYPES)
- src/core/ingestion/dedup.ts (24h content-hash LRU, 5000-entry cap)
- src/core/ingestion/skillpack-load.ts (gbrain.plugin.json discovery for
third-party sources, api_version compat with paste-ready upgrade hints,
in-process trust model for v1)
- src/core/ingestion/daemon.ts (IngestionDaemon: in-process source
supervision sibling to v0.34.3.0 ChildWorkerSupervisor pattern, plus
validate -> dedup -> rate-limit -> dispatch pipeline + health surface)
- src/core/ingestion/test-harness.ts (publisher-facing test utility with
fake clock + in-memory event bus + expectEvent matchers + engine proxy
that throws on access so publishers know what they're depending on)
- src/core/ingestion/index.ts (barrel for gbrain/ingestion subpath)
First two built-in sources prove the abstraction:
- file-watcher (chokidar over the brain repo; 1s debounce; honors
pruneDir from src/core/sync.ts; symlinks rejected; Linux ENOSPC
surfaces a paste-ready sysctl hint at runtime)
- inbox-folder (~/.gbrain/inbox/ target for iOS Shortcuts / AirDrop /
Drafts; auto-archives processed files into .archived/YYYY-MM-DD/;
symlink rejection; world-writable dir warning; routes content-type by
extension)
Public exports surface (count 18 -> 20) pinned in:
- package.json exports map
- test/public-exports.test.ts EXPECTED_EXPORTS + count gate
- scripts/check-exports-count.sh baseline
ARCHITECTURE-LOCKED DECISIONS (from /plan-eng-review)
E1 webhook source process boundary: webhook source will live INSIDE
serve --http (NOT this daemon) when it lands in the next commit. Daemon
supervises only daemon-side sources.
E2 content-type processor execution: hybrid by size (inline <1MB,
Minion handlers >1MB). Processors land in a later commit.
E3 publisher TTHW: chokidar v4.0.3 across platforms; ephemeral PGLite
persistence and Linux inotify-limit doctor probe land in later commits.
E4 migration v80 (provenance columns) + forward-reference bootstrap:
lands with put_page write-through in a later commit.
DX-locked decisions (from /plan-devex-review):
- Source error semantics: throws bubble to daemon; supervisor backoff.
- IngestionTestHarness exported as gbrain/ingestion/test-harness.
- api_version field on gbrain.plugin.json with loud-fail on mismatch.
TESTS
192 cases across 8 test files, 0 failures:
- test/ingestion/types.test.ts (28 cases pinning the contract)
- test/ingestion/dedup.test.ts (15 cases for LRU + TTL + collision)
- test/ingestion/skillpack-load.test.ts (22 cases for manifest
validation + api_version compat + collision policy + module load)
- test/ingestion/test-harness.test.ts (24 cases for harness lifecycle +
clock + healthCheck + every expectEvent matcher)
- test/ingestion/daemon.test.ts (19 cases for supervision + dispatch
pipeline + health surface + per-source config + logger wrapping)
- test/ingestion/sources/file-watcher.test.ts (10 cases including
ENOSPC sysctl-hint surfacing)
- test/ingestion/sources/inbox-folder.test.ts (24 cases including
symlink rejection + world-writable warning + archive-loop-prevention)
- test/public-exports.test.ts (2 new cases for the new subpaths)
typecheck clean. bun run verify gate passes.
NEXT IN WAVE
Subsequent commits in this PR ship webhook source (serve --http route),
cron-scheduler refactor + OpenClaw credential auto-migrate, content-type
processors (PDF + image OCR + audio transcribe + video keyframe), put_page
write-through with serializePageToMarkdown DRY extract, migration v80
+ bootstrap probes, gbrain capture verb, publisher DX cathedral (init
scaffold extension + gbrain ingest test [--watch] + tail + validate),
daemon rename autopilot -> ingest with forever-alias, doctor inotify
probe on Linux, skillpack contract docs + reference pack.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(ingestion): webhook source — POST /ingest + ingest_capture Minion handler
Lands the v0.38 ingestion cathedral's webhook source. Per the
/plan-eng-review E1 decision, the webhook source lives INSIDE
`serve --http` (NOT the ingestion daemon) so there is no new IPC: the
HTTP route submits Minion jobs directly into the existing queue, and
the daemon supervises only daemon-side sources.
WHAT YOU CAN NOW DO
With `gbrain serve --http` running and an OAuth client minted, any
HTTP caller (Zapier, IFTTT, n8n, Make, Apple Shortcuts) can POST a
captured thought into the brain:
curl -X POST https://your-brain.example.com/ingest \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: text/markdown" \
-d "# captured from my Shortcut"
The route auths via OAuth (write scope required), validates the
content-type, enforces a 1MB payload cap and per-IP rate limit
(100 events / 10s), submits an `ingest_capture` Minion job tagged
`untrusted_payload: true`, and returns 202 Accepted with the job id.
The job materializes the page under `inbox/YYYY-MM-DD-<hash6>` by
default (overridable via X-Gbrain-Slug header) so the user has a
predictable triage location.
WHAT THIS COMMIT BUILDS
- src/core/minions/handlers/ingest-capture.ts (new) — handler that
takes an IngestionEvent payload, resolves a slug via fallback chain
(job.data.slug -> event.metadata.slug -> inbox/<date>-<hash6>),
validates the event at the handler boundary, REJECTS binary
content_types with a paste-ready hint to install a processor
skillpack, and routes through importFromContent. Defaults
noEmbed: true (embed is a separate Minion job, matching the sync
handler's pattern).
- src/commands/jobs.ts — registers `ingest_capture` in
registerBuiltinHandlers alongside sync/embed/extract.
- src/commands/serve-http.ts — POST /ingest route with:
- OAuth write-scope gate via requireBearerAuth({requiredScopes:['write']})
- 100 events / 10s rate limiter (sibling to ccRateLimiter)
- Content-type allowlist: text/markdown, text/plain, text/html,
application/json; binary REJECTED with HTTP 415
- 1 MB payload cap (configurable via GBRAIN_INGEST_MAX_BYTES)
- Caller-overridable source identity via X-Gbrain-Source-Id /
X-Gbrain-Source-Uri / X-Gbrain-Content-Type / X-Gbrain-Slug
headers — useful for downstream tools that want clean provenance
- untrusted_payload: true ALWAYS (network input)
- Idempotency on (client_id, content_hash) so simultaneous retries
collapse to one job
- maxWaiting: 50 per client so a runaway integration can't
monopolize the queue
- Audit row in mcp_request_log + SSE broadcast for the admin feed
TESTS
test/ingestion/ingest-capture.test.ts (15 cases against PGLite):
- defaultSlugForEvent helper (3 cases pinning shape + UTC + determinism)
- slug resolution fallback chain (3 cases)
- validation + content-type routing (5 cases including binary rejection
+ untrusted_payload round-trip)
- importFromContent integration (3 cases including content_hash dedup
via status='skipped' on repeat)
207 total ingestion tests passing. typecheck clean.
NEXT IN WAVE
cron-scheduler refactor + OpenClaw credential auto-migrate; content-type
processors (PDF + image OCR + audio transcribe + video keyframe);
put_page write-through + serializePageToMarkdown DRY extract +
migration v80 + bootstrap probes; gbrain capture verb; publisher DX
cathedral (init scaffold + gbrain ingest test --watch + tail + validate);
daemon rename autopilot -> ingest with forever-alias; doctor inotify
probe; skillpack contract docs + reference pack + VERSION bump.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(ingestion): put_page write-through + migration v80 + DRY extract
WHAT YOU CAN NOW DO
The drift class is dead. Every `gbrain put_page` (CLI or MCP, local
or remote) now lands its markdown file on disk alongside the DB row
whenever `sync.repo_path` is configured. The page is queryable
immediately AND visible to git, your editor, and downstream tools.
Pre-v0.38, put_page wrote ONLY to the DB and synthesize/extract paths
had to reverse-render later. The v0.35.6.0 phantom-redirect pass was
the cleanup for what THIS commit prevents in the first place.
# local CLI
gbrain put inbox/test < my-thought.md
# file lands at ${sync.repo_path}/inbox/test.md AND in the DB
# MCP remote (Zapier / Cursor / Claude Desktop)
curl -X POST /mcp ... '{"method":"tools/call","params":{"name":"put_page",...}}'
# server-side write-through fires, agent gets a normal success response
# untrusted_payload tagging applied (no auto-link, slug-allowlist gate)
Provenance frontmatter stamped on every write so future sync round-trips
know where the page came from:
ingested_via: put_page # local CLI
ingested_via: 'mcp:put_page' # MCP remote
ingested_at: 2026-05-21T04:...
WHAT THIS COMMIT BUILDS
1. Migration v80 — `pages_provenance_columns` adds four nullable
columns to `pages`: `ingested_via`, `ingested_at`, `source_uri`,
`source_kind`. ADD COLUMN with no DEFAULT is metadata-only on
Postgres 11+ and PGLite 17.5; instant on tables of any size. The
four columns get NULL on every historical page (pre-v0.38 pages
never had provenance).
2. DRY extract — `serializePageToMarkdown(page, tags, opts)` and
`resolvePageFilePath(brainDir, slug, sourceId)` in `src/core/markdown.ts`.
The dream-cycle's `renderPageToMarkdown` (synthesize.ts) and the new
put_page write-through path were going to have 90% duplicate bodies.
They now share one foundation; the dream version is a 4-line wrapper
that passes `frontmatterOverrides: {dream_generated: true, ...}`.
Future markdown-shape changes happen in one place.
3. put_page write-through (`src/core/operations.ts`) — after
importFromContent succeeds, resolves sync.repo_path, computes the
v0.32.8 source-aware path layout (default: brainDir/<slug>.md;
non-default: brainDir/.sources/<id>/<slug>.md), serializes the
freshly-written Page via `serializePageToMarkdown`, writes the file.
Returns a `write_through: {written, path}` field in the put_page
response so callers can see what happened.
Trust gating:
- subagent sandbox (viaSubagent without allowedSlugPrefixes) → DB-only
- dry-run → DB-only (handler's early-return short-circuits before
write-through; documented via the dry_run response field)
- no sync.repo_path configured → DB-only, skipped reason returned
- sync.repo_path points at a non-existent dir → DB-only, skipped
- all other writes → write-through
Failure isolation: disk-write failures are LOGGED loud but do NOT
roll back the DB write. DB is the durable record; the
phantom-redirect pass exists for drift cleanup if it ever shows up.
TESTS
- test/ingestion/put-page-write-through.test.ts (10 cases against PGLite):
happy path (file land, provenance stamp local + remote), trust gating
(subagent sandbox, dry-run, trusted-workspace), config edges (no
repo_path, missing dir), multi-source filing (.sources/<id>/),
failure isolation (DB write survives a disk failure).
- Migration v80 verified across both engines via the existing
test/migrate.test.ts + test/bootstrap.test.ts coverage (~125 cases).
369 total tests passing in the ingestion + markdown + migrate bundle.
typecheck clean.
NOTES
- Bootstrap probes for the v80 provenance columns are NOT yet added
to applyForwardReferenceBootstrap on either engine. This is safe
for v0.38 because no SCHEMA_SQL CREATE INDEX or FK references the
new columns — migration v80 is the only consumer, and it runs
AFTER SCHEMA_SQL replay. A future commit may add bootstrap probes
+ REQUIRED_BOOTSTRAP_COVERAGE entries as defense-in-depth (eng
review E4).
- The trusted-workspace path (dream cycle's reverseWriteRefs in
synthesize.ts) still runs its own write at synthesize phase time.
Both paths writing the same file is idempotent (byte-identical
serialization), but a future commit may simplify reverseWriteRefs
to skip pages whose file already matches.
NEXT IN WAVE
gbrain capture verb (the single human-facing entrypoint); daemon
rename autopilot -> ingest with forever-alias + plist migration;
doctor inotify probe (Linux); content-type processor router
(PDF + image OCR + audio transcribe stubs); cron-scheduler refactor
+ OpenClaw credential auto-migrate; skillpack contract docs +
reference pack; VERSION bump.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(ingestion): gbrain capture — the single human-facing entrypoint
WHAT YOU CAN NOW DO
One command, local or thin-client, synchronous receipt with the resulting
page slug. The answer to "what is the best way to get data into the brain?"
is now: just type `gbrain capture` and the right thing happens.
# the basic case
gbrain capture "remember to follow up on the X deal"
# from a file
gbrain capture --file ./notes/today.md --slug daily/2026-05-21
# from a pipe (shell pipelines)
echo "from stdin" | gbrain capture --stdin
# script-friendly: print just the slug
SLUG=$(gbrain capture "a thought" --quiet)
# JSON for agents
gbrain capture "..." --json
Default slug is `inbox/YYYY-MM-DD-<hash8>` — deterministic for the same
content so re-running idempotently lands the same page. Receipt block
on stdout shows slug + status + content_hash + on-disk path so you
can confirm where the page went without rerunning `gbrain query`.
The local-install path routes through the put_page operation with the
v0.38 write-through plumbing landed in the prior commit, so the page
hits both the DB AND the file tree in one move. Thin-client installs
route through `callRemoteTool('put_page', ...)` so the server's
write-through handles disk persistence the same way.
WHAT THIS COMMIT BUILDS
- src/commands/capture.ts (new ~290 LOC):
- `defaultSlug(content)` — UTC-stable `inbox/YYYY-MM-DD-<hash8>`
- `parseArgs(args)` — positional + flag parsing with --file / --stdin
/ --slug / --type / --source / --quiet / --json / --help
- `buildContent(rawBody, opts)` — wraps unstructured prose in
frontmatter (type + title + captured_via + captured_at) and a
leading `# Title` heading; passes through if the body already
looks like markdown
- `runCapture(engine, args)` — local install routes through the
in-process put_page operation; thin-client routes through MCP.
`--quiet` prints just the slug; `--json` prints structured output;
default prints a 5-line receipt block.
- src/cli.ts:
- Adds `case 'capture'` dispatch
- Adds `'capture'` to the CLI_ONLY set so cli.ts wires it correctly
TESTS
test/commands/capture.test.ts (21 cases against PGLite):
- defaultSlug helper: shape + determinism + UTC math
- parseArgs: positional + multi-token join + every flag
- buildContent: prose wrapping, --type override, no double-wrap
for pre-frontmattered content, title cap at 80 chars,
--source provenance stamp
- Integration: inline content lands in DB + on disk, default slug
shape, --file reads from disk, --json structured output,
--help returns without engine roundtrip
271 total tests passing in the bundle. typecheck clean.
NOTES
- Thin-client routing relies on `callRemoteTool('put_page', ...)` from
src/core/mcp-client.ts. Identical UX to the local path because the
server's put_page handler runs the same write-through plumbing.
- buildContent's "looks like markdown" heuristic is intentionally
simple — first-line heading or frontmatter delimiter is the trigger.
Users who care about exact formatting pass a pre-formatted --file.
NEXT IN WAVE
Daemon rename autopilot -> ingest with forever-alias + plist migration;
doctor inotify probe (Linux); content-type processor router
(PDF + image OCR + audio transcribe stubs); cron-scheduler refactor
+ OpenClaw credential auto-migrate; skillpack contract docs +
reference pack; VERSION bump.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(ingestion): v0.38.0.0 release hygiene + e2e roundtrip + capture skill
VERSION 0.37.1.0 → 0.38.0.0 (trio: VERSION, package.json, CHANGELOG header).
CHANGELOG entry written in user-facing ELI10-lead voice per CLAUDE.md
release-summary rules. README's pre-loop section gains a new "How to get
data in (v0.38+)" block leading with `gbrain capture`.
skills/capture/SKILL.md (NEW) so agents route "capture this" / "save this
thought" / "remember this" / "drop this in the inbox" / "save to brain" to
the capture verb. RESOLVER.md updated with the new triggers (sits above
idea-ingest/media-ingest/meeting-ingestion in the content-ingestion
section as the "simple thought" path).
E2E roundtrip test (test/e2e/ingestion-roundtrip.test.ts) covers the gap:
inbox-folder source -> daemon -> ingest_capture handler -> DB page,
including:
- Full pipeline: file drop appears as page in DB + file moves to .archived/
- Dedup catches byte-identical content from a different filename
- Multi-source coordination: two distinct inbox dirs, two sources, daemon
ingests both events independently
The test runs against an in-memory PGLite (no DATABASE_URL needed) so it
exercises the substrate-level wiring in the standard test suite. A
follow-up commit can add a full-process e2e (gbrain serve --http + real
OAuth client + POST /ingest) that requires DATABASE_URL.
399/399 v0.38 wave tests passing (910 assertions). typecheck clean.
bun run verify gate green across all 14 shell checks.
DEFERRED TO FOLLOW-UP RELEASES (called out in CHANGELOG)
- Daemon rename autopilot -> ingest + forever-alias + plist migration
- cron-scheduler skill refactor + OpenClaw credential auto-migrate
- Content-type processors (PDF / OCR / audio / video)
- gbrain doctor inotify probe (Linux)
- Publisher DX cathedral: gbrain skillpack init --kind=ingestion-source,
gbrain ingest test --watch, ingest tail, ingest validate
- Reference pack at examples/skillpack-ingestion-reference/ + 3-stage
tutorial in docs/ingestion-source-skillpack.md
These are polish items; the substrate is shipped and queryable, and
skillpack publishers can build sources against the IngestionTestHarness
public export today.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ingestion): test-gap fills — bootstrap probes + manifest entry + conformance
The v0.38.0.0 release-hygiene commit landed cleanly against the v0.38 wave
suite but tripped 3 categories of full-suite tests. This commit fixes
each. The remaining failure (doctorReportRemote > "healthy" status) was
verified pre-existing via `git stash + bun test` and is not caused by
v0.38; left alone.
Fix 1 — `schema-bootstrap-coverage.test.ts` (s1)
The test parses MIGRATIONS for ALTER TABLE ADD COLUMN statements and
fails if any column is not covered by `applyForwardReferenceBootstrap`
on both engines. Migration v80's four provenance columns triggered
the failure. Bootstrap probes added to both engines + 4 entries
appended to REQUIRED_BOOTSTRAP_COVERAGE:
- src/core/pglite-engine.ts — 4 EXISTS probes + state field + needs
flag + ALTER TABLE block when bootstrap fires
- src/core/postgres-engine.ts — same pattern
- test/schema-bootstrap-coverage.test.ts — 4 coverage entries
Fix 2 — `check-resolvable.test.ts` (s3 — orphan_trigger)
RESOLVER.md references skills via name; check-resolvable cross-checks
against skills/manifest.json. The new `capture` skill was missing the
manifest entry; added between `brain-ops` and `idea-ingest` so the
manifest order mirrors the resolver order.
Fix 3 — `skills-conformance.test.ts` (s8)
Every SKILL.md must have `## Contract`, `## Output Format`, and
`## Anti-Patterns` sections. skills/capture/SKILL.md was missing all
three (initial draft skipped them); now compliant with concrete
content per the v0.38 contract.
Fix 4 — `build-llms.test.ts` (s6)
README + CHANGELOG edits in the release-hygiene commit caused
llms-full.txt to drift behind. Regenerated via `bun run build:llms`.
Per CLAUDE.md: any user-facing docs edit MUST run build:llms before
push.
The full bun-test parallel runner now passes everywhere except the
pre-existing `doctorReportRemote > healthy status` failure (50/100
score on an empty fresh brain — this is a pre-v0.38 health-score
tuning issue and orthogonal to ingestion work).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(version): bump 0.38.0.0 → 0.38.1.0
Renumbers the in-flight ingestion-cathedral release to v0.38.1.0.
Trio (VERSION, package.json, CHANGELOG.md) bumped together.
bun run typecheck → clean.
* chore(version): bump 0.38.1.0 → 0.38.0.0
Master sits at 0.37.11.0; 0.38.0.0 is the natural next slot rather than
skipping a release. Trio (VERSION, package.json, CHANGELOG.md) bumped
together. Migration v81 + ingestion substrate stay identical — this is a
header-only renumber.
bun run typecheck → clean.
* test(ingestion): fill v0.38 test gaps — markdown helpers + migration v81 + webhook E2E
Three gaps surfaced from a v0.38 audit against what shipped vs what was
covered. All three filled:
1. **test/markdown-serializer.test.ts** (NEW, 19 cases) — pure-function
coverage of `serializePageToMarkdown` + `resolvePageFilePath`, the
DRY extract that the dream-cycle reverse-render and put_page
write-through both consume. Pre-fix nothing pinned the
frontmatter-override merge precedence, the type/title defaults, or
the source-aware filing layout (default → `<brainDir>/<slug>.md`,
non-default → `<brainDir>/.sources/<source_id>/<slug>.md`). Future
schema-shape changes to either helper now surface immediately.
2. **test/migrate.test.ts — v81 cases** (10 new cases, two describe
blocks) — structural assertions on `pages_provenance_columns`
(four nullable columns, no NOT NULL, no DEFAULT, no index — the
ADD COLUMN stays metadata-only) plus a PGLite round-trip that
asserts the columns appear post-`initSchema`, accept direct UPDATEs,
and survive the historical-page NULL scenario. The
schema-bootstrap-coverage test already pinned the forward-reference
probe contract; this fills the migrate.test.ts contract gap.
3. **test/e2e/serve-http-ingest-webhook.test.ts** (NEW, 16 cases) — HTTP
contract coverage for POST /ingest. The pre-existing
ingestion-roundtrip E2E explicitly notes "e2e (gbrain serve --http +
POST /ingest + real OAuth) is a separate" thing — it covers the
in-process daemon → handler → DB pipeline, NOT the real HTTP route.
This file fills that gap. Spawns real gbrain serve --http against
real Postgres, mints OAuth tokens with various scopes, exercises:
- Auth gate (missing → 401; read-only → 403)
- Body validation (empty → 400 with error: empty_body)
- Content-type allowlist (image/png → 415 with skillpack hint;
application/pdf → 415; text/plain + application/json + text/html
all accepted; unknown text/* falls through to text/plain)
- X-Gbrain-Content-Type / Source-Id / Source-Uri / Slug header
overrides
- Idempotency (same content + same client = identical job_id via
queue dedup on content_hash)
Also wires three new entries into `scripts/e2e-test-map.ts` so changes
to `src/commands/serve-http.ts`, `src/core/ingestion/**`, or the
`ingest-capture` Minion handler auto-trigger the relevant E2Es under
`bun run ci:local:diff`.
Verified locally:
- bun test test/markdown-serializer.test.ts → 19/19 green
- bun test test/migrate.test.ts -t "v81" → 10/10 green
- bun test test/e2e/serve-http-ingest-webhook.test.ts (real Postgres on
ephemeral 5435) → 16/16 green
- bun test test/select-e2e.test.ts → 24/24 green (selector test still
honors the v0.38 entries)
- bun run typecheck → clean
E2E DB lifecycle handled per CLAUDE.md (spin up pgvector:pg16 on a free
port, bootstrap via `gbrain doctor --json`, run, tear down).
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
|
3933eb6a79 |
v0.35.1.0: embedder shootout prereqs (pricing + gateway export + --resume-from) (#1055)
* docs(designs): 2026-05 embedder shootout eval plan Adds docs/designs/2026_05_EVAL_PLAN.md — the approved plan + 6 Conductor session briefs for the OpenAI vs Voyage vs ZeroEntropy embedder comparison. Why: produce a publishable comparison report for v0.35.x release notes pinning "which embedder wins, and does zerank-2 carry the win for ZeroEntropy" against public LongMemEval + in-house BrainBench. Each session brief is self-contained — repo, branch, commits, verify, ship, deliverable, hand-off. Stewardable one section per Conductor session. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(pricing): add voyage-4-large + zembed-1 to EMBEDDING_PRICING v0.35.0.0 shipped ZeroEntropy zembed-1 + zerank-2 reranker support and expanded the Voyage allow-list to include voyage-4-large. The pricing table missed both, so `gbrain upgrade`'s post-upgrade reembed prompt silently fell back to "estimate unavailable" for users on these models. - voyage:voyage-4-large @ $0.18/MTok (same as voyage-3-large) - zeroentropyai:zembed-1 @ $0.05/MTok New test file pins both entries plus the openai/voyage-3-large baselines, case-insensitive provider matching, bare-model openai-default fallback, table integrity (lowercase providers, finite non-negative prices), and the estimateCostFromChars approximation. 11 cases, 46 expect() calls. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(exports): expose gbrain/ai/gateway with canary test Adds ./ai/gateway to the package.json exports map so external eval consumers (notably gbrain-evals, the sibling repo running the embedder shootout in docs/designs/2026_05_EVAL_PLAN.md) can call configureGateway directly to swap embedding providers per cell. Why: pre-v0.35.1.0, gbrain-evals adapters hardcoded gbrain/embedding, which means every retrieval adapter was OpenAI-only. The newly-exposed gateway lets adapters route through Voyage and ZeroEntropy without forking gbrain or duplicating the recipe wiring. - package.json: add "./ai/gateway" -> "./src/core/ai/gateway.ts" - scripts/check-exports-count.sh: bump expected count 17 -> 18 - test/public-exports.test.ts: add canary pinning configureGateway + embed, bump expected count assertion Pre-existing import-resolution failures in this test file (16 on master) are unrelated to this change — they're a longstanding Bun package self-import behavior. The count + EXPECTED_EXPORTS list-match assertions both pass cleanly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(eval): add --resume-from <jsonl> to gbrain eval longmemeval Multi-cell embedder shootouts spend $50+/cell on the gpt-4o judge after gbrain emits hypotheses. A mid-run abort (rate-limit, cost-cap, OS interrupt, SIGKILL) previously meant re-paying the full cell. This flag makes those aborts cheap: re-invoke with --resume-from pointed at the partial JSONL and only the unanswered question_ids re-run. Behavior: - Read question_ids from the file; skip them on this run. - Rows with non-empty hypothesis count as done. - Rows with hypothesis="" AND an error field are NOT skipped (retry case for per-question failures recorded by the existing try/catch). - Corrupt trailing lines (SIGKILL'd writer mid-line) are silently skipped with a stderr warn. - When --resume-from path == --output path, the output emitter opens the file in append mode instead of truncating, so the existing rows survive. - Empty resume case (all questions already done) returns immediately without spinning up the brain or calling the client. New exported helper loadResumeSet() makes the parser unit-testable. 6 new test cases pinning: - File-not-found returns empty set - Well-formed JSONL load - Error-row retry semantics (empty hypothesis + error -> not in set) - Truncated final line recovery - End-to-end resume against the 5-question mini fixture - All-done early-return (stub client must NOT be invoked) All 18 cases in test/eval-longmemeval.test.ts green; bun run typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: v0.35.1.0 Bumps VERSION + package.json + CHANGELOG entry for the embedder-shootout prereq release. Three additive changes from the prior 4 commits: - pricing: voyage-4-large + zembed-1 entries - exports: gbrain/ai/gateway is now public - eval: gbrain eval longmemeval --resume-from <jsonl> Each commit on this branch is independently bisect-friendly and CI-green; the CHANGELOG entry is the user-facing rollup. No migrations, no breaking changes — the gateway export expands the surface, the resume-from flag is additive, the pricing patch only changes "estimate unavailable" -> a real dollar figure for two specific models. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
736e8de1ec |
v0.25.0 feat: BrainBench-Real session capture + public-exports contract test (#437)
* feat(v0.22.0): eval_candidates + eval_capture_failures schema (Lane 1A)
R1 substrate for BrainBench-Real, replayed onto master after Cathedral II
landed. Migration v30 (slotted after master's v25-v29 Cathedral II wave)
creates two tables:
eval_candidates: per-call capture of MCP/CLI/subagent query+search
traffic. Column set lets gbrain-evals replay with full fidelity —
source_ids from v0.18 multi-source, vector_enabled/detail_resolved/
expansion_applied so replay knows what hybridSearch actually did,
remote + job_id + subagent_id so rows are traceable to their origin.
query is CHECK-capped at 50KB; PII scrubber (Lane 1B) runs before insert.
eval_capture_failures: cross-process audit trail. In-process counters
don't work because `gbrain doctor` runs in a separate process from
the MCP server. Persistent rows let doctor query capture health via
COUNT(*) GROUP BY reason over the last 24h.
Both tables get RLS on Postgres gated on BYPASSRLS (matches v24/v29
posture). PGLite ignores RLS; sqlFor split carries only DDL.
5 new BrainEngine methods (breaking-interface addition, drives v0.22.0
minor bump): logEvalCandidate, listEvalCandidates,
deleteEvalCandidatesBefore, logEvalCaptureFailure, listEvalCaptureFailures.
listEvalCandidates uses ORDER BY created_at DESC, id DESC so
`gbrain eval export` is deterministic across same-millisecond inserts.
Also adds HybridSearchMeta type for the side-channel callback used by
Lane 1C's op-layer capture (no change to hybridSearch return shape —
that respects Cathedral II's existing SearchResult[] contract).
Tests: 14 PGLite round-trip cases + 8 v30 structural assertions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(v0.22.0): PII scrubber + op-layer capture module (Lane 1B)
Replayed onto master post-Cathedral II. Same semantics as the original
v0.21.0 work — only adjusted to import HybridSearchMeta from types.ts
(canonical home) instead of redeclaring it locally.
src/core/eval-capture-scrub.ts — pure-function regex scrubber with 6
pattern families: emails, phones (US + E.164), SSN (year-aware),
Luhn-verified credit cards, JWT-shaped tokens, bearer tokens. Zero
deps. Adversarial-input safe.
src/core/eval-capture.ts — op-layer hook helper:
- buildEvalCandidateInput(ctx, {scrub_pii}) — pure row builder
- classifyCaptureFailure(err) — Postgres SQLSTATE → reason tag
- captureEvalCandidate(engine, ctx, opts) — best-effort, never throws
- isEvalCaptureEnabled / isEvalScrubEnabled — file-plane config checks
GBrainConfig gains `eval?: {capture?, scrub_pii?}`. Both default ON.
File-plane only — `gbrain config set` writes the DB plane, doesn't
control capture.
Tests: 17 scrubber + 21 capture-module cases. Zero regressions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(v0.22.0): hybridSearch onMeta callback + op-layer capture (Lane 1C)
Replayed onto master. Adapted from the original v0.21.0 work to keep
Cathedral II's contract intact: hybridSearch's return stays
`Promise<SearchResult[]>` (unchanged), and meta surfaces via an optional
`onMeta?: (meta: HybridSearchMeta) => void` callback in HybridSearchOpts.
Cathedral II callers leave onMeta undefined and pay no cost. The
op-layer capture wrapper passes a closure that threads meta into the
captured row so gbrain-evals can distinguish:
- "with OPENAI_API_KEY" vs "keyword-only fallback" (vector_enabled)
- "expansion fired" vs "expansion requested + silently fell back" (expansion_applied)
- what hybridSearch actually used after auto-detect (detail_resolved)
Op-layer capture wired into both `query` and `search` op handlers in
src/core/operations.ts. Single hook site catches MCP dispatch + CLI +
subagent tool-bridge from the same place. Fire-and-forget, never throws,
respects ctx.config.eval.capture off-switch.
Tests:
- test/hybrid-meta.test.ts (8 cases) — onMeta accuracy across the 4
return paths in hybridSearch + verification that omitting onMeta
leaves Cathedral II callers unchanged.
- test/mcp-eval-capture.test.ts (10 cases) — query/search ops capture
correctly with MCP/CLI/subagent contexts, scrub on/off, capture=false
off-switch, non-captured ops (list_pages, get_page), F1 failure
isolation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(v0.22.0): gbrain eval export/prune + doctor eval_capture check (Lane 1D)
Replayed onto master. Same semantics as the original v0.21.0 work.
CLI:
gbrain eval export [--since DUR] [--limit N] [--tool query|search]
NDJSON to stdout, every row prefixed with "schema_version":1 per
docs/eval-capture.md contract. EPIPE-safe streaming, stderr
heartbeats, deterministic ordering (created_at DESC, id DESC).
gbrain eval prune --older-than DUR [--dry-run]
Explicit retention cleanup. Requires --older-than (never deletes
without a window). Duration strings: 30d, 7d, 1h, 90m, 3600s.
Legacy bare `gbrain eval --qrels …` still works via sub-subcommand
fall-through.
gbrain doctor gains an eval_capture check between markdown_body_completeness
and queue_health: reads eval_capture_failures for the last 24h, groups by
reason, warns when non-zero. Pre-v30 brains get "Skipped (table
unavailable)" — non-fatal.
docs/eval-capture.md ships the stable NDJSON schema reference for
gbrain-evals consumers.
Tests: 9 export cases + 5 prune cases. Doctor check covered by
existing doctor tests on master.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(v0.22.0): public-exports contract test + CI count guard (Lane 2 / R2)
Master locks 17 public subpath exports as gbrain's stable third-party
contract. Zero enforcement existed. This PR locks the surface in two
layers:
1. test/public-exports.test.ts — runtime contract test.
Reads package.json "exports" at startup. For each subpath, imports
via the package name ("gbrain/engine"), NOT the relative filesystem
path — that's the difference between exercising the actual resolver
and bypassing it. Every subpath gets a canary symbol pinned (e.g.
gbrain/search/hybrid must export hybridSearch + rrfFusion) so a
refactor that renames or removes one fails CI before downstream
consumers (gbrain-evals) silently break.
2. scripts/check-exports-count.sh — CI structural guard.
Wired into `bun test` after check-jsonb-pattern.sh +
check-progress-to-stdout.sh + check-wasm-embedded.sh per master's
precedent. EXPECTED_COUNT=17 baseline — shrinks fail loudly,
growth also fails so the new canary must be pinned in the runtime
test deliberately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs+e2e(v0.22.0): VERSION/CHANGELOG/CLAUDE/README + Postgres E2E (Lane 3)
Bump VERSION + package.json to 0.22.0 (next free slot after master's
v0.21.0 Code Cathedral II minor).
CHANGELOG.md v0.22.0 entry follows the Garry voice template:
- Bold 2-line headline
- Lead paragraph contextualizing v0.20 + v0.21 + v0.22 progression
- Numbers-that-matter table comparing v0.21.0 → v0.22.0
- "What this means for you" sectioned by audience
- "## To take advantage of v0.22.0" operator runbook
- Itemized changes
CLAUDE.md updates:
- Key files: 8 new module entries (eval-capture*, eval-export,
eval-prune, docs/eval-capture.md, public-exports test).
hybrid.ts entry rewritten to reflect the additive `onMeta` callback
(return shape unchanged).
- Key commands: new v0.22.0 section for `gbrain eval export`,
`gbrain eval prune`, and the doctor `eval_capture` check, with the
file-plane vs DB-plane config gotcha called out.
README.md: one-paragraph pointer after the BrainBench blurb so anyone
reading the landing page sees the new session-capture feature.
llms.txt + llms-full.txt regenerated to pick up the doc additions.
test/e2e/eval-capture.test.ts (Postgres-only E1 spec):
- CHECK violation surfaces as Postgres SQLSTATE 23514 on oversize input
- RLS is actually enabled on both eval_candidates + eval_capture_failures
- 50 concurrent logEvalCandidate calls — no deadlock, all distinct IDs
Skips gracefully when DATABASE_URL is unset.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(todos): P0 — PGLite test-runner concurrency flake
Pre-existing on master, surfaces ~27 false failures when bun test runs all
174 files together. Each failing file passes in isolation. Tracked for a
dedicated investigation branch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(v0.22.0): adversarial review post-fixes (doctor RLS, onMeta safety)
Two surgical fixes from /ship adversarial review, plus 6 follow-ups TODO'd
into v0.22.1:
- doctor.ts: distinguish pre-v30 missing-table (42P01, ok skip) from
RLS-denied SELECT (42501, warn) and other DB errors (warn). The check
exists specifically to surface capture-failure misconfigs cross-process,
so silently reporting "ok / skipped" on the most diagnostic class
defeated the purpose.
- hybrid.ts: wrap onMeta invocation in try/catch via small emitMeta
helper. The callback is part of the public gbrain/search/hybrid
contract; a throwing user-supplied closure must never break the search
hot path.
- TODOS.md: 6 P1 follow-ups (eval prune real COUNT, scrubber CC false
positives, dead 'scrubber_exception' enum value, id-cursor for
cross-window dedup, public-export canary pinning, EXPECTED_COUNT dedup).
- TODOS.md: P0 entry for the pre-existing PGLite test-runner concurrency
flake (~27 false failures in full bun test on master).
- CHANGELOG.md: 2 bullets noting the doctor + onMeta hardening.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(version): bump v0.22.0 → v0.25.0 (queue-aware version pick)
Master is at v0.21.0. Open PRs claim v0.21.1 (#432) and v0.24.0 (#387).
v0.25 is the first uncontested slot, so this branch claims it. Pure
rename across VERSION, package.json, CHANGELOG header, and every "v0.22.0"
reference in CLAUDE.md / README.md / TODOS.md / docs/eval-capture.md /
src/ / test/ files. CHANGELOG date bumped to 2026-04-26.
llms.txt + llms-full.txt regenerated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(v0.25.0): gbrain eval replay + contributor doc + CONTRIBUTING link
Closes the gap between "session capture works" (this PR's core) and
"contributors actually use it before merging." Three artifacts:
- src/commands/eval-replay.ts (~340 LOC) — reads NDJSON from `gbrain eval
export`, re-runs each captured query/search against the current brain,
computes set-Jaccard@k, top-1 stability, and latency delta. Stable JSON
shape (schema_version:1) for CI gating; human mode prints a regression
table sorted worst-first. Pure Bun, zero new deps. Stub-engine tests
cover Jaccard math, NDJSON parser (including v2 forward-compat
rejection + line-numbered errors), --limit, --verbose, --json, and
graceful per-row error handling. 16/16 passing.
- docs/eval-bench.md (~80 lines) — contributor guide. The 4-command loop
(export → change → replay → diff), metric definitions with healthy
ranges (Jaccard ≥0.85, top-1 ≥85%, latency Δ within ±50ms), trigger
paths, CI integration snippet, hand-crafted NDJSON corpus path for
fresh installs, and the off-switch. Pairs with the existing
docs/eval-capture.md which is the consumer-facing wire format.
- CONTRIBUTING.md gains a "Running real-world eval benchmarks (touching
retrieval code)" section with the trigger paths and a link to
docs/eval-bench.md. Reviewers now have a one-line ask: "did you run
replay?"
CLAUDE.md key files updated. CHANGELOG bullets added. llms.txt
regenerated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(v0.25.0): CONTRIBUTOR_MODE flag — capture off by default for users
Eval capture was on for everyone in the v0.25.0 draft. Privacy footgun:
end users had retrieval traffic accumulate in their brain DB without
asking, even with PII scrubbing. Flips to off by default + explicit
opt-in for contributors who actually use the replay loop.
Resolution order in isEvalCaptureEnabled():
1. config.eval.capture === true → on
2. config.eval.capture === false → off
3. process.env.GBRAIN_CONTRIBUTOR_MODE === '1' → on
4. otherwise → off
The env var is the contributor-facing toggle (one line in .zshrc, no
JSON edit). Explicit config wins both directions for users who want to
override per-brain.
PII scrubbing gate stays independent — default true regardless of
CONTRIBUTOR_MODE — so any brain that does capture still scrubs.
Tests rewritten: env var hygiene per-test (origMode preserved + restored
in finally). 9/9 pass; total v0.25.0 suite is 198/198.
Docs:
- README.md gains a Contributing-section pointer to the env var.
- CONTRIBUTING.md gains a "CONTRIBUTOR_MODE — turn on the dev loop"
section with verification commands and resolution-order table.
- docs/eval-bench.md leads with the prerequisite (must set the env var
for the rest of the doc to be useful).
- docs/eval-capture.md "Config" section split into Path A (env var) +
Path B (config) with explicit resolution-order rules.
- CHANGELOG v0.25.0 entry corrected ("on by default" was wrong) plus a
new top itemized bullet calling out the gate change.
- CLAUDE.md eval-capture entry annotated with the new gate logic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: post-ship documentation pass for v0.25.0
Cross-references every doc against the final state of the branch
(CONTRIBUTOR_MODE flag, eval replay tool, off-by-default capture):
- README.md: top callout rewritten — was implying capture-on-by-default
contradicting the gate landed in
|