d28be5d091
* v0.40.4.0 T1: shared audit-writer primitive Extract createAuditWriter() helper. Five hand-rolled JSONL audit modules (rerank-audit, shell-audit, supervisor-audit, audit-slug- fallback, phantom-audit) duplicated the same ISO-week filename math, best-effort write loop, and read-current-plus-previous-week loop. T2 refactors all 5 onto this primitive. Behavior preservation: filename format, JSONL line shape, mkdir recursive, appendFileSync utf8, stderr-on-failure all byte-identical to the existing modules so their tests pass unchanged. resolveAuditDir() moves here from shell-audit.ts; shell-audit.ts will re-export for back-compat (T2). Honors GBRAIN_AUDIT_DIR with whitespace-trim, falls back to ~/.gbrain/audit/. Test coverage: 22 cases covering ISO-week math + year-boundary edges (2027-01-01 → 2026-W53), env override, mkdir-recursive, fail-open stderr-warn shape, cross-week readback, corrupt-row skip, non-finite- ts skip, round-trip with nested fields, computeFilename + resolveDir accessors. Plan ref: D5=B audit unification cathedral expansion. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 T2: refactor 5 audit modules onto shared writer Replace the duplicated ISO-week filename math + best-effort write loop + read-current-plus-previous-week loop in: - src/core/rerank-audit.ts (rerank-failures-*.jsonl) - src/core/audit-slug-fallback.ts (slug-fallback-*.jsonl) - src/core/minions/handlers/shell-audit.ts (shell-jobs-*.jsonl) - src/core/minions/handlers/supervisor-audit.ts (supervisor-*.jsonl) - src/core/facts/phantom-audit.ts (phantoms-*.jsonl) All five now delegate file I/O to createAuditWriter from T1. Public API preserved bit-for-bit: - logRerankFailure, readRecentRerankFailures, computeRerankAuditFilename - logSlugFallback, readRecentSlugFallbacks, computeSlugFallbackAuditFilename - logShellSubmission, computeAuditFilename, resolveAuditDir - writeSupervisorEvent, readSupervisorEvents, computeSupervisorAuditFilename plus isCrashExit, summarizeCrashes, CrashSummary (domain-specific helpers stay in supervisor-audit.ts; only file I/O moves) - logPhantomEvent, readRecentPhantomEvents, computePhantomAuditFilename Domain-specific behavior preserved: - audit-slug-fallback emits per-call stderr (D7 dual logging) in the caller; the shared writer is failure-only stderr - rerank-audit truncates error_summary to 200 chars before write - phantom-audit spreads optional fields conditionally (skip undefined) - supervisor-audit keeps single-file readback (no cross-week walk) to preserve pre-v0.40.4 doctor assertions resolveAuditDir lives in src/core/audit/audit-writer.ts; shell-audit.ts re-exports it so existing imports keep working (every other audit module + gbrain-home-isolation.test.ts + minions.test.ts + minions-shell.test.ts pull resolveAuditDir from shell-audit.ts). Operator-visible drift: rerank-audit stderr line drops the 'rerank-failure audit' qualifier — was '[gbrain] rerank-failure audit write failed (...)' now '[gbrain] write failed (...); search continues'. Stderr is human-debugging, not machine-parsed; the file written gives the qualifier away in `tail -f audit/*`. Test coverage: 128/128 audit-touching tests pass unchanged. Plan ref: D5=B audit unification cathedral expansion. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 T3: getAdjacencyBoosts engine method (PG+PGLite parity) Add BrainEngine.getAdjacencyBoosts(pageIds) returning Map<page_id, AdjacencyRow{hits, cross_source_hits}>. Returns ALL pages with hits >= 1 (callers apply their own threshold). Cross-source semantic (D15=A): cross_source_hits EXCLUDES the target page's own source. A page in source A linked from 2 pages in source A reports cross_source_hits = 0. Linked from 1 in source B + 1 in source C reports 2. Source-scope contract: pageIds MUST already be source-scoped by the caller. Method does NOT filter by source_id. The in-set restriction makes cross-source leakage impossible by construction. JSDoc spells this out; same trust posture as cosineReScore's chunk_id handling. COALESCE(p.source_id, 'default') on both target and from-page sides for defense-in-depth even though pages.source_id is NOT NULL today. JSDoc/SQL contract alignment (codex #2): HAVING >= 1 matches the "returns ALL pages with hits >= 1" contract; threshold of 2 is the caller's call in applyGraphSignals. Known limitation (codex #15): cross_source_hits cannot distinguish "genuinely linked from another team" from "mirrored imports from another source." T-todo-4 captures the v0.41+ refinement. SearchResult type extension (D4=A flat fields, D12=A attribution): - graph_adjacency_hits, graph_cross_source_hits, graph_session_demoted, graph_session_prefix - base_score, backlink_boost, salience_boost, recency_boost, exact_match_boost, graph_adjacency_boost, graph_cross_source_boost, session_demote_factor, reranker_delta All optional; T4-T6 populate them. Test coverage: 7/7 hermetic PGLite cases. Empty input, singleton, same-source hub, cross-source attribution including the "linked-only-from-other-source" case (widget in source b, linked from alice+bob in source a → cross_source_hits=1), JSDoc HAVING>=1 contract. Postgres parity asserted by SQL-shape identity (will get a mirror Postgres E2E in T10's eval gate work via DATABASE_URL when set; PGLite hermetic case shipped now). NULL source_id COALESCE branch noted as untestable in current PGLite schema (pages.source_id is NOT NULL); kept as defense-in-depth. Plan ref: T3 in v0.40.4.0 wave plan; D1=A, D3=A, D15=A. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 T4+T11: applyGraphSignals 4th stage in runPostFusionStages New file src/core/search/graph-signals.ts. Three signals: 1. Adjacency-within-top-K (×1.05): hits >= 2 inbound from in-set. 2. Cross-source adjacency (×1.10, stacks): cross_source_hits >= 2. Dormant on single-source brains. 3. Session diversification (×0.95): if multiple top-K share a slug prefix, keep highest scoring, DEMOTE the rest. NOT amplify — codex caught the original framing was backwards (amplification of redundancy makes the cited "weak chunks compete for budget" problem worse, not better). Conservative magnitudes (D14=B): 1.05/1.10/0.95. Score-distribution probe (onScoreDistribution) collects min/p25/p50/p75/p95/max + reorder_band_width to feed T-todo-2 magnitude calibration wave. Slot: 4th stage inside runPostFusionStages (hybrid.ts:248), AFTER backlink/salience/recency, pre-dedup. Inherits the v0.35.6.0 floor-ratio gate from computeFloorThreshold — this is the structural protection that prevents a low-cosine hub from outranking a strong non-hub (codex T2 / D1=A). PostFusionOpts extends with graphSignalsEnabled, onGraphMeta, onScoreDistribution. Caller (hybridSearch in subsequent T5 work) resolves graph_signals from the mode bundle. Source-scope contract preserved: getAdjacencyBoosts takes raw page_ids, no source filter. Adjacency is in-set restricted so cross-source leakage is impossible by construction (D3=A). Fail-open: engine throw → JSONL audit row via shared createAuditWriter (T1/T2 primitive, featureName='graph-signals-failures') + meta.errored + caller's results unchanged. Session diversification ALSO skips on failure (predictable all-or-nothing posture). Mutation note (codex #9): score mutated in place. base_score must be stamped at runPostFusionStages entry BEFORE this stage so eval-capture sees pre-boost score (T6 attribution wave). Test coverage (24 cases, including T11 IRON RULE regression): - sessionPrefix multi/single/empty cases - computeScoreDistribution percentile math - Disabled + empty short-circuits - Adjacency hit, no-hit, cross-source stacking, cross-source alone - Session diversification 3-share + single-segment + singleton - Test seam injection (no engine call) - Fail-open: throw → audit row + meta.errored + unchanged - Empty Map → session still runs - Score-distribution always emits when enabled - Meta carries fire counts + duration_ms - Missing page_id silently skipped from dedup set - **T11 IRON RULE regression (3 cases):** * weak hub BELOW floor_threshold does NOT get boosted past above-floor non-hub (the bug class the floor gate exists for) * hub AT floor still gets boosted (gate is < not <=) * NaN score → NaN >= threshold is false → no boost Plan ref: T4 + T11 in v0.40.4.0 wave plan; D1=A, D2=A, D11=B, D14=B, D9=A, D5=B. Codex outside-voice #1 + #2 + #6 + #8 + #9 addressed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 T5: graph_signals mode-bundle knob + KNOBS_HASH bump 3→4 ModeBundle gains graph_signals: boolean. Per-mode defaults: - conservative: false (cost-sensitive tier) - balanced: true (the wave's primary surface for default-on) - tokenmax: true (power-user tier, capstone fit) SearchKeyOverrides + SearchPerCallOpts gain optional graph_signals field. resolveSearchMode picks via the standard per-call → config override → mode bundle chain. loadOverridesFromConfig parses 'search.graph_signals' from the config table ('1' or 'true' → true). SEARCH_MODE_CONFIG_KEYS adds the key so `gbrain search modes --reset` clears it alongside other knobs. KNOBS_HASH_VERSION bump 3→4 (append-only per CDX2-F13). New `gs=` parts entry appended AFTER cross-modal + column + prov entries. A graph-on cache write cannot be served to a graph-off lookup — mid-deploy hit-rate dip clears within cache.ttl_seconds (3600s). src/commands/search.ts KNOB_DESCRIPTIONS gains graph_signals entry so `gbrain search modes` dashboard renders the new knob. Test coverage: - test/search-mode.test.ts (+ 8 new cases): per-mode defaults canonical, config override both directions, per-call override wins, knobsHash distinct for on/off, config key registered, attributeKnob reports per-call + mode sources correctly. - test/search/knobs-hash-reranker.test.ts: version assertion bumped 3→4 with v0.40.4 rationale comment. - test/cross-modal-phase1.test.ts: version assertion bumped 3→4 with v0.40.4 rationale comment. - Canonical-bundle assertions updated to include graph_signals in expected shape (3 cases). 50/50 search-mode tests pass. 45/45 cross-modal pass. 17/17 knobs-hash-reranker pass. 10/10 balanced-reranker pass. Plan ref: T5 in v0.40.4.0 wave plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 T6: per-stage attribution stamping in every boost Every boost stage that mutates SearchResult.score now stamps a field recording WHAT it multiplied: - applyBacklinkBoost → backlink_boost (skipped when count == 0) - applySalienceBoost → salience_boost (skipped when score == 0) - applyRecencyBoost → recency_boost (skipped on evergreen prefix) - applyExactMatchBoost → exact_match_boost (skipped on no-match OR when intent's exactMatchBoost == 1.0 no-op) - runPostFusionStages → base_score stamped ONCE at entry, BEFORE any boost mutates r.score. Idempotent: caller-pre-stamped value preserved. Empty-results short-circuit unchanged. - applyReranker → reranker_delta = original_index - new_index (positive = rank improved; raw rerank score stays in rerank_score) - applyGraphSignals → graph_adjacency_boost, graph_cross_source_boost, session_demote_factor (T4 already stamped these) Why: feeds the T7 `gbrain search --explain` formatter so it can attribute the final score to its components. Without these stamps, "why did this rank where it did?" is grep-and-guess. SearchResult.reranker_delta doc updated to clarify it's a RANK delta (positive = improved), not a score delta. The raw relevance score stays in `rerank_score` (untyped, for back-compat with telemetry that already reads it). Test coverage: 16 new cases in test/search/attribution-stamping.test.ts. Pins: every boost stamps when it fires AND skips stamping when it doesn't (no false attribution on no-op stages). base_score idempotency preserved. reranker_delta computed correctly across rank-improved + rank-degraded cases. All 178/178 search tests pass (no regressions). Plan ref: T6 cathedral expansion in v0.40.4.0 wave plan; D12=A. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 T7: gbrain search --explain per-stage attribution New file src/core/search/explain-formatter.ts renders SearchResult[] as a multi-line breakdown of how the final score was formed: 1. people/alice (score=12.4) base=10.2 (rrf+cosine) + backlink ×1.08 + salience ×1.05 + adjacency ×1.05 (hits=3) + cross_source ×1.10 (other_sources=2) ↑ reranker rank +2 = final 12.4 Reads the boost_* / base_score / *_hits fields populated by T4 + T6. Empty path: "no boosts applied" when no stage stamped anything. Session demote rendered with `-` prefix (not `+`) so the demotion direction is visually distinct from boosts. CliOptions gains `explain: boolean`; parseGlobalFlags recognizes `--explain` anywhere in argv. cli.ts formatResult for `search` + `query` cases reads CliOptions.explain via the module-level singleton and routes to formatResultsExplain when set. Lazy import keeps the hot path narrow for the common non-explain case. Number formatting: 4-decimal precision, trailing zeros stripped ('1.0000' → '1', '0.1234' → '0.1234'). NaN preserved as 'NaN'. Test coverage: - test/search/explain-formatter.test.ts: 19 cases pin output format. Each boost type renders correctly, every-stage stacking composes, reranker_delta=0 doesn't render, empty list short- circuits, rank numbering 1-based, number formatting edge cases. - test/cli-options.test.ts: 3 new cases for --explain parsing (basic, absent default, any-argv-position). Existing CliOptions literals in test/cli-options.test.ts + test/thin-client-upgrade-prompt.test.ts updated for new required explain field. JSON envelope unchanged — the same attribution fields surface in existing --json output via JSON.stringify; no separate JSON formatter needed. Plan ref: T7 cathedral expansion in v0.40.4.0 wave plan; D12=A + D6=A. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 T8: doctor check graph_signals_coverage New checkGraphSignalsCoverage in src/commands/doctor.ts. Wired into both runDoctor (local engine) and doctorReportRemote (HTTP MCP / JSON path) so local AND remote-server brains both surface the metric. Logic: 1. Resolve active graph_signals setting: config override 'search.graph_signals' wins, else mode bundle default ('search.mode' → conservative=false, balanced/tokenmax=true). 2. When disabled → silent ok ("disabled — coverage not checked"). Avoids polluting doctor output on installs that don't use the feature. 3. When enabled, compute global inbound-link density: COUNT(DISTINCT to_page_id) / COUNT(*) across non-deleted pages. 4. <10% → warn ("signal will rarely fire") with paste-ready `gbrain extract all` fix hint. 5. >=30% → ok ("fire on most queries") with metric. 6. 10-29% → ok ("fire occasionally") with metric. Known limitation (codex outside-voice #14): global density is an imperfect proxy for "top-K subgraphs have enough edges to fire." T-todo-5 captures the v0.41+ refinement that measures actual fire rate from search-stats after 30 days of data. Best-effort: SQL errors → warn with the underlying message. Never breaks doctor. Test coverage (7 new cases in test/doctor.test.ts): - conservative mode → silent ok regardless of coverage - balanced default + 0 links → warn at 0% with fix hint - balanced default + 40% inbound → ok "fire on most queries" - balanced default + 20% inbound → ok "fire occasionally" - explicit search.graph_signals=false overrides mode default - empty brain → ok with explanation - check is wired into runDoctor (source-grep regression guard) All 55/55 doctor.test.ts cases pass. Plan ref: T8 in v0.40.4.0 wave plan; D6=A. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 T9: gbrain search stats graph_signals section runStatsSubcommand in src/commands/search.ts gains a graph_signals section in both --json and human output: Graph signals: enabled: true (mode default) failures: 3 fail-open event(s) ECONNREFUSED 2 timeout 1 Data sources: - config: 'search.graph_signals' override → enabled + source=config, otherwise mode-bundle default → enabled + source=mode_default. - JSONL audit: readRecentGraphSignalsFailures(days) returns events; failures_count is len, failures_by_reason buckets by first word of error_summary (e.g. 'ECONNREFUSED', 'timeout'). JSON envelope (schema_version 2 unchanged; graph_signals is a new sibling property of stats, so consumers reading the existing fields keep working): { "schema_version": 2, ...stats..., "graph_signals": { "enabled": bool, "source": "config" | "mode_default", "failures_count": int, "failures_by_reason": { reason: count } }, "_meta": { metric_glossary: { ..., graph_signals_enabled: ..., graph_signals_failures_count: ... } } } Fire-rate metrics (adjacency_fires, cross_source_fires, session_demotions) and score-distribution stats are NOT in this section yet — they require telemetry-table writes from the applyGraphSignals onMeta callback. Wired in v0.41+ via T-todo-2 calibration wave (the wave that needs them). For v0.40.4: status + error count is the actionable surface for "is graph_signals on, and is it failing?" Human output: prints the section after the existing stats block. Edge case: when total_calls is 0 BUT graph_signals is enabled OR has historical failures, still prints the section so operators don't lose the signal on a brain with no telemetry yet. Test coverage (6 cases in test/search/search-stats-graph-signals.test.ts): - search.graph_signals=true → enabled true, source=config - mode=conservative → enabled false, source=mode_default - no config → enabled true (balanced default), source=mode_default - JSONL failures bucketed by first word of error_summary - empty audit → failures_count 0, empty failures_by_reason - human output includes "Graph signals:" header Plan ref: T9 in v0.40.4.0 wave plan; D6=A. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 T10: eval gates (longmemeval-mini A/B + paired bootstrap) New test/e2e/graph-signals-eval.test.ts runs each longmemeval-mini question twice (graph_signals off, graph_signals on) and asserts: Gate 1 (QUALITY) — paired bootstrap, 10,000 resamples: - If signals-on is significantly WORSE than off (delta < 0 AND p < 0.05) → fail. - Otherwise pass. p>=0.05 either direction OR delta >= 0 → ok. Gate 2a (CHANGE-MAGNITUDE): mean Jaccard@5 over result-set overlap must be >= 0.5. If results overlap less than half, the change is too large and needs human review before default-on. Gate 2b (CHANGE-MAGNITUDE): top-1 stability rate >= 0.7. If 30%+ of top picks change, hard look required. Gate 3 (HARD ABSOLUTE FLOOR): recall@5 drop <= 5pt. Catastrophic regression catch (codex outside-voice #18 — addresses the "top-5 must not drop at all" brittleness on tiny fixtures). Bootstrap implementation: - Per-question observation is binary (recall@5 hit/miss). - Paired pairing on question_id between on/off branches. - Centered distribution under null (subtract observed mean) per standard paired-bootstrap-shift approach for binary outcomes. - Two-tailed p-value: |resampled delta| >= |observed delta|. - Deterministic seeded RNG so test runs are stable across CI. pairedBootstrapPValue exported as a pure function with separate tests for edge cases (empty input, all-equal, strong positive, strong negative, determinism). Reusable from future calibration waves. Hermetic: in-memory PGLite via createBenchmarkBrain + resetTables between questions. No API keys needed (--no-embed import path exercises keyword-only retrieval). Skips gracefully via describe.skip when the fixture is missing. Plan ref: T10 in v0.40.4.0 wave plan; D7=C absolute floor + D13=A paired bootstrap; codex #4 + #18 stability-vs-quality distinction. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 T12: VERSION + package.json + CHANGELOG + TODOS VERSION: 0.37.11.0 → 0.40.4.0 package.json: 0.37.11.0 → 0.40.4.0 CHANGELOG.md: top entry for v0.40.4.0 in ELI10-lead voice per CLAUDE.md release rules. Lead is plain-English ("Your search now notices when a page is a hub for your query"); precise file paths / SQL semantics / numbers live in the "Itemized changes" section below. Includes the cathedral-expansion notes (D5=B audit unification, D12=A per-stage attribution, D13=A eval gates) and the "To take advantage of v0.40.4.0" verify-and-fix block. TODOS.md: 5 new items captured under "v0.40.4 graph signals — deferred follow-ups (v0.41+)": - T-todo-1: profile graph-signal SQL latency, merge if hot (D8=C) - T-todo-2: magnitude calibration wave from probe data (D14=B / D17) - T-todo-3: DB-backed audit table for cross-deploy observability (codex #15) - T-todo-4: sync-topology-aware cross-source signal (codex #11) - T-todo-5: replace doctor's global density with fire-rate (codex #14) Verified the 3-line audit: VERSION + package.json + CHANGELOG topmost all match 0.40.4.0. `bun install` ran (lockfile unchanged — root package version isn't stored in bun.lock). `bun run build:llms` refreshed llms.txt + llms-full.txt for the next commit. Plan ref: T12 in v0.40.4.0 wave plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 TODO: document pre-existing shard-2 flake noticed during ship 3 isCacheSafe test failures in shard 2 reproduce on stashed clean master. Confirmed pre-existing — not introduced by v0.40.4. Filed under "Pre-existing flake on master (noticed during v0.40.4 ship)" with reproduction commands + remediation options. Shipping v0.40.4 through it; future wave can fix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 privacy scrub: replace wintermute → media in example slugs CLAUDE.md line 550 bans the private OpenClaw fork name in public artifacts. Example session prefix in sessionPrefix() docs + 3 test fixtures swept to 'media/chat/...' instead. Pre-existing scripts/check-privacy.sh in `bun run verify` caught it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 fix: wire graph_signals from mode bundle to runPostFusionStages CRITICAL: pre-landing review (codex outside-voice via /ship Step 9) caught that hybrid.ts's `postFusionOpts` literal at line 566 was building PostFusionOpts WITHOUT threading `resolvedMode.graph_signals` to `graphSignalsEnabled`. The gate at hybrid.ts:358 read the field from a literal that never set it. Result before this fix: the entire v0.40.4 graph-signals wave was dead code in production. Mode bundles set `balanced.graph_signals = true` and `tokenmax.graph_signals = true`, but no production call site ever reached applyGraphSignals. The KNOBS_HASH bump 3→4 correctly varied the cache key by the flag, so contamination was prevented — but the feature itself never fired. All shipped infrastructure (engine SQL, fail-open audit, attribution stamps, --explain formatter, doctor coverage check, search-stats section) was reachable only through the unit-test seam (`opts.adjacencyFn`). The CHANGELOG-advertised behavior never landed in user-visible search. Fix: thread `graphSignalsEnabled: resolvedMode.graph_signals` into the postFusionOpts literal (1 line). Inline comment names codex's catch so future refactors see the regression class. Tests: new test/search/graph-signals-wire-integration.test.ts pins the wire end-to-end. Three cases: 1. balanced mode → hybridSearch on a seeded brain with adjacency hub produces a result with base_score stamped (proves runPostFusionStages actually ran). 2. search.graph_signals=false config override → no graph_* fields stamped (proves the gate honors the override path). 3. Source-grep regression guard pinning the `graphSignalsEnabled: resolvedMode.graph_signals` literal in hybrid.ts so a future refactor can't silently disconnect. All 57 existing v0.40.4 wave tests still pass. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 fix: pre-landing review AUTO-FIX findings (audit msg drift + deleted_at) Two informational findings from /ship pre-landing review (Step 9): 1. Stderr message qualifier drift (rerank/slug-fallback/phantom audits) Pre-v0.40.4 messages included a per-feature qualifier: [gbrain] rerank-failure audit write failed (...) [gbrain] slug-fallback audit write failed (...) [gbrain] phantom audit write failed (...) The T2 refactor dropped the qualifier (plan promised "byte-identical" operator-visible behavior, but stderr lines did drift). Restored via new `errorMessagePrefix` option on `createAuditWriter` (optional, '' default). Three modules pass the per-feature qualifier; shell-audit and supervisor-audit unaffected (their pre-v0.40.4 messages didn't have a separate qualifier — label already carried the feature name). 2. Defense-in-depth `deleted_at IS NULL` on getAdjacencyBoosts SQL was previously protected by-construction (hybridSearch's visibility filter ensures input pageIds are live), but matches the v0.35.5.0 findOrphanPages pattern and closes the bug class if a future caller bypasses hybridSearch. Added to both Postgres and PGLite engines for parity. Three JOIN sites guarded (targets CTE, FROM-pages join). One inline comment per engine cites the codex review and the v0.35.5.0 precedent. Plan ref: /ship pre-landing review v0.40.4.0 (codex finding C and F). All 84 audit+graph-signals tests pass. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 fix: adversarial review HIGH findings (codex H1+H2 + Claude F1) Three HIGH-severity issues from /ship adversarial pass: H1 (Codex): Eval gate was a no-op. Test passed `graph_signals: graphSignalsOn` via `as any` cast, but SearchOpts had no field and hybridSearch's perCall didn't thread it. Both off/on branches resolved to the mode-bundle default — gate measured identical behavior, could pass while detecting nothing. Fix: add `graph_signals?: boolean` to SearchOpts (types.ts:794). Thread `opts.graph_signals` into perCall in both hybridSearch (hybrid.ts:425) AND hybridSearchCached (hybrid.ts:1027) so the cache-key resolver also sees the override. Drop the `as any` from the eval test — types are real now. H2 (Codex): Session diversification fired on entity directories. sessionPrefix() used "any shared parent directory" as the session signal. Result: a search for "people in SF" returned `people/alice` + `people/bob` + `people/charlie` and the latter two got demoted to 0.95×. Every common entity-search query silently penalized legitimate same-type results. Default-on for balanced/tokenmax means production behavior was wrong. Fix: narrow sessionPrefix() to fire ONLY when the slug contains a session-like marker (`chat`/`session`/`sessions` segment OR a `YYYY-MM-DD` date segment). Entity directories (`people/`, `companies/`, `docs/`) return null → diversification skips. Returns NULL (not the slug itself) so the loop skips clean. Examples in JSDoc: your-agent/chat/2026-05-20-foo → 'your-agent/chat/2026-05-20-foo' daily/2026-05-20/journal-entry-1 → 'daily/2026-05-20' transcripts/chat/funding-discussion → 'transcripts/chat/funding-discussion' people/alice → null ← codex H2 regression docs/quickstart → null F1 (Claude adversarial subagent): case-sensitivity drift across 3 sites. loadOverridesFromConfig in mode.ts is case-insensitive + whitespace-trimmed for 'search.graph_signals' values. But doctor's checkGraphSignalsCoverage (doctor.ts:899) AND search-stats's readGraphSignalsStats (search.ts:288) used case-sensitive compare. User sets `search.graph_signals TRUE`: production enables the feature, but doctor + search-stats both silently report disabled. Operators lose the only observability surface for the new feature on values like 'True'/'TRUE'. Fix: trim + lowercase parity at both sites. Mirror the parser's semantic. Also case-normalized `search.mode` reads at both sites for the same divergence class. Tests: - sessionPrefix block rewritten with 7 cases covering chat marker + date anchor + entity dirs (now-NULL) + degenerate (no /). - Added regression test pinning codex H2: people/alice + people/bob + people/charlie do NOT get diversified. - graph-signals-eval.test.ts drops `as any` — typed field works. - Existing tests using `chat/a`/`chat/b` updated to session-shaped `media/2026-05-20/chunk-a` so the date anchor actually fires. 111/111 graph-signals + doctor + search-stats tests pass. Typecheck clean. Plan ref: /ship adversarial review v0.40.4.0 (codex H1, H2; Claude F1). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.40.4.0 TODOs: capture 11 LOW adversarial findings for v0.41+ Codex L1 (audit window underreport) + Claude F2/F3/F5-F8/F11/F12/F14/F16 from /ship adversarial review. None are load-bearing; all captured under 'v0.40.4 adversarial review LOW findings — captured for v0.41+'. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update project documentation for v0.40.4.0 - README: surface v0.40.4.0 graph signals + --explain in Hybrid search capability - CLAUDE.md: annotate engine.ts getAdjacencyBoosts, new graph-signals.ts / explain-formatter.ts / audit/audit-writer.ts, plus hybrid.ts post-fusion 4th stage, mode.ts graph_signals knob + KNOBS_HASH 3→4, cli-options.ts --explain flag, search stats + doctor coverage check - llms-full.txt: regenerated from CLAUDE.md per the build:llms chaser rule Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(ci): pin bun-version to 1.3.13 across all workflows setup-bun action with `bun-version: latest` calls the GitHub API (https://api.github.com/repos/oven-sh/bun/git/refs/tags) to resolve the tag. CI started failing today with HTTP 401 "Bad credentials" even though the action receives a token (visible as `token: ***` in the run log). Pinning the version eliminates the API call entirely. Affected workflows: test.yml, e2e.yml, release.yml, heavy-tests.yml (5 invocations total). Pinned to 1.3.13 — matches package.json engines (`bun >= 1.3.10`) and the version v0.40.4.0 was developed against. Bump cadence: when a new bun version is required, update this pin in one PR. Trading "always-latest" for "always-deterministic" is the right trade for a 5-shard CI matrix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GBrain
Your AI agent is smart but forgetful. GBrain gives it a brain.
Built by the President and CEO of Y Combinator to run his actual AI agents. The production brain behind his OpenClaw and Hermes deployments: 146,646 pages, 24,585 people, 5,339 companies, 66 cron jobs running autonomously. The agent ingests meetings, emails, tweets, voice calls, and original ideas while you sleep. It enriches every person and company it encounters. It fixes its own citations and consolidates memory overnight. You wake up smarter than when you went to bed.
The brain wires itself. Every page write extracts entity references and creates typed links (attended, works_at, invested_in, founded, advises) with zero LLM calls. Hybrid search. Self-wiring knowledge graph. Structured timeline. Backlink-boosted ranking. Ask "who works at Acme AI?" or "what did Bob invest in this quarter?" and get answers vector search alone can't reach. Benchmarked side-by-side: gbrain lands P@5 49.1%, R@5 97.9% on a 240-page Opus-generated rich-prose corpus, beating its graph-disabled variant by +31.4 points P@5 and ripgrep-BM25 + vector-only RAG by a similar margin. Full BrainBench scorecards live in the sibling gbrain-evals repo.
New default in v0.36.2.0: ZeroEntropy for both embedding (zembed-1 at 1280d via Matryoshka) and reranker (zerank-2). On a real-corpus benchmark vs OpenAI and Voyage: 2.2× faster (442ms vs OpenAI 973ms), 2.6× cheaper at regular pricing ($0.05/M vs OpenAI $0.13), wins 11 of 20 queries head-to-head, reshuffles 60% of top-1 results when used as a second-pass reranker. Bring your own key from zeroentropy.dev, or switch to OpenAI/Voyage at install time via gbrain init --pglite --embedding-model <provider:model> --embedding-dimensions <N> — your choice is sticky. To switch an existing brain, run gbrain reinit-pglite --embedding-model <provider:model> --embedding-dimensions <N> (PGLite) or follow the SQL recipe in docs/embedding-migrations.md (Postgres). gbrain config set embedding_model is refused as of v0.37.11.0 because the schema column has to resize too.
GBrain is those patterns, generalized. Install in 30 minutes. Your agent does the work. As Garry's personal agent gets smarter, so does yours.
New in v0.40.2.0 — gbrain think grounds temporal answers in the typed-claim timeline. Ask "when did Marco last switch jobs" or "what was the ARR in March" and the answer comes back rooted in a real chronological timeline of the metric + event facts your brain already extracted via the extract_facts cycle phase. Default ON. The intent classifier (temporal / knowledge_update / other) is a regex pass with zero LLM cost; the 'other' fast path short-circuits with zero extra SQL. Migration v82 adds a nullable facts.event_type column so the same plumbing carries event-shaped rows ('meeting', 'job_change', 'location_change') alongside metric rows. Flip think.trajectory_enabled=false to opt out. Debug with GBRAIN_THINK_DEBUG=1 gbrain think "..." to see the spliced prompt. The same trajectory plumbing also lands in the LongMemEval benchmark with a methodology change disclosed in methodology_note: extractor=haiku-preprocess-full-haystack-v1 — published scores are "gbrain + Haiku-preprocess pipeline" vs "gbrain alone", NOT directly comparable to baseline LongMemEval numbers without that note.
New in v0.36.4.0 — Your agent drives the brain to 90/100 by itself. One command does the loop you used to run by hand: gbrain doctor --remediate --yes --target-score 90 --max-usd 5. It computes a dependency-ordered plan (sync before extract, embed after consolidate), submits each step as a Minion job, re-checks score between every step, and refuses to spend past your cost cap. Cron can drive it unattended. gbrain doctor --remediation-plan --json previews what would run. Autopilot now does the same thing on its 5-minute tick: small problems get targeted handlers, big problems get the full cycle, a healthy brain sleeps for 60 minutes instead of grinding through synthesize+patterns+embed every tick. Eleven new things you can submit as background jobs (reindex, repair-jsonb, orphans, integrity, purge, plus six cycle phases); three of them (synthesize, patterns, consolidate) are PROTECTED so an MCP-connected agent can't silently burn Anthropic credits. New --background flag on gbrain embed submits the job and exits with job_id=N for shell composition.
New in v0.35.7 — Temporal trajectory + founder scorecard. Author typed metric assertions in the ## Facts fence (mrr=50000, arr=2000000, team_size=12) and gbrain stores them as first-class typed columns. gbrain eval trajectory companies/acme-example prints the chronological history with regressions auto-flagged inline. gbrain founder scorecard companies/acme-example rolls up claim accuracy, consistency, growth direction, and red flags into a stable schema_version: 1 JSON contract. New MCP op find_trajectory exposes the same data to agents (read scope, visibility-filtered for remote callers). The consolidate cycle phase now writes valid_until on chronologically-superseded facts AND uses semantic upsert on (page_id, claim, since_date) — re-running the dream cycle on stable input is now a true no-op (fixed a pre-existing duplicate-takes bug from prior versions).
~30 minutes to a fully working brain. Database ready in 2 seconds (PGLite, no server). You just answer questions about API keys.
LLMs: fetch
llms.txtfor the documentation map, orllms-full.txtfor the same map with core docs inlined in one fetch. Agents: start withAGENTS.md(orCLAUDE.mdif you're Claude Code).
Install
GBrain runs in three shapes. Pick the one that matches how you use AI agents today.
Run with your agent platform
Already using OpenClaw or Hermes? GBrain installs as a skillpack scaffold into your agent's workspace.
gbrain init --pglite
gbrain skillpack scaffold --all # or: scaffold <name> per skill
That's it. Your agent picks up 43 skills (signal detection, brain-ops, ingest, enrich, citation-fixer, daily-task-manager, cron-scheduler, eval framework, and 35 more). Routing lives in skills/RESOLVER.md — the agent reads it once per request, picks the right skill, executes. Scaffolded skills are first-class members of your agent repo — you own them, edit freely; gbrain skillpack reference <name> diffs your copy against gbrain's bundle when you want to pull upstream improvements. (The legacy gbrain skillpack install managed-block model was retired in v0.36.0.0; run gbrain skillpack migrate-fence once if you're upgrading from an older release.)
CLI standalone
Use gbrain from any shell, no agent platform required.
bun install -g github:garrytan/gbrain
gbrain init --pglite # 2 seconds; no server, no Docker
gbrain doctor # verify health
Then point any MCP-aware client (Claude Code, Cursor, Windsurf) at it, or use it from your shell:
gbrain search "who works at acme AI?"
gbrain query "what did bob invest in this quarter?"
gbrain graph-query people/garry-tan --depth 2
Detailed setup paths (Postgres at scale, Supabase, thin-client mode) live in docs/INSTALL.md.
MCP server (any MCP client)
gbrain serve # stdio MCP (Claude Desktop / Code / Cursor)
gbrain serve --http # HTTP MCP with OAuth 2.1 + admin dashboard
# at /admin, SSE activity feed at /admin/events
Per-client guides (Claude Desktop, Code, Cursor, ChatGPT, Perplexity, Cowork) live under docs/mcp/. HTTP server supports DCR-style client registration, scope-gated access (read/write/admin), and built-in rate limiting.
How to get data in (v0.38+)
One command, local or hosted, synchronous receipt:
gbrain capture "the thought I want to remember"
gbrain capture --file ./notes/today.md
echo "from a pipe" | gbrain capture --stdin
SLUG=$(gbrain capture "..." --quiet)
The page lands in the DB AND on disk in one move (the v0.38 put_page
write-through plumbing). Default slug inbox/YYYY-MM-DD-<hash8> so
captures cluster in a predictable triage location. On thin-client installs
the verb routes through MCP to the server — same command, same UX.
For webhook ingestion (Zapier / IFTTT / Apple Shortcuts):
curl -X POST https://your-brain/ingest \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: text/markdown" \
-d "# a thought from a Shortcut"
For mobile capture, the inbox folder source picks up anything dropped into
~/.gbrain/inbox/ from iOS Shortcuts / AirDrop / Drafts / Finder.
Third-party skillpacks can ship custom ingestion sources (Granola, Linear,
voice, OCR) against the versioned IngestionSource contract at
gbrain/ingestion. See docs/skillpack-anatomy.md.
What it does (the loop)
signal → search → respond → write → auto-link → sync
(every (brain-first (informed (page + (typed edges (cron
message) retrieval) by context) timeline) + backlinks) keeps fresh)
- Signal detector runs on every message your agent receives. Captures ideas, entity mentions, time-sensitive todos, names, links.
- Brain-first lookup before any external API call. The cheapest, fastest, most personal information source you have.
- Auto-link fires on every page write. No LLM calls; pure pattern matching on
[[wiki/people/bob]]style references. New entity → new page stub → graph grows. - Cron-driven enrichment runs while you sleep: dedup people pages, fix citations, score salience, find contradictions, prep tomorrow's tasks.
The whole loop is described in docs/architecture/topologies.md with diagrams.
Capabilities
Hybrid search. Vector (HNSW on pgvector) + BM25 keyword + reciprocal-rank fusion + source-tier boost + intent-aware query rewriting. Three named search modes (conservative, balanced, tokenmax) bundle the cost/quality knobs into a single config key. Live cost/recall comparisons in docs/eval/SEARCH_MODE_METHODOLOGY.md. Default: balanced with ZeroEntropy reranker on. New in v0.40.4.0: per-query graph signals notice when a top result is a hub for THAT query (adjacency boost), is corroborated across team brains (cross-source boost), or is being crowded out by weak chunks from a chatty session (session demote). Run gbrain search "<query>" --explain to see per-stage attribution: base score, every boost that fired, what it multiplied. gbrain doctor ships a graph_signals_coverage check; gbrain search stats shows fire counts and failure breakdowns.
Self-wiring knowledge graph. Every put_page extracts entity refs from markdown/wikilinks/typed-link syntax and writes edges with zero LLM calls. Typed edges (attended, works_at, invested_in, founded, advises, mentions, …). Multi-hop traversal via gbrain graph-query. The graph is what produces the +31.4 P@5 lift over vector-only RAG.
Job queue (Minions). BullMQ-shaped, Postgres-native job queue. Durable subagents (LLM tool loops that survive crashes via two-phase pending→done persistence), shell jobs with audit, child jobs with cascading timeouts, rate leases for outbound providers, attachments via S3/Supabase storage. Replaces "spawn subagent as fire-and-forget Promise" with something that recovers from anything.
43 curated skills. Routing lives in skills/RESOLVER.md. Covers signal capture, ingest (idea / media / meeting), enrichment, querying, brain ops, citation fixing, daily task management, cron scheduling, reports, voice, soul audit, skill creation, eval framework, and migrations. Skills are markdown files (tool-agnostic), packaged as a single skillpack the installer drops into your agent workspace.
Eval framework. gbrain eval longmemeval runs the public LongMemEval benchmark against your hybrid retrieval. gbrain eval export + gbrain eval replay capture real queries and replay them against code changes (set GBRAIN_CONTRIBUTOR_MODE=1). gbrain eval cross-modal cross-checks an output against the task using three different-provider frontier models. Full methodology in docs/eval/SEARCH_MODE_METHODOLOGY.md.
Brain consistency. gbrain eval suspected-contradictions samples retrieval pairs, layered date pre-filter, query-conditioned LLM judge, persistent cache. Surfaces conflicts between takes + facts the agent has written. Wired into the daily dream cycle.
Integrations
Data flowing into the brain. Each integration is a recipe — markdown + setup hints — that ships in recipes/ and is discoverable via gbrain integrations list.
- Voice: Phone calls create brain pages via Twilio + OpenAI Realtime (or DIY STT+LLM+TTS). Setup recipe:
recipes/twilio-voice-brain.md. - Email + calendar: webhook handlers that route to brain signals.
docs/integrations/meeting-webhooks.md. - Embedding providers: 16 recipes covering OpenAI (default fallback), OpenRouter, Voyage, ZeroEntropy (default), Google Gemini, Azure OpenAI, MiniMax, Alibaba DashScope, Zhipu, Ollama (local), llama.cpp llama-server (local), LiteLLM proxy. Pricing matrix + decision tree in
docs/integrations/embedding-providers.md. - Credential gateway: vault-aware secret distribution.
docs/integrations/credential-gateway.md. - MCP clients: every major MCP client is supported.
docs/mcp/per-client setup.
Architecture
Two engines, one contract. PGLite (Postgres 17 via WASM, zero-config, default) for personal brains up to ~50K pages. Postgres + pgvector (Supabase or self-hosted) for shared / large / multi-machine deployments. The contract-first BrainEngine interface in src/core/engine.ts defines ~47 operations both engines implement; CLI and MCP server are generated from one source.
Brain repo is the system of record. Your knowledge lives in a regular git repo (your "brain repo") as markdown files. GBrain syncs the repo into Postgres for retrieval; deletes in git become soft-deletes in DB. You can publish public subsets, share team mounts, run thin-client setups pointing at a colleague's brain server. Topologies in docs/architecture/topologies.md.
Two organizational axes (brain ⊥ source). A brain is a database (your personal brain, a team mount you joined). A source is a repo inside that brain (wiki, gstack, an essay, a knowledge base). Routing lives in .gbrain-source dotfiles and resolves via a documented 6-tier precedence chain. Full diagrams in docs/architecture/brains-and-sources.md.
Why the graph matters. Vector search returns chunks that are semantically close. The graph returns chunks that are factually connected. Hybrid search pulls from both; auto-linking on every write keeps the graph fresh. Deep dive: docs/architecture/RETRIEVAL.md.
Troubleshooting
gbrain import fails with expected N dimensions, not M? Run gbrain doctor. It will print the exact gbrain config set ... or gbrain retrieval-upgrade command to repair the mismatch. You should not need to delete ~/.gbrain. As of v0.37, fresh gbrain init --pglite auto-detects your embedding provider from API keys in your environment — set OPENAI_API_KEY (or ZEROENTROPY_API_KEY / VOYAGE_API_KEY) before running init, or pass --embedding-model <provider>:<model> explicitly. With multiple keys set, init fires an interactive picker. In non-TTY contexts (CI, Docker) with no keys, init exits 1 with a paste-ready setup hint; pass --no-embedding to defer setup until runtime. See docs/integrations/embedding-providers.md for the full provider matrix and docs/operations/headless-install.md for Docker/CI sequencing.
Docs
docs/INSTALL.md— every install path, end to enddocs/architecture/— system design, topologies, retrieval theorydocs/guides/— how-to runbooks (sub-agent routing, minion deployment, skill development, brain-first lookup, idea capture, diligence ingestion)docs/integrations/— connecting external data sources (voice, email, calendar, embedding providers)docs/mcp/— per-client MCP setup (Claude Desktop, Code, Cursor, ChatGPT, Perplexity, Cowork)docs/eval/— eval framework, metric glossary, methodologydocs/ethos/— philosophy (thin harness, fat skills, markdown as recipes, origin story)AGENTS.md— entry point for non-Claude agentsCLAUDE.md— entry point for Claude Code (deep operating context)CONTRIBUTING.md— contributor guide, test discipline, eval-capture modeSECURITY.md— OAuth threat model, hardening defaults
Contributing
Run bun run test for the fast loop, bun run verify for the pre-push gate, bun run ci:local to run the full Docker-backed CI stack locally. Detailed test discipline in CONTRIBUTING.md.
Community PRs are batched into release waves rather than merged one-by-one — see the "PR wave workflow" section in CLAUDE.md. Contributor attribution stays attached via Co-Authored-By: trailers. We credit every accepted contribution in CHANGELOG.md.
If you find a bug or want a feature: open an issue first. Quick fixes (typo, doc bug, obvious regression) can go straight to a PR. Anything touching schema, retrieval ranking, MCP protocol, or the security boundary needs a design discussion in the issue first.
License + credit
MIT. Built by Garry Tan to run his OpenClaw and Hermes deployments — the production brain behind his actual AI agents.
Origin story: docs/ethos/ORIGIN.md.
Community PR contributors are credited in CHANGELOG.md per release. ZeroEntropy (@zeroentropy) for the embedding + reranker stack that became the v0.36.2.0 default. Voyage AI for the asymmetric-encoding recipe template. Ramp Labs for the search quality improvements lineage.