mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
814258dda67945ffec9457a1e73980e947b7e462
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
a19ee8bafe |
v0.40.2.0 feat: trajectory routing for temporal + knowledge_update (gbrain think + LongMemEval) (#1296)
* feat(facts): add event_type column + trajectory-format helper (v0.40.2.0 Commit 1)
Substrate work for v0.40.2.0 Track B (trajectory routing for temporal +
knowledge_update). This commit lands the schema + the shared formatter;
think wiring + LongMemEval extractor + intent routing come in Commits 2-4.
Migration v81 (facts_event_type_column):
ALTER TABLE facts ADD COLUMN event_type TEXT (nullable, metadata-only).
Lets the v0.35.4 typed-claim substrate carry event-shaped rows
(event_type='meeting'/'job_change'/'location_change') alongside the
metric-shaped rows (claim_metric/claim_value etc) it has carried since
v67. Temporal-reasoning questions ("when did I last meet Marco") need
the event shape; the metric shape doesn't fit them.
Engine changes (pglite + postgres parity):
- TrajectoryPoint.event_type: string | null added; projection in both
findTrajectory SQL paths returns the column.
- TrajectoryOpts.kind?: 'metric' | 'event' | 'all' added (default 'all').
Defensive opt that future-proofs filtering once event rows accumulate.
- Both engines apply the new kind filter at SQL level when set.
Back-compat (codex outside-voice concern):
Existing callers (founder-scorecard, eval-trajectory) already defensively
skip metric === null rows in their per-metric math. Event-only rows
(metric=NULL, event_type='meeting') ride through invisibly to those
callers — verified by the new regression test that asserts byte-identical
computeFounderScorecard + computeTrajectoryStats output with and without
event rows in the input. Both callers now pass kind:'metric' explicitly
for call-site clarity (no behavior change).
MCP find_trajectory op:
- event_type added to the wire-shape map.
- kind param added to the op declaration (enum metric/event/all).
Shared formatter (src/core/trajectory-format.ts, new):
formatTrajectoryBlock(points, entitySlug, opts) — sibling shape to
renderTakesBlock + renderChatBlock. Groups by (metric ?? event_type).
Per-metric cap 20, total cap 100 (prompt-budget guardrail). For
knowledge_update intent, annotates value-change rows with
"(superseded prior)" — the explicit signal codex flagged was missing
from default RRF-ordered retrieval. Promoted to src/core/ so both
gbrain think (Commit 2) and the LongMemEval harness (Commit 4)
consume one source of truth.
Prompt-injection coverage (codex Problem 10):
src/core/think/sanitize.ts INJECTION_PATTERNS extended with three
new entries — close-trajectory, open-trajectory, xml-attr-inject —
so adversarial </trajectory> sequences in extracted text get
escaped before reaching the model. Parity with the existing
</take> coverage.
Tests (all hermetic, no DATABASE_URL):
- test/trajectory-format.test.ts (17 cases, all green): grouping,
caps, sanitization, supersession annotation, determinism,
provenance, text-cap, adversarial </trajectory> escape.
- test/engine-parity-event-type.test.ts (6 cases): PGLite round-trip
of the column + kind filter matrix.
- test/regressions/v0_40_2_0-trajectory-backcompat.test.ts (4 cases):
pins the byte-identical-output contract that founder-scorecard's
per-metric math ignores event rows.
- test/migrate.test.ts: v81 round-trip verified via existing
structural assertion harness.
- 209 tests across 5 impacted suites pass; bun run verify clean
(17 pre-checks including privacy, jsonb, type, fuzz purity).
Plan: ~/.claude/plans/system-instruction-you-are-working-crystalline-owl.md
GSTACK REVIEW REPORT: CEO + ENG + CODEX CLEARED.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(think): trajectory injection for temporal + knowledge_update (v0.40.2.0 Commit 2)
Wires the v0.40.2.0 substrate (Commit 1's facts.event_type column +
formatTrajectoryBlock) into the production `gbrain think` surface.
Default ON; flip `think.trajectory_enabled=false` to opt out.
New pure modules (zero engine dependency):
- src/core/think/intent.ts — classifyIntent(question): regex-first
routing into 'temporal' | 'knowledge_update' | 'other'. KU wins over
temporal when both match. The 'other' fast path short-circuits with
zero SQL.
- src/core/think/entity-extract.ts — extractCandidateEntities() pulls
high-precision candidates from retrieval slugs (people/, companies/,
organizations/, deals/) and medium-precision noun phrases from the
question. Word-level tokenization + stop-word boundaries stitch
"Blue Bottle" as one candidate while splitting "I last meet Marco"
correctly. Leading-verb stripper drops "meet", "visit" etc so
"marco" surfaces cleanly. Cap of 5 per question.
Engine-touching wiring (src/core/think/index.ts):
- RunThinkOpts gains 4 fields: withTrajectory (default true),
sourceId, allowedSources, remote.
- readThinkTrajectoryEnabled() reads the config kill switch; default
true; survives missing config table on legacy brains.
- Trajectory orchestration sits between gather and prompt assembly:
intent classify → extract candidates → per-candidate
resolveEntitySlugWithSource → skip fallback_slugify → 5s timeout
Promise.race + 3-wide concurrency cap → formatTrajectoryBlock.
Any error degrades to "no block" + TRAJECTORY_INJECTION_FAILED
warning; the think call itself never crashes from trajectory.
- On success, TRAJECTORY_INJECTED_<N>_POINTS warning records the
count for downstream telemetry.
Prompt placement (src/core/think/prompt.ts) — Codex Problem 6 fix:
buildThinkUserMessage's trajectoryBlock slot honors BOTH existing
orderings — calibration mode inserts trajectory between calibration
and question; default mode inserts between retrieval and the output
instruction. NO third ordering is introduced. Empty trajectoryBlock
skips the "Known trajectory:" header entirely (don't cue the model
we tried).
Resolution-source signal (src/core/entities/resolve.ts) — Codex Problem 5:
New companion resolveEntitySlugWithSource() returns
{slug, source: 'exact_page' | 'fuzzy_match' | 'fallback_slugify'}
so trajectory routing can skip fallback-only resolutions —
querying findTrajectory on an invented slug always returns [] and
wastes a SQL round-trip. The original resolveEntitySlug keeps its
contract for pre-v0.40 callers.
MCP think op handler (src/core/operations.ts):
Extracts sourceScopeOpts(ctx) into scalar sourceId + allowedSources
+ remote, threads through to runThink. CLI callers omit (engine
default source, remote=false). Mirrors the same source-scope
discipline applied to all other read paths in v0.34.1.0.
Sanitization (Commit 1 already extended INJECTION_PATTERNS for
</trajectory> — consumed here).
Test coverage (all hermetic, no DATABASE_URL, no API keys):
- test/think-intent.test.ts (14 cases) — temporal, KU, other,
precedence (KU wins when both match), defensive non-string inputs.
- test/think-entity-extract.test.ts (10 cases) — retrieved-slug
source, noun-phrase source, stop-word stripping, leading-verb
stripping, dedup across sources, 5-candidate cap.
- test/think-trajectory-injection.test.ts (7 cases against PGLite
in-memory) — temporal intent injection happy path with superseded-
prior annotation, "other" intent short-circuit, withTrajectory:
false bypass, think.trajectory_enabled=false config bypass,
empty-trajectory skip, engine.findTrajectory throw is caught
(Promise.allSettled defense), TRAJECTORY_INJECTED warning count.
- Existing test/think-pipeline.serial.test.ts re-asserted unchanged
(10 cases — calibration mode parity, gather, sanitization,
cite-render all intact).
72 tests pass across 7 impacted suites; bun run verify clean (17 pre-
checks). Defaulted on per CEO + Eng D1; kill switch via config.
Plan: ~/.claude/plans/system-instruction-you-are-working-crystalline-owl.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(longmemeval): inline Haiku claim extractor + content-hash cache (v0.40.2.0 Commit 3)
Populates the LongMemEval benchmark brain's facts table inline at
import time so Commit 4's intent routing has data to retrieve. Per the
CHANGELOG D1 decision, this is full-haystack preprocessing — disclosed
explicitly in the benchmark output's methodology_note field (Commit 4).
New module src/eval/longmemeval/extract.ts:
extractAndInsertClaims({engine, client, model, sessionSlug,
sessionId, sessionBody, sourceId, aliasMap})
- Hashes the session body (sha256) for cache lookup.
- Cache hit → reuses parsed claims (cuts a 3-iteration benchmark
run from $1.50 to $0.50 when sessions repeat across questions,
as they do in LongMemEval).
- Cache miss → one Haiku call. System prompt asks for
{entity, metric, value, unit, period, event_type, valid_from, text}[]
JSON. New parseExtractedJsonArray() helper does fence-strip + parse
(parseModelJSON from cross-modal-eval is shaped for scored objects,
not arrays — different parser needed here).
- Per-record validateClaim() drops malformed records (missing
entity, bad date) silently; the rest land in NewFact rows.
- Per-question AliasMap (Codex Problem 4 — semantics pinned):
"Marco" + "Marco Smith" + "marco" in the SAME question collapse
to one slug via first-mention-wins canonicalization. Across
questions, the harness creates a fresh map (no leak).
- Real-page-aware entity resolution via the v0.40.2.0
resolveEntitySlugWithSource (Commit 2). Slugify-fallback rows
still insert (we need the data); the resolution_source signal
is only consulted at trajectory retrieval time (Commit 4).
- Bulk insert via engine.insertFacts with the
`gbrain-allow-direct-insert` allow-list comment per the
check-system-of-record CI guard contract — benchmark brain is
ephemeral in-memory PGLite, no markdown source-of-truth applies.
- Fail-open posture: Haiku throw, malformed JSON, insert collision
all return inserted=0 without throwing. One bad session never
kills the per-question loop.
- getCacheStats() exposes hits/misses/size for the per-run stderr
telemetry Codex Problem 14 asked for (empirical hit-rate
reporting; the optimistic claim self-verifies).
Substrate plumbing (extends Commit 1):
- NewFact.event_type?: string | null added in engine.ts so the
extractor can pass event-shaped rows through to insertFacts.
- PGLite engine + Postgres engine insertFacts() now persist
event_type. Param-positional dispatch extended to 20/21 placeholders
(null-embedding vs embedding-present); tx.unsafe vector cast on
Postgres path unchanged.
Test coverage (test/longmemeval-extract.test.ts, 13 cases, hermetic):
- Happy path: typed-claim + event rows both insert with correct
kind (event_type='meeting' → kind='event'; claim_metric='mrr'
→ kind='fact').
- Alias map: per-session collapsing ("Marco" + "Marco Smith"),
cross-session persistence within one question, fresh map per
question (caller-clears semantics pinned).
- Content-hash cache: identical body → cache hit, only ONE Haiku
call across two sessions; different bodies miss; getCacheStats
reports hits/misses/size.
- Fail-open: malformed JSON, Haiku throw, empty array output,
invalid records (missing entity, bad date) — none crash; 0
inserted in each case.
55 tests pass across 4 impacted suites; bun run verify clean.
Plan: ~/.claude/plans/system-instruction-you-are-working-crystalline-owl.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(longmemeval): trajectory intent routing + prompt splice + methodology disclosure (v0.40.2.0 Commit 4)
The final wiring: per-question intent classification + trajectory call
+ block splice into the answer-gen prompt. Plus the methodology
disclosure stamps that close out the Codex D1 contract.
New module src/eval/longmemeval/intent.ts:
classifyIntent(q): prefers q.question_type from the dataset
(LongMemEval ships labels like 'temporal-reasoning',
'knowledge-update', 'single-session-user') before falling back to
the SHARED regex set imported from src/core/think/intent.ts.
Single source of truth for the regex — think and longmemeval
cannot drift.
Harness wiring in src/commands/eval-longmemeval.ts:
- runEvalLongMemEval() spawns an extractor model via resolveModel
(tier:'utility' → haiku) when trajectory routing is enabled.
Calls resetExtractorState() once per benchmark run so the
content-hash cache + counters start clean.
- runOneQuestion() creates a FRESH per-question AliasMap (Codex
Problem 4 — first-mention-wins canonicalization stays scoped to
one question, never leaks across).
- Per session: after importFromContent lands, extractAndInsertClaims
populates the facts table. Fail-open if the Haiku call errors;
next session keeps going.
- After hybridSearch returns: classifyIntent(q) routes
temporal/knowledge_update through extractCandidateEntities (the
SHARED helper from Commit 2's think/entity-extract) → per-candidate
findTrajectory with 5s Promise.race timeout → formatTrajectoryBlock.
First candidate with a non-empty trajectory wins.
- generateAnswer() splices the trajectory block BEFORE the
Retrieved sessions block. Empty block (no entity match / no
points) → no "Known trajectory:" header (don't cue the model
we tried).
- JSON envelope gains 5 fields per question when trajectory routing
is on: intent, trajectory_points, entity_resolved,
resolution_source, methodology_note. methodology_note also
written to stderr at run completion.
Resolution-source gate DIVERGES from think (intentional):
In the think production path, fallback_slugify results are skipped
because querying invented slugs wastes SQL — production brains have
canonical pages. In the LongMemEval benchmark, there ARE no
canonical pages; both the extractor and the lookup go through
slugify-fallback on the same free-form name, so they cohere on the
same slug. Applying the think-path gate here would permanently
block trajectory injection on the benchmark. Comment in
runOneQuestion documents the divergence.
New CLI flag --no-trajectory:
Bypasses BOTH the Haiku extractor AND the per-question intent
routing. Used by the measurement protocol to baseline default-on vs
no-trajectory across 3 seeds per condition with paired-bootstrap
CI. Documented in the help text.
New RunOpts fields:
- extractorClient?: ThinkLLMClient — separate stub from the
answer-gen client so tests can isolate the two surfaces.
- extractorModel?: string — model override for the Haiku call.
methodology_note = 'extractor=haiku-preprocess-full-haystack-v1'
stamped on:
- Every per-question JSON envelope row.
- Stderr summary at run completion.
This is the Codex D1 contract: the temporal-reasoning delta we
publish is "gbrain + Haiku-preprocess pipeline" vs "gbrain alone",
not directly comparable to LongMemEval's published baselines
without that disclosure.
Extractor cache hit-rate stderr summary (Codex Problem 14):
'[longmemeval] extractor.cache_hits: 412 / 489 sessions (84.2%,
cached_bodies=412)' — empirical verification of the optimistic
hit-rate claim. The optimistic number self-verifies per run.
Test coverage (all hermetic, no API keys):
- test/longmemeval-intent.test.ts (9 cases) — dataset
question_type → Intent mapping for all six LongMemEval labels;
dataset label trumps question-text signal; unknown labels fall
through to the regex classifier.
- test/longmemeval-trajectory-routing.test.ts (4 cases) —
end-to-end through runEvalLongMemEval with both clients stubbed:
trajectory block lands in answer-gen prompt for temporal
intent + absent for 'other'; --no-trajectory bypasses extractor
AND injection AND omits envelope fields; methodology_note
stamped on every routed row; perf gate preserved (< 10s for
2-question fixture).
118 tests pass across 11 impacted suites; bun run verify clean.
Wave complete. CHANGELOG draft + measurement plan live in the plan
file. v0.40.2.0 ready for /ship after a real-LLM spot-check run.
Plan: ~/.claude/plans/system-instruction-you-are-working-crystalline-owl.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.40.2.0)
v0.40.2.0 trajectory routing wave — gbrain think now grounds answers
about temporal/knowledge-update questions in the typed-claim timeline
the brain has been quietly building via the extract_facts cycle phase.
Default ON; flip think.trajectory_enabled=false to opt out.
LongMemEval-side wiring lands the same plumbing in the benchmark
harness with explicit methodology disclosure (extractor=haiku-preprocess-
full-haystack-v1) in the JSON envelope and stderr summary — the published
temporal-reasoning number is "gbrain + Haiku-preprocess" vs "gbrain alone",
not directly comparable to LongMemEval's published baselines without that
disclosure.
Plan: ~/.claude/plans/system-instruction-you-are-working-crystalline-owl.md
3 review passes: CEO + ENG + CODEX all CLEARED.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: sync project docs for v0.40.2.0 trajectory routing
CLAUDE.md, README.md, AGENTS.md extended with the v0.40.2.0 trajectory
routing surface: gbrain think integration (default ON via
think.trajectory_enabled config key), facts.event_type schema column +
TrajectoryPoint.event_type + TrajectoryOpts.kind filter, shared
formatTrajectoryBlock helper in src/core/trajectory-format.ts,
LongMemEval extractor + intent routing + methodology disclosure,
migration v82.
llms-full.txt regenerated to match CLAUDE.md edits (CI test/build-llms
gate).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test: fill v0.40.2.0 unit + e2e gaps (71 new tests across 7 gap areas)
Audit of the trajectory-routing wave's test surface vs the shipped
code surfaced 7 gaps. All filled, all green. Total: 343 tests across
17 impacted suites (was 272 pre-fill).
Gap 1 — Migration v86 structural tests (11 new in test/migrate.test.ts):
- v86 entry exists with documented name + idempotent
- exactly one event_type column add to facts
- IF NOT EXISTS guard
- column is nullable (no NOT NULL, no DEFAULT regression guard)
- does NOT create any index (event_type is selectivity-poor)
- does NOT touch any other table (blast-radius pin)
- does NOT carry a sqlFor override (engine-shared SQL contract)
- PGLite round-trip: column exists with right type + nullable
- event_type INSERT/SELECT round-trip
- NULL round-trip for legacy + metric-only rows
- LATEST_VERSION >= 86 contract pin
Gap 2 — resolveEntitySlugWithSource branch coverage (12 new in
test/entity-resolve.test.ts):
- exact_page branch (full slug, slug-shape match)
- fuzzy_match branch (Title-cased display name, bare first name via
prefix expansion)
- fallback_slugify branch (unseeded name, multi-word non-match
phrase, accented input)
- null tail (empty + whitespace)
- back-compat parity with resolveEntitySlug for both exact_page and
fallback_slugify branches
Gap 3 — INJECTION_PATTERNS dedicated coverage for new entries (18 new
in test/think-sanitize-trajectory.test.ts):
- close-trajectory entry registered + matches canonical and
whitespace/case variations
- open-trajectory entry registered + matches both no-attr and
with-attrs forms
- xml-attr-inject strips entity=/metric=/event_type=/kind=
- does NOT strip non-trajectory attribute names (class/id/title)
- combined multi-vector attack: all three patterns fire
- formatTrajectoryBlock end-to-end with adversarial extractor text:
one live </trajectory> (the wrapper, not the injection); one
entity= attribute (the wrapper, not the injection)
- pattern ordering invariant: new entries land after close-take
Gap 4 — runThink calibration-mode placement contract (3 new in
test/think-trajectory-injection.test.ts):
- default mode: question → pages → takes → trajectory → instruction
- calibration mode: pages → takes → calibration → trajectory →
question → instruction (Codex P6 — no third ordering invented)
- empty trajectory in calibration mode preserves the existing
calibration shape (no false-positive cue)
Gap 5 — runThink resolution_source != fallback_slugify gate (1 new
in test/think-trajectory-injection.test.ts):
- candidate that only matches via fallback_slugify is NOT queried
(think-path divergence from longmemeval-path which accepts it)
Gap 6 — E2E for runThink trajectory injection (7 new in
test/e2e/think-trajectory-pglite.test.ts):
- full pipeline lands <trajectory> block in answer-gen prompt
- knowledge_update intent annotates value-change rows with
(superseded prior)
- 'other' intent short-circuits (no block, no SQL)
- think.trajectory_enabled=false config bypasses entire path
- empty brain → graceful no-op (no crash, no block)
- multi-entity deterministic ordering
- adversarial </trajectory> in seeded fact text is escaped before
reaching the LLM (end-to-end sanitization gate)
Gap 7 — longmemeval extractor stress + persistence pins (6 new in
test/longmemeval-extract.test.ts):
- alias map cross-session stress with 12 sessions in one question;
all 12 rows collapse under ONE entity_slug
- different entities stay separate across many sessions
- embedding + embedded_at both NULL on benchmark-inserted rows
(regression guard against accidental embed-on-write)
- row_num sequential + source_markdown_slug stamped per session
(v0.32.2 partial UNIQUE index contract)
- source field stamped "longmemeval:extractor" (audit-tag pin)
- cache key invariance: same body hash hits cache across different
sessionId/slug
bun run verify clean (17 pre-checks). No regressions in any of the
14 non-new impacted suites.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test: exempt facts.event_type from schema-bootstrap-coverage CI guard
The schema-bootstrap-coverage CI guard (test/schema-bootstrap-coverage.test.ts)
enforces that every ALTER TABLE ADD COLUMN in MIGRATIONS is covered by
applyForwardReferenceBootstrap OR by PGLITE_SCHEMA_SQL's CREATE TABLE
bodies OR by COLUMN_EXEMPTIONS.
v0.40.2.0's migration v87 adds facts.event_type but deliberately ships
without a bootstrap probe because:
- No CREATE INDEX in PGLITE_SCHEMA_SQL references event_type
- No FK references event_type
- All existing callers (founder-scorecard, eval-trajectory, gbrain
think trajectory injection) defensively skip NULL-metric rows in
per-metric math, so event_type=NULL on pre-v87 brains is invisible
- Pre-v87 brains land event_type=NULL via the migration ALTER
Exactly mirrors the precedent set by facts.claim_metric / claim_value /
claim_unit / claim_period exemptions (v67 typed-claim columns) which
are exempted for the same structural reason: column-only migration,
no forward-reference index, no downstream filter breaks on old brains.
Adding facts.event_type to COLUMN_EXEMPTIONS with a brief rationale
comment matching the existing v0.35.6 entry shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
|
0c6fcab555 |
v0.35.4.0 fix(doctor,entities): supervisor crash classification + bare-name resolver + 58x perf + stub guard observability (#1085)
* fix(doctor,entities): supervisor crash classification + bare-name resolver + stub guard - doctor.ts/jobs.ts: classify worker exits with code !== 0 as real crashes vs code === 0 clean restarts (separate counter); fixes false-positive WARN on healthy supervisors - entities/resolve.ts: prefix-expansion step between fuzzy match and slugify fallback catches bare first names that score too low on pg_trgm; picks highest-connection candidate as tiebreaker - facts/fence-write.ts: stub-creation guard refuses to spawn unprefixed entity pages at brain root - facts/backstop.ts: routes stubGuardBlocked facts to engine.insertFact so the fact still persists even when no markdown file is created - docs/issues/doctor-auto-heal-and-scoring.md: spec for follow-up doctor health-score improvements - .gitignore: guard reports/network-intelligence/ (private brain exports) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(privacy): scrub real names from entity-resolve test fixtures and JSDoc Replace YC partner names with placeholders per CLAUDE.md privacy rule: alice-example, bob-example, charlie-example, dave-example. Stripe and Stripe Atlas retained (allowed household brands; exercises the two-word company-prefix case). Test semantics preserved: - Alice / Dave: single-match cases - Bob / Charlie: multi-match tiebreaker cases (winner has more chunks) All 13 entity-resolve cases pass with the scrubbed fixtures. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(supervisor): extract classifyWorkerExit() helper (DRY) Three call sites were inline-classifying worker exits: supervisor's restart policy (child-worker-supervisor.ts:291), doctor's supervisor check (doctor.ts:1016), and jobs supervisor status (jobs.ts:806). Same rule, three copies — drift risk if one is updated without the others. Extract to src/core/minions/exit-classification.ts as a pure function. Signature consumes audit-JSON shape ({ code: number | null }) so doctor and jobs (which read serialized events from JSONL) and supervisor (which reads Node's exit callback) call the same function. Helper's classification rule: code === 0 → clean_exit, everything else (non-zero, null, undefined, missing) → crash. Default-to-crash prevents corrupted rows from silently demoting into the clean-restart bucket. 5 hermetic unit tests (test/exit-classification.test.ts) pin all edge cases. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(facts): audit + sunset comment for stub-guard fires Wire telemetry into the v0.34.5 stub-guard at fence-write.ts:190. Every guard fire now appends a JSONL line to ~/.gbrain/audit/stub-guard-YYYY-Www.jsonl with {ts, slug, source_id, fact_count}. Operator visibility for the sunset criterion: when the new audit log reads <5 hits/week for 3 consecutive weeks on production brains, the prefix-expansion in resolveEntitySlug is sufficient and the guard can be removed in v0.36. Reader (readRecentStubGuardEvents) deliberately diverges from supervisor-audit.ts:readSupervisorEvents — it reads BOTH the current AND previous ISO-week file before filtering by ts. supervisor-audit's reader only reads the current week, which loses 24h-window correctness across Monday 00:00 UTC (a Sunday 23:55 event lives in last week's file). The 2-file read costs nothing and makes the window actually 24h. 9 hermetic unit tests pin filename math, the writer's swallows-errors contract, the cross-week-boundary read, sort order, missing-file behavior, and malformed-row tolerance. The cross-week test is the regression guard: if a future refactor copies the supervisor's single-file pattern, that test fails. Follow-up TODO (not in this PR): fix readSupervisorEvents to use the same 2-file pattern. The new stub-guard reader becomes the canonical template to copy back. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(doctor): stub_guard_24h check surfaces resolver gaps Adds a new doctor check that reads ~/.gbrain/audit/stub-guard-YYYY-Www.jsonl (via the dual-week-aware reader from T8) and surfaces the 24h fire count. WARN at >10 fires — at that rate the prefix-expansion in resolveEntitySlug is probably missing a case (typo prefix, alias, non-Latin script) and operators should grep the audit log for the offending slugs. Below the threshold but non-zero shows as OK with a count, so operators can watch the v0.36 sunset criterion (<5/week for 3 weeks → guard can be removed). Zero hits emits no check, keeping the doctor output clean on healthy brains. 5 source-grep regression tests pin the contract: check name, WARN threshold, fix hint mentions the audit log + the resolver function name, reader is the dual-week-aware variant (NOT the supervisor-audit single- week pattern), and zero-hits stays silent. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(facts): pin stub-guard contract at writeFactsToFence + backstop layers - fence-write.test.ts: 3 new cases for the v0.34.5 stub guard. Bare slugs return {inserted: 0, stubGuardBlocked: true, ids: []} and create no file/.tmp at brain root. Prefixed slugs bypass the guard (regression guard against accidentally inverting the slug.includes('/') check). Empty facts array short-circuits before the guard fires. - facts-backstop.test.ts: 1 new case for the end-to-end routing. A bare-name LLM extraction resolves through to a bare slug, hits the guard, and lands in the facts table via engine.insertFact (DB-only). No phantom .md file; entity_slug stores the bare slug; source_markdown_slug is null. This is the routing contract Codex flagged as a "split-brain" data shape — the test pins the by-design behavior so a future refactor can't silently drop these facts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(supervisor): pin classifyWorkerExit consumer wire-up + regressions 12 new cases on top of the 5 helper unit tests: - doctor.ts / jobs.ts / child-worker-supervisor.ts each import the helper - All three call classifyWorkerExit at least once - doctor.ts and jobs.ts no longer carry the pre-T7 inline filter - supervisor uses the helper result to choose the clean_exit branch - audit-event shape round-trip: code=0 → clean_exit, code=1 → crash, code=null+SIGKILL → crash (catches future shape changes) The regression guards (3) and the wire-up checks (6) close the gap that motivated T7 in the first place: if a future change accidentally re-inlines the filter or shifts the audit event shape, the test fails before production sees the silent divergence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf(entities): correlated subqueries scoped to slug-LIKE candidates Replace the derived-table JOIN shape in tryPrefixExpansion with correlated subqueries. The pre-fix SQL did LEFT JOIN (SELECT to_page_id, COUNT(*) FROM links GROUP BY to_page_id) li ON ... which forced the planner to aggregate the entire links + content_chunks tables on every prefix-expansion call — O(N) per call where N is total links/chunks in the brain. On a 100K-link / 50K-chunk brain that's slow enough to bottleneck fact-extraction. New shape uses correlated subqueries: (SELECT COUNT(*) FROM links WHERE to_page_id = p.id) + (SELECT COUNT(*) FROM links WHERE from_page_id = p.id) + (SELECT COUNT(*) FROM content_chunks WHERE page_id = p.id) The slug LIKE filter is already selective (typical brain has 0-5 pages per prefix), so the three subqueries run N≈3 times per matched row against the existing indexes on links.to_page_id, links.from_page_id, and content_chunks.page_id. Behavior preserved: 13/13 entity-resolve tests pass (single-match + multi-match tiebreaker + edge cases). Codex's outside-voice review caught the dead-end design that an earlier draft of this plan proposed (a CTE with `LIMIT 50` candidate cap — would have excluded correct high-connection candidates if their slug sorted late). Correlated subqueries without a candidate cap are the cleaner shape that lets the LIKE filter do the bounding work. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(entities): perf regression guard for prefix-expansion (58x speedup) Hermetic PGLite benchmark with 5K pages + 50K links + 25K chunks. Runs the pre-T12 derived-table shape and the new correlated-subquery shape side-by-side against the same fixture, asserts NEW >= 5x faster than OLD. Baseline-ratio, not absolute wall-clock — different machines / Bun versions / CI load can shift absolute timings by 10x without indicating a real regression, but the SHAPE difference between "aggregate the full tables" and "correlated subquery per candidate" is what we care about. Measured: old_median=18.16ms, new_median=0.31ms, speedup=58.22x. The 5x assertion has plenty of headroom. The OLD SQL is embedded verbatim as the regression baseline. If a future refactor re-introduces full-table aggregation (LEFT JOIN against SELECT...GROUP BY over the whole links or content_chunks table), the test fails. PGLite-only — Postgres planner can shape derived-table JOINs differently enough that the 5x ratio could be noise on a 5K-page fixture. The structural correctness of the rewrite is the same on both; this is purely a planner-shape regression guard. .slow.test.ts suffix keeps it out of the fast loop (run via `bun run test:slow`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.35.2.0) Wave content: - Privacy scrub: PII rebuilt out of branch history; real names → placeholders - Bug fix: doctor + jobs no longer count clean worker exits as crashes - Bug fix: entity resolver prefix-expansion catches bare first names - DRY refactor: classifyWorkerExit() helper (one rule, 3 call sites) - Observability: stub_guard_24h doctor check + ISO-week audit log - Perf: 58x speedup on tryPrefixExpansion query shape Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: rebump v0.35.2.0 → v0.35.4.0 + scrub TODOS.md privacy violation VERSION/package.json/CHANGELOG header rebumped to v0.35.4.0 per user request (queue allocation). TODOS.md rephrased to not literally name the banned private-agent string — that was the CI failure root cause on the v0.35.2.0 push. CHANGELOG.md is on check-privacy.sh's allow-list (meta-documentation exception); TODOS.md is not. CI re-runs against this commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |