diff --git a/AGENTS.md b/AGENTS.md index aafe539c2..fdde87467 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -59,6 +59,14 @@ writing or reviewing an operation, consult `src/core/operations.ts` for the cont ` (v0.28.8) runs against an isolated in-memory PGLite per question — your `~/.gbrain` is never opened. Full guide: [`docs/eval-bench.md`](./docs/eval-bench.md). +- **Track a founder/company over time (v0.35.7):** when an entity has + typed metric claims in its `## Facts` fence (`metric: mrr`, `value: 50000`, + `unit: USD`, `period: monthly` columns), run + `gbrain eval trajectory ` for the chronological history + with regressions auto-flagged, or `gbrain founder scorecard ` + for a four-signal JSON rollup (claim_accuracy / consistency / + growth_trajectory / red_flags). MCP op `find_trajectory` exposes the + same data — read scope, visibility-filtered for remote callers. - **Everything else:** [`./llms.txt`](./llms.txt) is the full documentation map. [`./llms-full.txt`](./llms-full.txt) is the same map with core docs inlined for single-fetch ingestion. diff --git a/CHANGELOG.md b/CHANGELOG.md index c11e459c4..fbe80e39c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,80 @@ All notable changes to GBrain will be documented in this file. +## [0.35.7.0] - 2026-05-17 + +**The contradiction probe grew up. Typed claims over time, regressions detected automatically, founder scorecards as a one-liner.** + +v0.35.3.1 taught the probe to see dates. v0.35.7 turns dates into a proper time-series substrate. Your `## Facts` fences can now carry typed metric assertions (mrr=50000, arr=2000000, team_size=12), the consolidate cycle phase writes `valid_until` on chronologically-superseded facts, and two new commands turn that data into operator signal: `gbrain eval trajectory ` shows the sorted claim history with regressions flagged inline, and `gbrain founder scorecard ` rolls up claim accuracy, consistency, growth direction, and red flags into one JSON payload. + +This is the wave the original temporal-contradiction RFC deferred to "Phases 2-4." Three rounds of review (CEO, eng, codex outside-voice) caught a security regression in the planned API, a pre-existing cycle-idempotency bug that would have poisoned trajectory data on every dream cycle re-run, and a misidentified file path for the LLM extraction prompt — all fixed before any code landed. + +### What you can now do + +**Author typed metric claims in the `## Facts` fence.** The fence widens from 10 to 14 columns when a row carries `claim_metric`, `claim_value`, `claim_unit`, `claim_period`. Mixed fences (some typed rows, some not) work fine — the renderer stays at 10 cells when every row's typed fields are empty, so existing brains don't see diff churn. Metric labels normalize to lowercase snake_case (`MRR` → `mrr`, `Monthly Recurring Revenue` → `mrr`) so trajectory queries don't fragment across capitalization variants. Fifteen common founder metrics have canonical names in the seed map; unknown labels lowercase + underscore-collapse and pass through. + +**Get chronological metric trajectories on any entity.** `gbrain eval trajectory companies/acme-example` prints a sorted history with auto-detected regressions flagged inline. `--metric mrr` filters to a single metric. `--since 2026-01-01 --until 2026-07-31` narrows the window. `--json` returns the stable `schema_version: 1` envelope `{points, regressions, drift_score, schema_version}`. Regression threshold is 10% by default, override via `GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD`. The drift_score is `1 - mean(cosine(emb[i], emb[i-1]))` over the trajectory's existing embeddings: 0 means narrative stable, 1 means every claim is unrelated; null when fewer than 3 typed points have embeddings. + +**Roll up an entity into a founder scorecard.** `gbrain founder scorecard companies/acme-example` produces four signals: claim_accuracy (over resolved takes), consistency (1 minus the fraction of metric value-changes), growth_trajectory (direction + delta per metric), and red_flags (regressions + high drift + missed predictions). `--json` for a stable contract that any downstream system can consume. Default window is the last year; `--since` / `--until` to narrow. + +**Reach trajectories via the MCP `find_trajectory` op** — read scope, not localOnly. OAuth clients query other agents' brains directly. Visibility filtering matches `recall`'s posture: remote callers see only `visibility='world'` facts; local CLI sees everything. Source-scoped via the existing `sourceScopeOpts` v0.34.1.0 pattern — federated `allowedSources` clients get the union; scalar `sourceId` clients see their one source. + +**Run the dream cycle without poisoning trajectory data.** v0.35.7 fixes a pre-existing cycle idempotency bug: before this wave, running the full cycle twice in a row would append duplicate takes via `MAX(row_num)+1` after `extract_facts` wiped `consolidated_at` on every fact. The fix is a semantic upsert keyed on `(page_id, claim, since_date)` so re-promotion of a stable cluster reuses the existing take. The autopilot can now run on a cron without silently doubling your facts table's promotion count. + +**Cycle-inserted facts now arrive with embeddings.** Before this wave, the `extract_facts` cycle phase inserted fence-derived facts with `embedding = NULL`, which broke consolidate's cosine clustering and any downstream embedding-dependent feature (drift score, semantic similarity in recall). v0.35.7 batches `gateway.embed()` after fence parse and threads embeddings into `insertFacts`. ~$0.02 per 1K facts at OpenAI 3-large pricing; falls open when the embedding gateway is unavailable. + +**Fact valid_from now reflects claim dates, not import timestamps.** Eng review caught that `extractFactsFromFenceText` returned `valid_from: undefined` when a fence row lacked an explicit `validFrom:`, which fell through to `insertFacts`'s `now()` default. So a meeting page dated 2026-04-28 used to land its facts as claimed-on-today instead of claimed-on-the-meeting-date — and every trajectory query against existing fences showed import dates instead of claim dates. The fix threads `pages.effective_date` (v0.29.1+) as the fallback. Precedence chain: explicit fence `validFrom:` > `pages.effective_date` > `now()`. + +### Itemized changes + +- **Schema migration v67** (`facts_typed_claim_columns`) adds four optional columns to `facts`: `claim_metric TEXT`, `claim_value DOUBLE PRECISION`, `claim_unit TEXT`, `claim_period TEXT`. Plus partial index `facts_typed_claim_idx ON facts (entity_slug, claim_metric, valid_from) WHERE claim_metric IS NOT NULL`. All nullable; metadata-only on both engines. +- `ParsedFact` (`src/core/facts-fence.ts`) gains optional `claimMetric`, `claimValue`, `claimUnit`, `claimPeriod`. Parser tolerates both 10-cell (legacy) and 14-cell (widened) row shapes. Renderer emits 14 cells iff any row has a non-undefined typed field; otherwise stays at 10-cell for backward compat. Numeric `claimValue` cell tolerates comma thousand separators (`50,000` → `50000`). +- `extractFactsFromFenceText` (`src/core/facts/extract-from-fence.ts`) gains optional `pageEffectiveDate` in opts. Three-branch precedence: fence row > pageEffectiveDate > undefined (engine defaults to `now()`). Metric normalization via `normalizeMetricLabel` with a 15-entry seed map for common founder metrics. +- `src/core/facts/extract.ts` (the MCP put_page / backstop Haiku call site, NOT the cycle phase — Codex F2 fix) extends its system prompt to optionally emit `metric/value/unit/period` for metric-shaped claims. Backward compatible — non-metric claims emit nulls. +- `NewFact` interface (`src/core/engine.ts`) gains `claim_metric`, `claim_value`, `claim_unit`, `claim_period` (all nullable). Postgres + PGLite `insertFact` / `insertFacts` SQL extended. +- `BrainEngine.findTrajectory` method added to both engines. Source-scoped (`sourceId` scalar fast path + `sourceIds` federated array), visibility-filtered when `remote=true`. Single SQL query, `ORDER BY valid_from ASC, id ASC` for deterministic ordering (R3 pin). +- `src/core/trajectory.ts` (new file) — pure functions `detectRegressions`, `computeDriftScore`, `computeTrajectoryStats`. Exported `TRAJECTORY_SCHEMA_VERSION = 1`. Threshold knob via `GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD` env var. +- `find_trajectory` MCP op (`src/core/operations.ts`) — scope: read, not localOnly. Routes through `sourceScopeOpts(ctx)` + threads `ctx.remote` for visibility filtering. Strips raw `Float32Array` embeddings from the wire response. +- `gbrain eval trajectory ` CLI (`src/commands/eval-trajectory.ts`). Pure data fn + JSON formatter + human formatter + thin-client routing seam. +- `gbrain founder scorecard ` CLI (`src/commands/founder-scorecard.ts`). Pure aggregation function `computeFounderScorecard` exported for tests. Thin-client routing falls back to trajectory-only when remote. +- `consolidate` cycle phase (`src/core/cycle/phases/consolidate.ts`) gains semantic upsert keyed on `(page_id, claim, since_date)` + chronological `valid_until` writeback on each cluster. Re-running the full cycle on stable input produces zero new takes (R7 pin) and zero `valid_until` diffs (idempotent). +- `extract_facts` cycle phase (`src/core/cycle/extract-facts.ts`) batch-embeds via `gateway.embed()` before `insertFacts`. Threads `page.effective_date` as the `pageEffectiveDate` fallback. +- New tests: `test/facts-fence-typed.test.ts` (17 cases — round-trip, normalization, valid_from precedence), `test/consolidate-valid-until.test.ts` (4 cases — R4a chronological writeback, R4b/R7 cycle idempotency), `test/engine-find-trajectory.test.ts` (18 cases — engine + trajectory.ts pure functions), `test/operations-find-trajectory.test.ts` (9 cases — MCP op shape + visibility + source scoping), `test/eval-trajectory.test.ts` (7 cases — CLI), `test/founder-scorecard.test.ts` (9 cases — rollup math), `test/eval-contradictions/no-valid-until-write.test.ts` (4 cases — R1+R8 grep guards). +- Migration v67 test added to `test/migrate.test.ts` (6 cases — shape, source-shape, idempotency, runtime materialization, backward compat with nullable columns). + +## To take advantage of v0.35.7.0 + +`gbrain upgrade` should do this automatically. The wave ships migration v67 (`facts_typed_claim_columns`); `apply-migrations` runs it transparently. + +1. **Verify the migration applied:** + ```bash + gbrain doctor --json + ``` + Look for `schema_version: 67`. If it's still lower, run `gbrain apply-migrations --yes`. +2. **Try the new commands on an existing entity:** + ```bash + gbrain eval trajectory + gbrain founder scorecard + ``` + Without any typed-claim data yet, `eval trajectory` will say "(no typed claims for this entity in the window)". That's normal — the substrate is now ready; the data lands as you author typed fence rows or as the next dream cycle's Haiku extraction populates them on conversation-derived facts. +3. **Author a typed-claim fence row (optional):** + In any entity's `## Facts` fence, extend a row with typed columns: + ```markdown + | # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | claim_metric | claim_value | claim_unit | claim_period | + |---|-------|------|------------|------------|------------|------------|-------------|--------|---------|--------------|-------------|------------|--------------| + | 1 | MRR hit $50K | fact | 1.0 | private | high | 2026-01-15 | | OH transcript | | mrr | 50000 | USD | monthly | + ``` + On next `gbrain sync` + dream cycle, the fact lands with typed fields. `gbrain eval trajectory` picks it up immediately. +4. **(Optional) Tighten the regression threshold:** + ```bash + export GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD=0.05 # fire on 5%+ drops instead of 10% + ``` +5. **If any step fails or the numbers look wrong,** please file an issue: + https://github.com/garrytan/gbrain/issues with: + - output of `gbrain doctor` + - contents of `~/.gbrain/upgrade-errors.jsonl` if it exists + - which step broke + ## [0.35.6.0] - 2026-05-17 **Your search results stop letting weak pages climb to the top just because they have a lot of links pointing at them.** Off by default; turn it on with one config key. diff --git a/CLAUDE.md b/CLAUDE.md index 7c33c5c30..975ab0d99 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -84,6 +84,7 @@ strict behavior when unset. - `src/commands/eval-export.ts` (v0.25.0) — streams `eval_candidates` rows as NDJSON to stdout with `schema_version: 1` prefix on every line. EPIPE-safe, progress heartbeats on stderr, stable id-desc tiebreaker so `--since` windows never dupe/miss rows. - `src/commands/eval-prune.ts` (v0.25.0) — explicit retention cleanup. Requires `--older-than DUR`. `--dry-run` reports would-delete count. - `src/commands/eval-replay.ts` (v0.25.0) — contributor-facing replay tool. Reads NDJSON from `gbrain eval export`, re-runs each captured `query` / `search` op against the current brain, computes set-Jaccard@k between captured + current `retrieved_slugs`, top-1 stability rate, and latency Δ. Stable JSON shape (`schema_version: 1`) for CI gating; human mode prints a regression table. Pure Bun, zero new deps. The dev-loop half of BrainBench-Real that closes the gap between "data captured" and "data used to gate a PR." See `docs/eval-bench.md` for the workflow. +- `src/commands/eval-trajectory.ts` + `src/commands/founder-scorecard.ts` + `src/core/trajectory.ts` (v0.35.7) — temporal trajectory + founder scorecard. The wave that turns the v0.35.3.1 date-aware contradiction probe into a useful temporal substrate. `gbrain eval trajectory ` shows the chronological typed-claim history (mrr/arr/team_size/etc) with regressions auto-flagged inline; `gbrain founder scorecard ` rolls up claim_accuracy / consistency / growth_trajectory / red_flags into one JSON. Pure-function math lives in `trajectory.ts`: `detectRegressions(points, threshold)` walks consecutive metric-value pairs per metric (10% drop default, env override `GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD`); `computeDriftScore(points)` returns `1 - mean(cosine(emb[i], emb[i-1]))` over existing embeddings (null when <3 embedded points). Backed by `BrainEngine.findTrajectory(opts)` — both Postgres and PGLite, single SQL query, deterministic `ORDER BY valid_from ASC, id ASC` (R3). Source-scoped via the v0.34.1.0 `sourceId` scalar / `sourceIds` array dual pattern (D-CDX-6); visibility-filtered for remote callers (D-CDX-1) — `recall`-equivalent posture. MCP op `find_trajectory` (read scope, NOT localOnly) registered after `find_experts`. Migration v67 adds four optional typed-claim columns (`claim_metric`, `claim_value`, `claim_unit`, `claim_period`) + a partial index on `(entity_slug, claim_metric, valid_from) WHERE claim_metric IS NOT NULL`. Fence widens from 10 to 14 cells when any row has typed data; renderer stays at 10 cells when none do (no churn diff on existing fences). Metric labels normalize to lowercase snake_case via `normalizeMetricLabel` (15-entry seed map for common founder metrics). The `consolidate` cycle phase gains semantic upsert keyed on `(page_id, claim, since_date)` — fixes the pre-existing F4 duplicate-takes bug where re-running the full cycle after `extract_facts` cleared `consolidated_at` would silently append duplicate takes via `MAX(row_num)+1`. Also writes chronological `valid_until` on each cluster's older facts. The `extract_facts` cycle phase batch-embeds via `gateway.embed()` before insert AND threads `pages.effective_date` as the `pageEffectiveDate` fallback for `valid_from` (precedence chain: fence-row > pageEffectiveDate > now()). The contradiction probe MUST NOT write `valid_until` — R1+R8 grep guard at `test/eval-contradictions/no-valid-until-write.test.ts` pins this. Codex outside-voice round caught F1 (v66 collision → v67), F2 (Haiku lives in `facts/extract.ts` not `extract-facts.ts` cycle phase), F3 (cycle didn't embed before insert), F4 (idempotency bug), F5+F6 (missed `fence-write.ts` caller + no Page object there → pageEffectiveDate is OPTIONAL), F7 (privacy regression — visibility filter added), F8 (ParsedFact needed typed-field extension for markdown system-of-record), F9 (dual scalar+federated sourceId). Plan: `~/.claude/plans/system-instruction-you-are-working-curious-jellyfish.md`. Tests: 258 across 12 files. - `src/commands/eval-suspected-contradictions.ts` + `src/core/eval-contradictions/{judge,runner,types,date-filter,cost-tracker,cache,severity-classify,cross-source,trends,calibration,judge-errors,auto-supersession,fixture-redact}.ts` (v0.32.6) — `gbrain eval suspected-contradictions [run|trend|review]`. Probe samples top-K retrieval pairs per query (cross-slug + intra-page chunk-vs-take), date pre-filters (3-rule layered — same-paragraph-dual-date overrides separation rule), LLM judge (query-conditioned per Codex; UTF-8-safe truncation; C1 confidence-floor double-enforcement; resolution_kind output drives M7 paste-ready commands), persistent cache keyed on `(chunk_a_hash, chunk_b_hash, model_id, prompt_version, truncation_policy)` (Codex outside-voice fix — prompt edits cleanly invalidate prior verdicts), Wilson 95% CI calibration on the headline percentage with `small_sample_note` when n<30, judge_errors as first-class typed counters (parse_fail/refusal/timeout/http_5xx/unknown — Codex fix to bias from silent skip), M5 trend writes to `eval_contradictions_runs`, M6 source-tier breakdown reuses `DEFAULT_SOURCE_BOOSTS` prefix logic, deterministic sampling (combined_score DESC + lex tiebreaker — stable cache hit-rate across re-runs). Hermetic via `judgeFn` + `searchFn` DI in the runner; never touches the real gateway in tests. Engine surface: `BrainEngine.listActiveTakesForPages` (P1 batched), `writeContradictionsRun` + `loadContradictionsTrend` (M5), `getContradictionCacheEntry` + `putContradictionCacheEntry` + `sweepContradictionCache` (P2). Schema migrations v51 + v52. MCP op `find_contradictions` (read scope, NOT localOnly, NOT in subagent allowlist — user-initiated only). M1 doctor check surfaces high-severity findings with paste-ready resolution commands. M2 synthesize phase pre-fetches latest probe's top-5-by-severity findings and threads them into `buildSynthesisPrompt` as an informational block. 226 hermetic unit tests + 12 real-Postgres E2E. Plan: `~/.claude/plans/system-instruction-you-are-working-hashed-dewdrop.md`. Architecture doc: `docs/contradictions.md`. - `src/core/think/index.ts` (v0.35.5.0 — gateway adapter) — `runThink` no longer instantiates `new Anthropic()` directly. The internal `LLMClient` instance is now built by a small adapter that wraps `gateway.chat()` from `src/core/ai/gateway.ts`, the canonical AI seam v0.31.12 established for chat/embed/expansion. Closes #952: stdio MCP launches (Claude Desktop, Cursor) don't inherit shell env, so the Anthropic SDK's env-only key resolution lost the key any user had set via `gbrain config set anthropic_api_key`. The gateway reads from `~/.gbrain/config.json` AND from env, so both paths work. Test seam preserved: `opts.client?: ThinkLLMClient` injection still works for the 12+ existing tests (`test/think-pipeline.serial.test.ts`, `test/think-gateway-adapter.test.ts`, etc.); `opts.stubResponse` continues to short-circuit before any LLM call. When neither key nor client is available, the graceful "no LLM available" stub still fires with the same `NO_ANTHROPIC_API_KEY` warning. v0.36.x TODO: drop `ThinkLLMClient` indirection entirely, migrate tests to `__setChatTransportForTests` seam from `src/core/ai/gateway.ts`. - `src/core/operations.ts` extension (v0.35.5.0 orphans fix) — `findOrphanPages` (both engines) now filters `p.deleted_at IS NULL` on the candidate side AND adds `JOIN pages src ON src.id = l.from_page_id WHERE src.deleted_at IS NULL` to the EXISTS subquery on the link-source side. Pre-v0.35.5 the query filtered nothing on `deleted_at`, so soft-deleted pages (v0.26.5 soft-delete shipped without updating this query) appeared as orphans AND links from soft-deleted source pages still suppressed live pages from orphan results. Closes #1021. Pinned by `test/orphans.test.ts`'s soft-delete cases. diff --git a/README.md b/README.md index 482c9bd69..591634fd8 100644 --- a/README.md +++ b/README.md @@ -12,6 +12,8 @@ GBrain is those patterns, generalized. 34 skills. Install in 30 minutes. Your ag **New in v0.28.8 — LongMemEval in the box:** `gbrain eval longmemeval ` runs the public [LongMemEval](https://huggingface.co/datasets/xiaowu0162/longmemeval) benchmark against gbrain's hybrid retrieval. One in-memory PGLite per run, `TRUNCATE` between questions (runtime-enumerated tables, schema-migration-safe), 25.9ms p50 per question on Apple Silicon. Your `~/.gbrain` brain is never touched. Retrieved chat content is sanitized with the same `INJECTION_PATTERNS` that protect takes — one source of truth for prompt-injection defense. Hand the JSONL output to LongMemEval's `evaluate_qa.py` to score. +**New in v0.35.7 — Temporal trajectory + founder scorecard.** Author typed metric assertions in the `## Facts` fence (`mrr=50000`, `arr=2000000`, `team_size=12`) and gbrain stores them as first-class typed columns. `gbrain eval trajectory companies/acme-example` prints the chronological history with regressions auto-flagged inline. `gbrain founder scorecard companies/acme-example` rolls up claim accuracy, consistency, growth direction, and red flags into a stable `schema_version: 1` JSON contract. New MCP op `find_trajectory` exposes the same data to agents (read scope, visibility-filtered for remote callers). The `consolidate` cycle phase now writes `valid_until` on chronologically-superseded facts AND uses semantic upsert on `(page_id, claim, since_date)` — re-running the dream cycle on stable input is now a true no-op (fixed a pre-existing duplicate-takes bug from prior versions). + > **~30 minutes to a fully working brain.** Database ready in 2 seconds (PGLite, no server). You just answer questions about API keys. > **LLMs:** fetch [`llms.txt`](llms.txt) for the documentation map, or [`llms-full.txt`](llms-full.txt) for the same map with core docs inlined in one fetch. **Agents:** start with [`AGENTS.md`](AGENTS.md) (or [`CLAUDE.md`](CLAUDE.md) if you're Claude Code). @@ -770,6 +772,21 @@ EVAL gbrain eval longmemeval Run public LongMemEval against gbrain hybrid retrieval (v0.28.8) [--limit N] [--retrieval-only] [--keyword-only] [--expansion] [--top-k K] [--model M] [--output FILE] + gbrain eval suspected-contradictions Cached contradiction probe (v0.32.6, 6-verdict enum v0.35.3.1) + [run|trend|review] [--severity S] [--budget-usd N] + gbrain eval trajectory Chronological typed-claim trajectory + regressions (v0.35.7) + [--metric M] [--since YYYY-MM-DD] [--until YYYY-MM-DD] + [--limit N] [--json] + +TEMPORAL (v0.35.7) + gbrain founder scorecard Roll up an entity's typed claims + resolved-take outcomes into + four signals: claim_accuracy / consistency / growth_trajectory / + red_flags. Stable JSON contract (schema_version: 1). + [--since YYYY-MM-DD] [--until YYYY-MM-DD] [--json] + Backed by the find_trajectory MCP op (read scope, federated + source-scoped, visibility-filtered for remote callers). + Override regression threshold: + export GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD=0.05 ADMIN gbrain doctor [--json] [--fast] Health checks (resolver, skills, DB, embeddings) diff --git a/VERSION b/VERSION index 54246e16e..36f577733 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.35.6.0 \ No newline at end of file +0.35.7.0 diff --git a/docs/contradictions.md b/docs/contradictions.md index 79f490981..af02d7771 100644 --- a/docs/contradictions.md +++ b/docs/contradictions.md @@ -148,3 +148,19 @@ pay near-zero on re-runs (until you bump PROMPT_VERSION). - CHANGELOG: `## [0.32.6]` entry covers the whole release. - Cost discipline: `docs/eval-bench.md` for the recommended nightly cadence + trend-tracking workflow. +- **Temporal axis follow-on (v0.35.3.1 + v0.35.7):** v0.35.3.1 added a + six-member verdict enum (`no_contradiction | contradiction | + temporal_supersession | temporal_regression | temporal_evolution | + negation_artifact`) and threaded `pages.effective_date` into the judge + prompt so the probe stops crying wolf on legitimate change-over-time. + v0.35.7 lands the trajectory substrate the probe pointed at: + `gbrain eval trajectory ` shows the chronological typed-claim + history with regressions flagged inline; `gbrain founder scorecard + ` rolls up four signals (accuracy, consistency, growth + direction, red flags) into a stable JSON contract. MCP op + `find_trajectory` (read scope, visibility-filtered for remote callers) + exposes the same data to agents. The probe's `temporal_supersession` + verdict and the consolidate phase's `valid_until` writeback both + preserve the `auto-supersession.ts:4` "NEVER auto-applies" invariant + — the probe still emits paste-ready commands, only `consolidate` + writes `valid_until` (R1+R8 grep guard pins this). diff --git a/llms-full.txt b/llms-full.txt index a709c4f0a..392cd55e0 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -72,6 +72,14 @@ writing or reviewing an operation, consult `src/core/operations.ts` for the cont ` (v0.28.8) runs against an isolated in-memory PGLite per question — your `~/.gbrain` is never opened. Full guide: [`docs/eval-bench.md`](./docs/eval-bench.md). +- **Track a founder/company over time (v0.35.7):** when an entity has + typed metric claims in its `## Facts` fence (`metric: mrr`, `value: 50000`, + `unit: USD`, `period: monthly` columns), run + `gbrain eval trajectory ` for the chronological history + with regressions auto-flagged, or `gbrain founder scorecard ` + for a four-signal JSON rollup (claim_accuracy / consistency / + growth_trajectory / red_flags). MCP op `find_trajectory` exposes the + same data — read scope, visibility-filtered for remote callers. - **Everything else:** [`./llms.txt`](./llms.txt) is the full documentation map. [`./llms-full.txt`](./llms-full.txt) is the same map with core docs inlined for single-fetch ingestion. @@ -192,6 +200,7 @@ strict behavior when unset. - `src/commands/eval-export.ts` (v0.25.0) — streams `eval_candidates` rows as NDJSON to stdout with `schema_version: 1` prefix on every line. EPIPE-safe, progress heartbeats on stderr, stable id-desc tiebreaker so `--since` windows never dupe/miss rows. - `src/commands/eval-prune.ts` (v0.25.0) — explicit retention cleanup. Requires `--older-than DUR`. `--dry-run` reports would-delete count. - `src/commands/eval-replay.ts` (v0.25.0) — contributor-facing replay tool. Reads NDJSON from `gbrain eval export`, re-runs each captured `query` / `search` op against the current brain, computes set-Jaccard@k between captured + current `retrieved_slugs`, top-1 stability rate, and latency Δ. Stable JSON shape (`schema_version: 1`) for CI gating; human mode prints a regression table. Pure Bun, zero new deps. The dev-loop half of BrainBench-Real that closes the gap between "data captured" and "data used to gate a PR." See `docs/eval-bench.md` for the workflow. +- `src/commands/eval-trajectory.ts` + `src/commands/founder-scorecard.ts` + `src/core/trajectory.ts` (v0.35.7) — temporal trajectory + founder scorecard. The wave that turns the v0.35.3.1 date-aware contradiction probe into a useful temporal substrate. `gbrain eval trajectory ` shows the chronological typed-claim history (mrr/arr/team_size/etc) with regressions auto-flagged inline; `gbrain founder scorecard ` rolls up claim_accuracy / consistency / growth_trajectory / red_flags into one JSON. Pure-function math lives in `trajectory.ts`: `detectRegressions(points, threshold)` walks consecutive metric-value pairs per metric (10% drop default, env override `GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD`); `computeDriftScore(points)` returns `1 - mean(cosine(emb[i], emb[i-1]))` over existing embeddings (null when <3 embedded points). Backed by `BrainEngine.findTrajectory(opts)` — both Postgres and PGLite, single SQL query, deterministic `ORDER BY valid_from ASC, id ASC` (R3). Source-scoped via the v0.34.1.0 `sourceId` scalar / `sourceIds` array dual pattern (D-CDX-6); visibility-filtered for remote callers (D-CDX-1) — `recall`-equivalent posture. MCP op `find_trajectory` (read scope, NOT localOnly) registered after `find_experts`. Migration v67 adds four optional typed-claim columns (`claim_metric`, `claim_value`, `claim_unit`, `claim_period`) + a partial index on `(entity_slug, claim_metric, valid_from) WHERE claim_metric IS NOT NULL`. Fence widens from 10 to 14 cells when any row has typed data; renderer stays at 10 cells when none do (no churn diff on existing fences). Metric labels normalize to lowercase snake_case via `normalizeMetricLabel` (15-entry seed map for common founder metrics). The `consolidate` cycle phase gains semantic upsert keyed on `(page_id, claim, since_date)` — fixes the pre-existing F4 duplicate-takes bug where re-running the full cycle after `extract_facts` cleared `consolidated_at` would silently append duplicate takes via `MAX(row_num)+1`. Also writes chronological `valid_until` on each cluster's older facts. The `extract_facts` cycle phase batch-embeds via `gateway.embed()` before insert AND threads `pages.effective_date` as the `pageEffectiveDate` fallback for `valid_from` (precedence chain: fence-row > pageEffectiveDate > now()). The contradiction probe MUST NOT write `valid_until` — R1+R8 grep guard at `test/eval-contradictions/no-valid-until-write.test.ts` pins this. Codex outside-voice round caught F1 (v66 collision → v67), F2 (Haiku lives in `facts/extract.ts` not `extract-facts.ts` cycle phase), F3 (cycle didn't embed before insert), F4 (idempotency bug), F5+F6 (missed `fence-write.ts` caller + no Page object there → pageEffectiveDate is OPTIONAL), F7 (privacy regression — visibility filter added), F8 (ParsedFact needed typed-field extension for markdown system-of-record), F9 (dual scalar+federated sourceId). Plan: `~/.claude/plans/system-instruction-you-are-working-curious-jellyfish.md`. Tests: 258 across 12 files. - `src/commands/eval-suspected-contradictions.ts` + `src/core/eval-contradictions/{judge,runner,types,date-filter,cost-tracker,cache,severity-classify,cross-source,trends,calibration,judge-errors,auto-supersession,fixture-redact}.ts` (v0.32.6) — `gbrain eval suspected-contradictions [run|trend|review]`. Probe samples top-K retrieval pairs per query (cross-slug + intra-page chunk-vs-take), date pre-filters (3-rule layered — same-paragraph-dual-date overrides separation rule), LLM judge (query-conditioned per Codex; UTF-8-safe truncation; C1 confidence-floor double-enforcement; resolution_kind output drives M7 paste-ready commands), persistent cache keyed on `(chunk_a_hash, chunk_b_hash, model_id, prompt_version, truncation_policy)` (Codex outside-voice fix — prompt edits cleanly invalidate prior verdicts), Wilson 95% CI calibration on the headline percentage with `small_sample_note` when n<30, judge_errors as first-class typed counters (parse_fail/refusal/timeout/http_5xx/unknown — Codex fix to bias from silent skip), M5 trend writes to `eval_contradictions_runs`, M6 source-tier breakdown reuses `DEFAULT_SOURCE_BOOSTS` prefix logic, deterministic sampling (combined_score DESC + lex tiebreaker — stable cache hit-rate across re-runs). Hermetic via `judgeFn` + `searchFn` DI in the runner; never touches the real gateway in tests. Engine surface: `BrainEngine.listActiveTakesForPages` (P1 batched), `writeContradictionsRun` + `loadContradictionsTrend` (M5), `getContradictionCacheEntry` + `putContradictionCacheEntry` + `sweepContradictionCache` (P2). Schema migrations v51 + v52. MCP op `find_contradictions` (read scope, NOT localOnly, NOT in subagent allowlist — user-initiated only). M1 doctor check surfaces high-severity findings with paste-ready resolution commands. M2 synthesize phase pre-fetches latest probe's top-5-by-severity findings and threads them into `buildSynthesisPrompt` as an informational block. 226 hermetic unit tests + 12 real-Postgres E2E. Plan: `~/.claude/plans/system-instruction-you-are-working-hashed-dewdrop.md`. Architecture doc: `docs/contradictions.md`. - `src/core/think/index.ts` (v0.35.5.0 — gateway adapter) — `runThink` no longer instantiates `new Anthropic()` directly. The internal `LLMClient` instance is now built by a small adapter that wraps `gateway.chat()` from `src/core/ai/gateway.ts`, the canonical AI seam v0.31.12 established for chat/embed/expansion. Closes #952: stdio MCP launches (Claude Desktop, Cursor) don't inherit shell env, so the Anthropic SDK's env-only key resolution lost the key any user had set via `gbrain config set anthropic_api_key`. The gateway reads from `~/.gbrain/config.json` AND from env, so both paths work. Test seam preserved: `opts.client?: ThinkLLMClient` injection still works for the 12+ existing tests (`test/think-pipeline.serial.test.ts`, `test/think-gateway-adapter.test.ts`, etc.); `opts.stubResponse` continues to short-circuit before any LLM call. When neither key nor client is available, the graceful "no LLM available" stub still fires with the same `NO_ANTHROPIC_API_KEY` warning. v0.36.x TODO: drop `ThinkLLMClient` indirection entirely, migrate tests to `__setChatTransportForTests` seam from `src/core/ai/gateway.ts`. - `src/core/operations.ts` extension (v0.35.5.0 orphans fix) — `findOrphanPages` (both engines) now filters `p.deleted_at IS NULL` on the candidate side AND adds `JOIN pages src ON src.id = l.from_page_id WHERE src.deleted_at IS NULL` to the EXISTS subquery on the link-source side. Pre-v0.35.5 the query filtered nothing on `deleted_at`, so soft-deleted pages (v0.26.5 soft-delete shipped without updating this query) appeared as orphans AND links from soft-deleted source pages still suppressed live pages from orphan results. Closes #1021. Pinned by `test/orphans.test.ts`'s soft-delete cases. @@ -2134,6 +2143,8 @@ GBrain is those patterns, generalized. 34 skills. Install in 30 minutes. Your ag **New in v0.28.8 — LongMemEval in the box:** `gbrain eval longmemeval ` runs the public [LongMemEval](https://huggingface.co/datasets/xiaowu0162/longmemeval) benchmark against gbrain's hybrid retrieval. One in-memory PGLite per run, `TRUNCATE` between questions (runtime-enumerated tables, schema-migration-safe), 25.9ms p50 per question on Apple Silicon. Your `~/.gbrain` brain is never touched. Retrieved chat content is sanitized with the same `INJECTION_PATTERNS` that protect takes — one source of truth for prompt-injection defense. Hand the JSONL output to LongMemEval's `evaluate_qa.py` to score. +**New in v0.35.7 — Temporal trajectory + founder scorecard.** Author typed metric assertions in the `## Facts` fence (`mrr=50000`, `arr=2000000`, `team_size=12`) and gbrain stores them as first-class typed columns. `gbrain eval trajectory companies/acme-example` prints the chronological history with regressions auto-flagged inline. `gbrain founder scorecard companies/acme-example` rolls up claim accuracy, consistency, growth direction, and red flags into a stable `schema_version: 1` JSON contract. New MCP op `find_trajectory` exposes the same data to agents (read scope, visibility-filtered for remote callers). The `consolidate` cycle phase now writes `valid_until` on chronologically-superseded facts AND uses semantic upsert on `(page_id, claim, since_date)` — re-running the dream cycle on stable input is now a true no-op (fixed a pre-existing duplicate-takes bug from prior versions). + > **~30 minutes to a fully working brain.** Database ready in 2 seconds (PGLite, no server). You just answer questions about API keys. > **LLMs:** fetch [`llms.txt`](llms.txt) for the documentation map, or [`llms-full.txt`](llms-full.txt) for the same map with core docs inlined in one fetch. **Agents:** start with [`AGENTS.md`](AGENTS.md) (or [`CLAUDE.md`](CLAUDE.md) if you're Claude Code). @@ -2892,6 +2903,21 @@ EVAL gbrain eval longmemeval Run public LongMemEval against gbrain hybrid retrieval (v0.28.8) [--limit N] [--retrieval-only] [--keyword-only] [--expansion] [--top-k K] [--model M] [--output FILE] + gbrain eval suspected-contradictions Cached contradiction probe (v0.32.6, 6-verdict enum v0.35.3.1) + [run|trend|review] [--severity S] [--budget-usd N] + gbrain eval trajectory Chronological typed-claim trajectory + regressions (v0.35.7) + [--metric M] [--since YYYY-MM-DD] [--until YYYY-MM-DD] + [--limit N] [--json] + +TEMPORAL (v0.35.7) + gbrain founder scorecard Roll up an entity's typed claims + resolved-take outcomes into + four signals: claim_accuracy / consistency / growth_trajectory / + red_flags. Stable JSON contract (schema_version: 1). + [--since YYYY-MM-DD] [--until YYYY-MM-DD] [--json] + Backed by the find_trajectory MCP op (read scope, federated + source-scoped, visibility-filtered for remote callers). + Override regression threshold: + export GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD=0.05 ADMIN gbrain doctor [--json] [--fast] Health checks (resolver, skills, DB, embeddings) diff --git a/package.json b/package.json index 11380f93c..1351f94b8 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gbrain", - "version": "0.35.6.0", + "version": "0.35.7.0", "description": "Postgres-native personal knowledge brain with hybrid RAG search", "type": "module", "main": "src/core/index.ts", diff --git a/skills/migrations/v0.35.7.0.md b/skills/migrations/v0.35.7.0.md new file mode 100644 index 000000000..7c4522617 --- /dev/null +++ b/skills/migrations/v0.35.7.0.md @@ -0,0 +1,109 @@ +--- +version: v0.35.7.0 +date: 2026-05-17 +feature_pitch: > + Typed metric claims, chronological trajectory queries, and a founder + scorecard CLI. The temporal axis the v0.35.3.1 contradiction probe + needed in order to be useful, finally landed. +--- + +# v0.35.7.0 — Temporal trajectory + founder scorecard + +## What changed + +- **Migration v67** (`facts_typed_claim_columns`) adds four optional + columns to `facts`: `claim_metric`, `claim_value`, `claim_unit`, + `claim_period`. All nullable — existing facts persist identically. +- The `## Facts` fence widens from 10 to 14 columns when a row carries + typed-claim data. Backward compat: fences without typed data stay at + 10 cells. +- New MCP op: `find_trajectory` (read scope, not localOnly). Federated + source-scoped, visibility-filtered for remote callers. +- New CLI commands: `gbrain eval trajectory ` and + `gbrain founder scorecard `. +- The `extract_facts` cycle phase now batch-embeds facts before insert + AND threads `pages.effective_date` as the fallback `valid_from`. +- The `consolidate` cycle phase now uses semantic upsert on `(page_id, + claim, since_date)` so re-running the full cycle on stable input + produces zero duplicate takes. Also writes chronological `valid_until` + on each cluster's older facts. + +## What agents need to do + +Most of this is automatic on upgrade. Three things worth knowing: + +### 1. Author typed-claim rows in `## Facts` fences + +When you write facts about a company or person that include metric +assertions, add the four typed columns to the fence row: + +```markdown +| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | claim_metric | claim_value | claim_unit | claim_period | +|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|--------------|-------------|------------|--------------| +| 1 | MRR hit $50K (Jan 2026) | fact | 1.0 | private | high | 2026-01-15 | | OH transcript | | mrr | 50000 | USD | monthly | +| 2 | ARR target | commitment | 0.6 | private | high | 2026-01-15 | 2026-12-31 | OH transcript | predicted $1M ARR by Q4 | arr | 1000000 | USD | annual | +``` + +The `claim_metric` column normalizes to lowercase snake_case (mrr, arr, +team_size, runway, burn_rate, cac, ltv, mau, dau, churn_rate, revenue, +fundraise, headcount, gross_margin, users). Unknown labels lowercase + +underscore-collapse and pass through verbatim. + +### 2. When the user asks "how has X trended" + +Use `gbrain eval trajectory`: + +```bash +gbrain eval trajectory companies/acme-example +gbrain eval trajectory companies/acme-example --metric mrr +gbrain eval trajectory companies/acme-example --since 2026-01-01 --json +``` + +Or the MCP op `find_trajectory` with `{entity_slug, metric?, since?, +until?, limit?}`. + +Regressions are auto-flagged at 10% drops by default. Drift score +indicates narrative stability (0 = stable, 1 = every claim is unrelated; +null when <3 embedded points). + +### 3. When the user asks "is this founder consistent / reliable" + +Use `gbrain founder scorecard`: + +```bash +gbrain founder scorecard companies/acme-example --json +``` + +Returns four signals: `claim_accuracy` (over resolved takes), +`consistency` (1 minus the fraction of metric value-changes), +`growth_trajectory` (direction + delta per metric), `red_flags` +(regressions + high drift + missed predictions). + +JSON schema is `schema_version: 1`, additive-only across releases. + +## Iron-rule contracts to preserve + +- The contradiction probe MUST NOT write `valid_until` on facts. + `consolidate` writes it; the probe emits paste-ready commands instead. + This invariant is pinned by + `test/eval-contradictions/no-valid-until-write.test.ts` (R1 + R8 grep + guards). +- `find_trajectory` MUST visibility-filter remote callers (`ctx.remote === + true` → SQL adds `AND visibility = 'world'`). Mirrors `recall`'s + posture. +- `consolidate`'s semantic upsert MUST be keyed on `(page_id, claim, + since_date)`, NOT `MAX(row_num)+1`. Pinned by + `test/consolidate-valid-until.test.ts` (R4b/R7). + +## Where to look + +- Cycle phase: `src/core/cycle/extract-facts.ts`, + `src/core/cycle/phases/consolidate.ts` +- Engine method: `BrainEngine.findTrajectory` in + `src/core/engine.ts`, `src/core/postgres-engine.ts`, + `src/core/pglite-engine.ts` +- Trajectory math: `src/core/trajectory.ts` +- MCP op: `src/core/operations.ts` (search for `find_trajectory`) +- CLIs: `src/commands/eval-trajectory.ts`, + `src/commands/founder-scorecard.ts` +- Schema: `src/core/migrate.ts` (migration v67) diff --git a/src/cli.ts b/src/cli.ts index 32d2e5081..2671ec266 100755 --- a/src/cli.ts +++ b/src/cli.ts @@ -27,7 +27,7 @@ for (const op of operations) { } // CLI-only commands that bypass the operation layer -const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache']); +const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache', 'founder']); // CLI-only commands whose handlers print their own --help text. These are // excluded from the generic short-circuit so detailed per-command and // per-subcommand usage stays reachable. @@ -1165,6 +1165,15 @@ async function handleCliOnly(command: string, args: string[]) { await runTakes(engine, args); break; } + case 'founder': { + // v0.35.4 (T7) — founder scorecard. `gbrain founder scorecard ` + // rolls up Phase 2's typed-claim substrate into the four scorecard + // metrics (claim accuracy, consistency, growth trajectory, red flags). + // Thin-client routing handled inside the command file. + const { runFounder } = await import('./commands/founder-scorecard.ts'); + await runFounder(engine, args); + break; + } case 'think': { const { runThinkCli } = await import('./commands/think.ts'); await runThinkCli(engine, args); diff --git a/src/commands/eval-trajectory.ts b/src/commands/eval-trajectory.ts new file mode 100644 index 000000000..600fc82fd --- /dev/null +++ b/src/commands/eval-trajectory.ts @@ -0,0 +1,221 @@ +/** + * gbrain eval trajectory — chronological typed-claim + * trajectory + regression detection + narrative drift score. + * + * v0.35.4 (T6) — pure data fn + JSON formatter + human formatter + + * thin-client routing seam. Mirrors `gbrain salience` / `gbrain anomalies` + * shape so the four temporal-axis read CLIs feel consistent. + * + * Usage: + * gbrain eval trajectory companies/acme-example + * gbrain eval trajectory companies/acme-example --metric mrr + * gbrain eval trajectory companies/acme-example --since 2026-01-01 --until 2026-07-31 + * gbrain eval trajectory companies/acme-example --json + */ + +import type { BrainEngine } from '../core/engine.ts'; +import { loadConfig, isThinClient } from '../core/config.ts'; +import { callRemoteTool, unpackToolResult } from '../core/mcp-client.ts'; +import { + computeTrajectoryStats, + TRAJECTORY_SCHEMA_VERSION, + type TrajectoryRegression, +} from '../core/trajectory.ts'; + +interface RunOpts { + entitySlug: string; + metric?: string; + since?: string; + until?: string; + limit?: number; + json?: boolean; +} + +interface WireTrajectoryResult { + points: Array<{ + fact_id: number; + valid_from: string; + metric: string | null; + value: number | null; + unit: string | null; + period: string | null; + text: string; + source_session: string | null; + source_markdown_slug: string | null; + }>; + regressions: TrajectoryRegression[]; + drift_score: number | null; + schema_version: number; +} + +const HELP = `Usage: gbrain eval trajectory [options] + +Show the chronological claim trajectory for an entity (typed metric values +over time, plus regressions and narrative drift score). + +Examples: + gbrain eval trajectory companies/acme-example + gbrain eval trajectory companies/acme-example --metric mrr + gbrain eval trajectory companies/acme-example --since 2026-01-01 --until 2026-07-31 + gbrain eval trajectory companies/acme-example --json + +Options: + --metric M Filter to a single canonical metric (mrr, arr, team_size, …) + --since YYYY-MM-DD Lower bound on valid_from + --until YYYY-MM-DD Upper bound on valid_from + --limit N Max points (default 100, max 500) + --json JSON output for agents (stable schema_version: 1) + --help, -h Show this help +`; + +function parseArgs(args: string[]): RunOpts | { help: true } | { error: string } { + const opts: Partial = {}; + const positional: string[] = []; + for (let i = 0; i < args.length; i++) { + const a = args[i]; + if (a === '--help' || a === '-h') return { help: true }; + if (a === '--json') { opts.json = true; continue; } + if (a === '--metric') { opts.metric = args[++i]; continue; } + if (a === '--since') { opts.since = args[++i]; continue; } + if (a === '--until') { opts.until = args[++i]; continue; } + if (a === '--limit') { + const n = parseInt(args[++i] ?? '', 10); + if (Number.isFinite(n) && n > 0) opts.limit = n; + continue; + } + if (a.startsWith('-')) { + return { error: `Unknown flag: ${a}` }; + } + positional.push(a); + } + if (positional.length !== 1) { + return { error: 'Exactly one entity-slug positional argument is required.' }; + } + return { ...(opts as RunOpts), entitySlug: positional[0] }; +} + +export async function runEvalTrajectory(engine: BrainEngine, args: string[]): Promise { + const parsed = parseArgs(args); + if ('help' in parsed) { + console.log(HELP); + return; + } + if ('error' in parsed) { + console.error(parsed.error); + console.error(''); + console.error(HELP); + process.exit(1); + } + + let result: WireTrajectoryResult; + const cfg = loadConfig(); + if (isThinClient(cfg)) { + // Thin-client install: route through the remote find_trajectory MCP op. + const raw = await callRemoteTool(cfg!, 'find_trajectory', { + entity_slug: parsed.entitySlug, + metric: parsed.metric, + since: parsed.since, + until: parsed.until, + limit: parsed.limit, + }, { timeoutMs: 30_000 }); + result = unpackToolResult(raw); + } else { + // Local: call engine.findTrajectory directly, then compute derived + // metrics via trajectory.ts. ctx.remote is implicitly false here so + // visibility filtering is OFF — trusted local caller sees all facts. + const points = await engine.findTrajectory({ + entitySlug: parsed.entitySlug, + metric: parsed.metric, + since: parsed.since, + until: parsed.until, + limit: parsed.limit, + }); + const { regressions, drift_score } = computeTrajectoryStats(points); + result = { + points: points.map(p => ({ + fact_id: p.fact_id, + valid_from: p.valid_from.toISOString().slice(0, 10), + metric: p.metric, + value: p.value, + unit: p.unit, + period: p.period, + text: p.text, + source_session: p.source_session, + source_markdown_slug: p.source_markdown_slug, + })), + regressions, + drift_score, + schema_version: TRAJECTORY_SCHEMA_VERSION, + }; + } + + if (parsed.json) { + console.log(JSON.stringify(result, null, 2)); + return; + } + + // Human format. Title + per-point line + regression callouts + drift. + console.log(`Entity: ${parsed.entitySlug}`); + if (parsed.metric) console.log(`Metric: ${parsed.metric}`); + if (parsed.since || parsed.until) { + console.log(`Window: ${parsed.since ?? '(unbounded)'} → ${parsed.until ?? '(now)'}`); + } + console.log(''); + + if (result.points.length === 0) { + console.log('(no typed claims for this entity in the window)'); + return; + } + + // Build a regression lookup so each point's row can be annotated. + const regBy = new Map(); + for (const r of result.regressions) { + regBy.set(`${r.metric}|${r.to_date}|${r.to_value}`, r); + } + + for (const p of result.points) { + const metricCell = p.metric ?? '-'; + const valueCell = p.value === null ? '-' : formatValue(p.value, p.unit); + const sourceCell = p.source_session ?? p.source_markdown_slug ?? ''; + let line = ` ${p.valid_from} ${pad(metricCell, 14)} ${pad(valueCell, 12)} (${sourceCell})`; + const reg = p.metric && p.value !== null + ? regBy.get(`${p.metric}|${p.valid_from}|${p.value}`) + : undefined; + if (reg) { + const pct = Math.abs(reg.delta_pct * 100).toFixed(1); + line += ` [REGRESSION ↓${pct}%]`; + } + console.log(line); + } + + console.log(''); + if (result.drift_score === null) { + console.log('Drift score: (insufficient embedded points; need 3+)'); + } else { + const score = result.drift_score; + const tier = + score < 0.15 ? 'stable narrative' : + score < 0.35 ? 'mild drift' : + score < 0.6 ? 'moderate drift' : + 'narrative changing fast'; + console.log(`Drift score: ${score.toFixed(2)} (${tier})`); + } + if (result.regressions.length > 0) { + console.log(`Regressions detected: ${result.regressions.length}`); + } +} + +function pad(s: string, n: number): string { + return s.length >= n ? s : s + ' '.repeat(n - s.length); +} + +function formatValue(v: number, unit: string | null): string { + // Currency-style display for USD; plain for everything else. Keeps the + // output readable without locale assumptions. + if (unit === 'USD') { + if (Math.abs(v) >= 1_000_000) return `$${(v / 1_000_000).toFixed(1)}M`; + if (Math.abs(v) >= 1_000) return `$${(v / 1_000).toFixed(0)}K`; + return `$${v.toFixed(0)}`; + } + return String(v); +} diff --git a/src/commands/eval.ts b/src/commands/eval.ts index 3499c5f82..f24886741 100644 --- a/src/commands/eval.ts +++ b/src/commands/eval.ts @@ -67,6 +67,12 @@ export async function runEvalCommand(engine: BrainEngine, args: string[]): Promi const { runEvalSuspectedContradictions } = await import('./eval-suspected-contradictions.ts'); return runEvalSuspectedContradictions(engine, args.slice(1)); } + if (sub === 'trajectory') { + // v0.35.4 (T6) — chronological claim trajectory for an entity. Engine + // is connected; thin-client routing handled inside the command file. + const { runEvalTrajectory } = await import('./eval-trajectory.ts'); + return runEvalTrajectory(engine, args.slice(1)); + } // v0.32.3 search-lite — per-mode orchestrator + comparison report. if (sub === 'run-all') { const { runEvalRunAll } = await import('./eval-run-all.ts'); diff --git a/src/commands/founder-scorecard.ts b/src/commands/founder-scorecard.ts new file mode 100644 index 000000000..bb4ab3fed --- /dev/null +++ b/src/commands/founder-scorecard.ts @@ -0,0 +1,371 @@ +/** + * gbrain founder scorecard — rolls up Phase 2's typed-claim + * substrate + the takes outcome columns into a four-metric founder + * scorecard. + * + * v0.35.4 (T7). Pure aggregation over facts + takes + the trajectory + * derived metrics. Zero new schema. Zero LLM calls. + * + * Output (schema_version: 1, additive-only across releases per R5): + * + * { + * schema_version: 1, + * entity_slug: "companies/acme-example", + * window: { since, until }, + * claim_accuracy: { + * predicted: , + * accurate: , + * pct: , + * }, + * consistency: { + * score: <1 - (unique_metric_changes / typed_facts), clamped [0,1]>, + * metric_changes: , + * typed_facts: , + * }, + * growth_trajectory: [ + * { metric, direction: 'up'|'down'|'flat', latest_delta_pct }, + * ... + * ], + * red_flags: [ + * { kind: 'regression', metric, text }, + * { kind: 'narrative_drift', text }, + * { kind: 'missed_prediction', text }, + * ], + * } + * + * Usage: + * gbrain founder scorecard companies/acme-example + * gbrain founder scorecard companies/acme-example --since 2025-05-17 --until 2026-05-17 + * gbrain founder scorecard companies/acme-example --json + */ + +import type { BrainEngine, Take, TrajectoryPoint } from '../core/engine.ts'; +import { loadConfig, isThinClient } from '../core/config.ts'; +import { callRemoteTool, unpackToolResult } from '../core/mcp-client.ts'; +import { + computeTrajectoryStats, + TRAJECTORY_SCHEMA_VERSION, +} from '../core/trajectory.ts'; + +export interface FounderScorecard { + schema_version: number; + entity_slug: string; + window: { since: string | null; until: string | null }; + claim_accuracy: { + predicted: number; + accurate: number; + pct: number | null; + }; + consistency: { + score: number | null; + metric_changes: number; + typed_facts: number; + }; + growth_trajectory: Array<{ + metric: string; + direction: 'up' | 'down' | 'flat'; + latest_delta_pct: number; + }>; + red_flags: Array<{ + kind: 'regression' | 'narrative_drift' | 'missed_prediction'; + metric?: string; + text: string; + }>; +} + +/** + * Pure data function — given a sorted trajectory + takes window, compute + * the scorecard. Exported for tests so the rollup math is exercised + * without a DB round trip. + */ +export function computeFounderScorecard(args: { + entitySlug: string; + windowSince: string | null; + windowUntil: string | null; + points: TrajectoryPoint[]; + takes: Take[]; + driftThresholdRedFlag?: number; +}): FounderScorecard { + const driftRedFlag = args.driftThresholdRedFlag ?? 0.5; + const { regressions, drift_score } = computeTrajectoryStats(args.points); + + // claim_accuracy — over resolved takes only. + const resolved = args.takes.filter(t => t.resolved_outcome !== null); + const accurate = resolved.filter(t => t.resolved_outcome === true).length; + const accuracyPct = resolved.length > 0 ? accurate / resolved.length : null; + + // consistency — count consecutive value changes per metric. A 'change' + // is a pair where the relative delta is >=5%. The score normalizes + // by total typed facts so a long-stable trajectory scores 1.0. + const byMetric = new Map(); + for (const p of args.points) { + if (p.metric === null || p.value === null) continue; + if (!byMetric.has(p.metric)) byMetric.set(p.metric, []); + byMetric.get(p.metric)!.push(p); + } + let metricChanges = 0; + let typedFacts = 0; + for (const series of byMetric.values()) { + typedFacts += series.length; + for (let i = 1; i < series.length; i++) { + const a = series[i - 1].value!; + const b = series[i].value!; + if (a === 0) continue; + if (Math.abs(b - a) / Math.abs(a) >= 0.05) metricChanges += 1; + } + } + const consistencyScore = typedFacts > 0 + ? Math.max(0, Math.min(1, 1 - metricChanges / typedFacts)) + : null; + + // growth_trajectory — per metric, the most recent delta direction. + const growth: FounderScorecard['growth_trajectory'] = []; + for (const [metric, series] of byMetric) { + if (series.length < 2) continue; + const latest = series[series.length - 1].value!; + const prior = series[series.length - 2].value!; + if (prior === 0) continue; + const delta = (latest - prior) / prior; + const dir: 'up' | 'down' | 'flat' = + Math.abs(delta) < 0.01 ? 'flat' : (delta > 0 ? 'up' : 'down'); + growth.push({ metric, direction: dir, latest_delta_pct: delta }); + } + // Stable ordering: alphabetical by metric so the JSON is deterministic. + growth.sort((a, b) => a.metric.localeCompare(b.metric)); + + // red_flags — surface regressions, big narrative drift, and missed + // predictions (resolved=false on a take). + const redFlags: FounderScorecard['red_flags'] = []; + for (const r of regressions) { + const pct = Math.abs(r.delta_pct * 100).toFixed(1); + redFlags.push({ + kind: 'regression', + metric: r.metric, + text: `${r.metric} fell ${pct}% (${r.from_date} → ${r.to_date})`, + }); + } + if (drift_score !== null && drift_score >= driftRedFlag) { + redFlags.push({ + kind: 'narrative_drift', + text: `Narrative drift score ${drift_score.toFixed(2)} — claims are diverging rapidly`, + }); + } + const missed = args.takes.filter(t => t.resolved_outcome === false); + for (const m of missed) { + redFlags.push({ + kind: 'missed_prediction', + text: `Missed prediction: ${truncate(m.claim, 200)}`, + }); + } + + return { + schema_version: TRAJECTORY_SCHEMA_VERSION, + entity_slug: args.entitySlug, + window: { since: args.windowSince, until: args.windowUntil }, + claim_accuracy: { + predicted: resolved.length, + accurate, + pct: accuracyPct, + }, + consistency: { + score: consistencyScore, + metric_changes: metricChanges, + typed_facts: typedFacts, + }, + growth_trajectory: growth, + red_flags: redFlags, + }; +} + +function truncate(s: string, n: number): string { + return s.length <= n ? s : s.slice(0, n - 3) + '...'; +} + +interface RunOpts { + sub: 'scorecard'; + entitySlug: string; + since?: string; + until?: string; + json?: boolean; +} + +const HELP = `Usage: gbrain founder scorecard [options] + +Rolls up the entity's typed-claim trajectory + resolved-take outcomes into +the four founder-evaluation metrics: claim accuracy, consistency, growth +trajectory, red flags. + +Options: + --since YYYY-MM-DD Lower bound on valid_from / take created_at (default: 1 year ago) + --until YYYY-MM-DD Upper bound (default: today) + --json JSON output (stable schema_version: 1) + --help, -h Show this help + +Examples: + gbrain founder scorecard companies/acme-example + gbrain founder scorecard companies/acme-example --since 2025-05-17 --json +`; + +function parseArgs(args: string[]): RunOpts | { help: true } | { error: string } { + if (args[0] !== 'scorecard') { + return { error: `Unknown founder subcommand: ${args[0] ?? '(none)'}. Did you mean "founder scorecard"?` }; + } + const opts: Partial = { sub: 'scorecard' }; + const positional: string[] = []; + for (let i = 1; i < args.length; i++) { + const a = args[i]; + if (a === '--help' || a === '-h') return { help: true }; + if (a === '--json') { opts.json = true; continue; } + if (a === '--since') { opts.since = args[++i]; continue; } + if (a === '--until') { opts.until = args[++i]; continue; } + if (a.startsWith('-')) return { error: `Unknown flag: ${a}` }; + positional.push(a); + } + if (positional.length !== 1) { + return { error: 'Exactly one entity-slug positional argument is required.' }; + } + return { ...(opts as RunOpts), entitySlug: positional[0] }; +} + +export async function runFounder(engine: BrainEngine, args: string[]): Promise { + const parsed = parseArgs(args); + if ('help' in parsed) { + console.log(HELP); + return; + } + if ('error' in parsed) { + console.error(parsed.error); + console.error(''); + console.error(HELP); + process.exit(1); + } + + // Default window: last year. + const now = new Date(); + const oneYearAgo = new Date(now); + oneYearAgo.setFullYear(oneYearAgo.getFullYear() - 1); + const windowSince = parsed.since ?? oneYearAgo.toISOString().slice(0, 10); + const windowUntil = parsed.until ?? now.toISOString().slice(0, 10); + + let scorecard: FounderScorecard; + const cfg = loadConfig(); + if (isThinClient(cfg)) { + // Thin-client path: server-side find_trajectory + listTakes computed + // locally is too heavyweight for one HTTP call, so this path fetches + // the trajectory via the remote MCP op and reads takes through a + // future `takes_list` op. For now thin-client returns a degraded + // scorecard built from trajectory only (no take outcomes). + // TODO(v0.35.5): add `takes_list_by_entity` MCP op + thin-client wire-up. + const raw = await callRemoteTool(cfg!, 'find_trajectory', { + entity_slug: parsed.entitySlug, + since: windowSince, + until: windowUntil, + }, { timeoutMs: 30_000 }); + const trajWire = unpackToolResult<{ + points: Array<{ + fact_id: number; valid_from: string; + metric: string | null; value: number | null; + unit: string | null; period: string | null; + text: string; source_session: string | null; source_markdown_slug: string | null; + }>; + }>(raw); + const points: TrajectoryPoint[] = trajWire.points.map(p => ({ + fact_id: p.fact_id, + valid_from: new Date(p.valid_from), + metric: p.metric, + value: p.value, + unit: p.unit, + period: p.period, + text: p.text, + source_session: p.source_session, + source_markdown_slug: p.source_markdown_slug, + embedding: null, + })); + scorecard = computeFounderScorecard({ + entitySlug: parsed.entitySlug, + windowSince, + windowUntil, + points, + takes: [], + }); + } else { + // Local: full pipeline. Get the trajectory + the entity's resolved takes. + const points = await engine.findTrajectory({ + entitySlug: parsed.entitySlug, + since: windowSince, + until: windowUntil, + }); + let takes: Take[] = []; + try { + takes = await engine.listTakes({ + page_slug: parsed.entitySlug, + active: true, + resolved: true, + limit: 100, + }); + } catch { + // Some entity pages don't exist; treat as no takes. + takes = []; + } + scorecard = computeFounderScorecard({ + entitySlug: parsed.entitySlug, + windowSince, + windowUntil, + points, + takes, + }); + } + + if (parsed.json) { + console.log(JSON.stringify(scorecard, null, 2)); + return; + } + + // Human format. + console.log(`Entity: ${scorecard.entity_slug}`); + console.log(`Window: ${scorecard.window.since} → ${scorecard.window.until}`); + console.log(''); + + console.log('Claim accuracy:'); + if (scorecard.claim_accuracy.predicted === 0) { + console.log(' (no resolved predictions in window)'); + } else { + const pct = scorecard.claim_accuracy.pct === null + ? 'n/a' + : `${(scorecard.claim_accuracy.pct * 100).toFixed(1)}%`; + console.log(` ${scorecard.claim_accuracy.accurate}/${scorecard.claim_accuracy.predicted} predictions accurate (${pct})`); + } + console.log(''); + + console.log('Consistency:'); + if (scorecard.consistency.typed_facts === 0) { + console.log(' (no typed claims in window)'); + } else { + const score = scorecard.consistency.score === null + ? 'n/a' + : scorecard.consistency.score.toFixed(2); + console.log(` score ${score} (${scorecard.consistency.metric_changes} changes across ${scorecard.consistency.typed_facts} typed facts)`); + } + console.log(''); + + console.log('Growth trajectory:'); + if (scorecard.growth_trajectory.length === 0) { + console.log(' (no metrics with 2+ data points)'); + } else { + for (const g of scorecard.growth_trajectory) { + const sign = g.direction === 'up' ? '↑' : g.direction === 'down' ? '↓' : '→'; + const pct = (g.latest_delta_pct * 100).toFixed(1); + console.log(` ${g.metric}: ${sign} ${pct}%`); + } + } + console.log(''); + + console.log('Red flags:'); + if (scorecard.red_flags.length === 0) { + console.log(' (none)'); + } else { + for (const r of scorecard.red_flags) { + console.log(` [${r.kind}] ${r.text}`); + } + } +} diff --git a/src/core/cycle/extract-facts.ts b/src/core/cycle/extract-facts.ts index 28c9ac8e4..3118e8146 100644 --- a/src/core/cycle/extract-facts.ts +++ b/src/core/cycle/extract-facts.ts @@ -34,6 +34,7 @@ import type { BrainEngine } from '../engine.ts'; import { parseFactsFence } from '../facts-fence.ts'; import { extractFactsFromFenceText } from '../facts/extract-from-fence.ts'; +import { embed, isAvailable } from '../ai/gateway.ts'; export interface ExtractFactsOpts { /** Subset of slugs to reconcile. undefined = walk every page in the brain. */ @@ -136,7 +137,40 @@ export async function runExtractFacts( if (parsed.facts.length === 0) continue; - const extracted = extractFactsFromFenceText(parsed.facts, slug, sourceId); + // v0.35.4 (D-ENG-1) — thread page.effective_date as the fallback + // valid_from. Without this, fence rows without explicit `validFrom:` + // land with `valid_from = now()` (import timestamp) and every + // trajectory query against the page returns import dates instead of + // claim dates. + const pageEffectiveDate = page.effective_date ? new Date(page.effective_date) : null; + const extracted = extractFactsFromFenceText(parsed.facts, slug, sourceId, { pageEffectiveDate }); + + // v0.35.4 (D-CDX-3) — batch-embed before insert. Without this, + // cycle-inserted facts land with `embedding = NULL`, which breaks + // consolidate's cosine clustering AND the drift_score formula in + // find_trajectory. Falls open: if the embedding gateway is + // unavailable (no API key configured), facts still insert with + // NULL embeddings — drift_score gracefully returns null and + // clustering falls back to recency. + if (isAvailable('embedding') && extracted.length > 0) { + try { + const texts = extracted.map(e => e.fact); + const embeddings = await embed(texts); + // Defensive: embed should return one vector per input; if the + // gateway returns a partial array (provider partial-batch retry + // returning fewer than requested), only fill what we have. + for (let i = 0; i < extracted.length && i < embeddings.length; i++) { + extracted[i].embedding = embeddings[i]; + } + } catch (err) { + // Embedding failure is non-fatal — facts still get inserted, just + // without embeddings. Cycle phase status stays 'ok'. + result.warnings.push( + `${slug}: extract_facts batch embed failed: ${err instanceof Error ? err.message : String(err)}`, + ); + } + } + const inserted = await engine.insertFacts(extracted, { source_id: sourceId }); // gbrain-allow-direct-insert: extract_facts cycle phase reconciles fence → DB result.factsInserted += inserted.inserted; } diff --git a/src/core/cycle/phases/consolidate.ts b/src/core/cycle/phases/consolidate.ts index 2ae110c39..37875c811 100644 --- a/src/core/cycle/phases/consolidate.ts +++ b/src/core/cycle/phases/consolidate.ts @@ -147,37 +147,95 @@ export async function runPhaseConsolidate( continue; } - const inserted = await engine.addTakesBatch([{ - page_id: pageId, - row_num: nextRowNum, - claim: best.fact, - kind: 'fact', - holder: 'self', - weight: clamp01(avgWeight), - since_date: sinceISO, - source: sources.slice(0, 200), - active: true, - }]); - if (inserted < 1) continue; - - // Read back the new takes.id by (page_id, row_num). - const idRows = await engine.executeRaw<{ id: number }>( - `SELECT id FROM takes WHERE page_id = $1 AND row_num = $2`, - [pageId, nextRowNum], + // v0.35.4 (D-CDX-4) — semantic upsert. The full dream cycle runs + // `extract_facts` BEFORE `consolidate`; `extract_facts` hard-deletes + // and re-inserts page facts via deleteFactsForPage + insertFacts, + // which clears `consolidated_at` on every fact. Without this lookup, + // a second cycle run would re-INSERT a duplicate take via + // `MAX(row_num)+1`, silently poisoning trajectory + scorecard data. + // Match on (page_id, claim, since_date) — the natural identity of a + // promoted take. + const existing = await engine.executeRaw<{ id: number }>( + `SELECT id FROM takes + WHERE page_id = $1 AND claim = $2 AND since_date = $3 + LIMIT 1`, + [pageId, best.fact, sinceISO], ); - if (idRows.length === 0) { + + let takeId: number; + if (existing.length > 0) { + // Re-promotion of a cluster we already wrote a take for. Refresh + // the source-aggregation string (new fact rows may carry new + // source_session values that the prior run didn't see); leave + // row_num + weight untouched to keep the take's identity stable. + takeId = existing[0].id; + await engine.executeRaw( + `UPDATE takes SET source = $1, updated_at = now() WHERE id = $2`, + [sources.slice(0, 200), takeId], + ); + } else { + const inserted = await engine.addTakesBatch([{ + page_id: pageId, + row_num: nextRowNum, + claim: best.fact, + kind: 'fact', + holder: 'self', + weight: clamp01(avgWeight), + since_date: sinceISO, + source: sources.slice(0, 200), + active: true, + }]); + if (inserted < 1) continue; + + const idRows = await engine.executeRaw<{ id: number }>( + `SELECT id FROM takes WHERE page_id = $1 AND row_num = $2`, + [pageId, nextRowNum], + ); + if (idRows.length === 0) { + nextRowNum += 1; + continue; + } + takeId = idRows[0].id; nextRowNum += 1; - continue; + takesWritten += 1; } - const takeId = idRows[0].id; - nextRowNum += 1; - takesWritten += 1; // Mark all contributing facts consolidated. for (const f of cluster) { await engine.consolidateFact(f.id, takeId); factsConsolidated += 1; } + + // v0.35.4 (D-CDX-4 part 2) — chronological valid_until writeback. + // Sort the cluster by (valid_from ASC, id ASC); walk consecutive + // pairs; stamp the older fact's valid_until = next_newer.valid_from. + // The newest fact keeps valid_until = NULL. This makes the facts + // table a proper bitemporal record without the contradiction probe + // having to mutate it (preserves auto-supersession.ts:4 invariant — + // see also R8 test guard). + // + // Idempotent: re-running on the same cluster produces the same + // chronological order and the same valid_until values. No-op if + // valid_until is already correct. + const chronological = [...cluster].sort((a, b) => { + const t = a.valid_from.getTime() - b.valid_from.getTime(); + if (t !== 0) return t; + return a.id - b.id; + }); + for (let i = 0; i < chronological.length - 1; i++) { + const older = chronological[i]; + const newer = chronological[i + 1]; + await engine.executeRaw( + // Only UPDATE when the new value would actually change. Avoids + // touching updated_at on no-op rewrites and keeps idempotency + // observable in the DB (zero affected rows on stable re-run). + `UPDATE facts + SET valid_until = $1 + WHERE id = $2 + AND (valid_until IS DISTINCT FROM $1)`, + [newer.valid_from, older.id], + ); + } } } diff --git a/src/core/engine.ts b/src/core/engine.ts index 101db1445..6f2474991 100644 --- a/src/core/engine.ts +++ b/src/core/engine.ts @@ -369,6 +369,17 @@ export interface NewFact { confidence?: number; // [0,1], default 1.0 notability?: 'high' | 'medium' | 'low'; // salience filter for extraction gate embedding?: Float32Array | null; // pre-computed; if null, insertFact computes via gateway + /** + * v0.35.4 (D-CDX-5) — typed-claim fields. Optional. When populated, + * `gbrain eval trajectory` + `find_trajectory` MCP op consume them for + * chronological regression detection and drift_score. `claim_metric` is + * normalized to lowercase snake_case by the extraction layer before + * this method sees it; the engine stores verbatim. + */ + claim_metric?: string | null; + claim_value?: number | null; + claim_unit?: string | null; + claim_period?: string | null; } /** Options shared by list-facts methods. */ @@ -402,6 +413,58 @@ export interface FactsHealth { p99_latency_ms?: number; } +/** + * v0.35.4 (D-CDX-6) — Options for `BrainEngine.findTrajectory`. + * + * `sourceId` (scalar fast path) and `sourceIds` (federated array) follow + * the v0.34.1.0 search* pattern: when `sourceIds` is set the engine + * applies `WHERE source_id = ANY($N::text[])`; otherwise scalar predicate + * with `sourceId ?? 'default'`. + * + * `remote` (D-CDX-1) gates the visibility filter: when true the engine + * adds `AND visibility = 'world'`, mirroring `recall`'s posture for + * untrusted callers. Local CLI keeps `remote: false` and sees both + * private + world facts. + */ +export interface TrajectoryOpts { + entitySlug: string; + /** Single-source scope; default 'default' when both this and sourceIds are unset. */ + sourceId?: string; + /** Federated array scope (mutually exclusive with sourceId; the array wins when set). */ + sourceIds?: string[]; + /** When true, filters to visibility='world' only. Set by MCP layer from ctx.remote. */ + remote?: boolean; + /** Metric filter. When set, only facts with this canonical metric label participate. */ + metric?: string; + /** Lower bound on valid_from (inclusive). YYYY-MM-DD or full ISO. */ + since?: string | Date; + /** Upper bound on valid_from (inclusive). YYYY-MM-DD or full ISO. */ + until?: string | Date; + /** Cap on points returned. Default 100, max 500. */ + limit?: number; +} + +/** + * A single point in an entity's claim trajectory. Carries the typed-claim + * fields when populated (drives regression detection), the underlying + * fact text (for display), provenance (source_session, source_markdown_slug), + * and the raw embedding so the caller can compute drift_score without a + * second SQL round-trip. + */ +export interface TrajectoryPoint { + fact_id: number; + valid_from: Date; + metric: string | null; + value: number | null; + unit: string | null; + period: string | null; + text: string; + source_session: string | null; + source_markdown_slug: string | null; + /** Raw embedding for drift computation; null when the fact was inserted without one. */ + embedding: Float32Array | null; +} + /** Maximum results returned by search operations. Internal bulk operations (listPages) are not clamped. */ export const MAX_SEARCH_LIMIT = 100; @@ -1134,6 +1197,21 @@ export interface BrainEngine { */ consolidateFact(id: number, takeId: number): Promise; + /** + * v0.35.4 (D-CDX-1 + D-CDX-6) — chronological fact trajectory for an + * entity. Returns points ordered by (valid_from ASC, id ASC) so the + * caller can compute regressions and drift_score deterministically. + * + * - Source-scoped via `sourceId` (scalar) OR `sourceIds` (federated array). + * - Visibility-filtered: when `opts.remote=true`, only `visibility='world'` + * facts are returned. Trusted local callers see both private + world. + * - Optional metric filter restricts to a single normalized metric label. + * - Active-only by default (expired_at IS NULL); soft-deleted entities + * on the pages side are NOT filtered here — trajectory is a facts-table + * query and doesn't JOIN pages. + */ + findTrajectory(opts: TrajectoryOpts): Promise; + /** Per-source operational metrics for `gbrain doctor` facts_health check. */ getFactsHealth(source_id: string): Promise; diff --git a/src/core/facts-fence.ts b/src/core/facts-fence.ts index 44de97d6a..a9602178c 100644 --- a/src/core/facts-fence.ts +++ b/src/core/facts-fence.ts @@ -98,6 +98,25 @@ export interface ParsedFact { */ supersededBy?: number; forgotten?: boolean; + /** + * v0.35.4 typed-claim fields (D-CDX-5). Optional. When present, drives + * `gbrain eval trajectory` + the `find_trajectory` MCP op chronological + * regression detection. The fence layout widens from 10 to 14 columns + * when any row in the table has a non-undefined typed field; otherwise + * stays 10-cell for backward compat with existing fences. + * + * - `claimMetric`: lowercase snake_case after normalization + * (`mrr`, `arr`, `team_size`, …). Free-text labels accepted; the + * parser does not enforce the seed-map allow-list. + * - `claimValue`: numeric, finite. Empty cell → undefined. + * - `claimUnit`: free-form unit string (`USD`, `people`, `pct`, …). + * - `claimPeriod`: free-form period string (`monthly`, `annual`, …) + * or undefined for non-periodic metrics. + */ + claimMetric?: string; + claimValue?: number; + claimUnit?: string; + claimPeriod?: string; } export interface FactsFenceParseResult { @@ -112,6 +131,20 @@ function parseConfidenceCell(raw: string): number | undefined { return Number.isFinite(n) ? n : undefined; } +/** + * v0.35.4 — parse a free-form numeric cell for typed-claim values. + * Empty / non-numeric → undefined (caller decides whether to drop or warn). + * Tolerates plain numbers and standard scientific notation. Locale-dependent + * thousand separators (`,`) are stripped so `50,000` parses to `50000`. + */ +function parseNumericCell(raw: string): number | undefined { + const trimmed = raw.trim(); + if (!trimmed) return undefined; + const stripped = trimmed.replace(/,/g, ''); + const n = parseFloat(stripped); + return Number.isFinite(n) ? n : undefined; +} + function parseSupersededByFromContext(context: string | undefined): number | undefined { if (!context) return undefined; const m = context.match(/superseded by #(\d+)/i); @@ -175,8 +208,10 @@ export function parseFactsFence(body: string): FactsFenceParseResult { // Separator row (just dashes/colons) — skip. if (isSeparatorRow(cells)) continue; - // Expect 10 cells: row_num, claim, kind, confidence, visibility, - // notability, valid_from, valid_until, source, context. + // Expect 10 cells (legacy 10-cell fence) OR 14 cells (v0.35.4 + // typed-claim wide fence): row_num, claim, kind, confidence, + // visibility, notability, valid_from, valid_until, source, context, + // [claim_metric, claim_value, claim_unit, claim_period]. // Tolerate 9 (missing trailing context cell) — markdown editors often // drop empty trailing cells. if (cells.length < 9) { @@ -190,6 +225,10 @@ export function parseFactsFence(body: string): FactsFenceParseResult { validFromRaw, validUntilRaw, sourceRaw, contextRaw = '', + claimMetricRaw = '', + claimValueRaw = '', + claimUnitRaw = '', + claimPeriodRaw = '', ] = cells; const rowNum = parseInt(rowNumStr, 10); @@ -246,6 +285,11 @@ export function parseFactsFence(body: string): FactsFenceParseResult { active: !struck, supersededBy, forgotten: struck ? forgotten : false, + // v0.35.4 — typed-claim fields, all optional. + claimMetric: parseStringCell(claimMetricRaw), + claimValue: parseNumericCell(claimValueRaw), + claimUnit: parseStringCell(claimUnitRaw), + claimPeriod: parseStringCell(claimPeriodRaw), }); } @@ -273,11 +317,28 @@ function formatConfidence(c: number): string { * fence state survives unrelated edits to other rows. */ export function renderFactsTable(facts: ParsedFact[]): string { - const header = `| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context |`; - const separator = `|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|`; + // v0.35.4 (D-CDX-5): widen to 14 cells when ANY row has a non-undefined + // typed-claim field. Otherwise stay at the 10-cell legacy shape so + // existing fences don't get widened on unrelated rewrites (no churn diff + // noise). + const anyTyped = facts.some(f => + f.claimMetric !== undefined || + f.claimValue !== undefined || + f.claimUnit !== undefined || + f.claimPeriod !== undefined, + ); + const header = anyTyped + ? `| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | claim_metric | claim_value | claim_unit | claim_period |` + : `| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context |`; + const separator = anyTyped + ? `|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|--------------|-------------|------------|--------------|` + : `|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|`; const rows = facts.map(f => { const claimCell = f.active ? f.claim : `~~${f.claim}~~`; - return `| ${f.rowNum} | ${escapeFenceCell(claimCell)} | ${f.kind} | ${formatConfidence(f.confidence)} | ${f.visibility} | ${f.notability} | ${escapeFenceCell(f.validFrom ?? '')} | ${escapeFenceCell(f.validUntil ?? '')} | ${escapeFenceCell(f.source ?? '')} | ${escapeFenceCell(f.context ?? '')} |`; + const base = `| ${f.rowNum} | ${escapeFenceCell(claimCell)} | ${f.kind} | ${formatConfidence(f.confidence)} | ${f.visibility} | ${f.notability} | ${escapeFenceCell(f.validFrom ?? '')} | ${escapeFenceCell(f.validUntil ?? '')} | ${escapeFenceCell(f.source ?? '')} | ${escapeFenceCell(f.context ?? '')} |`; + if (!anyTyped) return base; + const valueCell = f.claimValue === undefined ? '' : String(f.claimValue); + return `${base} ${escapeFenceCell(f.claimMetric ?? '')} | ${escapeFenceCell(valueCell)} | ${escapeFenceCell(f.claimUnit ?? '')} | ${escapeFenceCell(f.claimPeriod ?? '')} |`; }); const inner = ['', header, separator, ...rows, ''].join('\n'); return `${FACTS_FENCE_BEGIN}${inner}${FACTS_FENCE_END}`; @@ -317,6 +378,13 @@ export function upsertFactRow( source: newRow.source, context: newRow.context, active: newRow.active ?? true, + // v0.35.4 — typed-claim pass-through. When undefined the renderer + // stays at the 10-cell shape so unrelated edits don't widen the + // fence. + claimMetric: newRow.claimMetric, + claimValue: newRow.claimValue, + claimUnit: newRow.claimUnit, + claimPeriod: newRow.claimPeriod, }, ]; diff --git a/src/core/facts/extract-from-fence.ts b/src/core/facts/extract-from-fence.ts index 93e253221..a404df7b4 100644 --- a/src/core/facts/extract-from-fence.ts +++ b/src/core/facts/extract-from-fence.ts @@ -82,6 +82,83 @@ export interface ExtractFromFenceOpts { * the mapper uses real UTC midnight today. */ nowOverride?: Date; + /** + * v0.35.4 (D-ENG-1 + D-CDX-5) — optional fallback for `valid_from` when + * the fence row lacks an explicit `validFrom:`. Threaded by the + * `extract-facts` cycle phase from `engine.getPage(slug).effective_date` + * so a meeting page dated 2026-04-28 stamps its facts as claimed-on + * that date instead of "the import timestamp". + * + * Precedence chain: + * 1. Explicit `validFrom:` in the fence row (today's behavior, preserved). + * 2. `pageEffectiveDate` when set. + * 3. `undefined` → engine.insertFact defaults to now() at insert time. + * + * Optional because `facts/fence-write.ts` calls this from a context + * with no `Page` object available (Codex F6). Null and undefined are + * treated identically: fall through to behavior (3). + */ + pageEffectiveDate?: Date | null; +} + +/** + * v0.35.4 (D-ENG-4) — normalized metric vocabulary. + * + * Seed map for common founder/company metrics. Free-text labels normalize + * to lowercase snake_case so trajectory queries don't fragment across + * capitalization variants. Unknown labels still pass through (lowercased + * + spaces → underscores) so the user can author arbitrary metrics + * without a code change. Exported so tests can pin the map. + */ +export const METRIC_NORMALIZATION_MAP: ReadonlyMap = new Map([ + // Revenue / financial + ['mrr', 'mrr'], + ['monthly recurring revenue', 'mrr'], + ['arr', 'arr'], + ['annual recurring revenue', 'arr'], + ['revenue', 'revenue'], + ['burn', 'burn_rate'], + ['burn rate', 'burn_rate'], + ['runway', 'runway'], + ['cash', 'cash'], + ['gross margin', 'gross_margin'], + // Funding + ['fundraise', 'fundraise'], + ['raise', 'fundraise'], + // People + ['headcount', 'headcount'], + ['team size', 'team_size'], + ['team', 'team_size'], + // Users / engagement + ['users', 'users'], + ['mau', 'mau'], + ['monthly active users', 'mau'], + ['dau', 'dau'], + ['daily active users', 'dau'], + ['churn', 'churn_rate'], + ['churn rate', 'churn_rate'], + // Unit economics + ['cac', 'cac'], + ['ltv', 'ltv'], +]); + +/** + * Normalize a free-text metric label to lowercase snake_case. Known + * labels (seed map above) map to canonical names; unknown labels are + * lowercased + whitespace-collapsed → underscores. Returns undefined for + * empty / whitespace-only input so the caller can treat it as "no + * metric set" without an extra null check. + */ +export function normalizeMetricLabel(raw: string | undefined | null): string | undefined { + if (raw === undefined || raw === null) return undefined; + const trimmed = raw.trim().toLowerCase(); + if (!trimmed) return undefined; + const seed = METRIC_NORMALIZATION_MAP.get(trimmed); + if (seed) return seed; + // Collapse runs of whitespace to single underscore, strip non-alphanumeric + // edges. Allows users to write "Net Promoter Score" → `net_promoter_score` + // without registering it. + return trimmed.replace(/\s+/g, '_').replace(/[^a-z0-9_]/g, ''); } /** @@ -91,7 +168,7 @@ export interface ExtractFromFenceOpts { * @param slug The entity page slug (also becomes source_markdown_slug) * @param sourceId The source binding (resolved from sources.local_path * by the caller; multi-source brains thread this through) - * @param opts Test-only overrides + * @param opts Optional overrides (test nowOverride; v0.35.4 page-date fallback) */ export function extractFactsFromFenceText( facts: ParsedFact[], @@ -100,9 +177,12 @@ export function extractFactsFromFenceText( opts: ExtractFromFenceOpts = {}, ): FenceExtractedFact[] { const today = opts.nowOverride ?? todayUtcDate(); + const pageDateFallback = opts.pageEffectiveDate ?? undefined; return facts.map(f => { - const validFrom = parseValidDate(f.validFrom); + // v0.35.4 (D-ENG-1) valid_from precedence: fence > pageEffectiveDate > engine default (now). + const fenceDate = parseValidDate(f.validFrom); + const validFrom = fenceDate ?? pageDateFallback ?? undefined; // valid_until derivation. Three branches: // 1. Explicit validUntil in the fence → honor as-is. @@ -137,6 +217,13 @@ export function extractFactsFromFenceText( confidence: f.confidence, row_num: f.rowNum, source_markdown_slug: slug, + // v0.35.4 (D-CDX-5) — typed-claim threading. Metric label normalized + // here so the DB-side index hits use the canonical name; value / + // unit / period stored verbatim. + claim_metric: normalizeMetricLabel(f.claimMetric) ?? null, + claim_value: f.claimValue ?? null, + claim_unit: f.claimUnit ?? null, + claim_period: f.claimPeriod ?? null, }; return row; }); diff --git a/src/core/facts/extract.ts b/src/core/facts/extract.ts index 683b3c007..919667bbf 100644 --- a/src/core/facts/extract.ts +++ b/src/core/facts/extract.ts @@ -26,6 +26,7 @@ import type { ChatResult } from '../ai/gateway.ts'; import { INJECTION_PATTERNS } from '../think/sanitize.ts'; import { resolveModel } from '../model-config.ts'; import type { BrainEngine, NewFact, FactKind } from '../engine.ts'; +import { normalizeMetricLabel } from './extract-from-fence.ts'; /** * v0.31 (D15): kill-switch for fact extraction. @@ -101,7 +102,9 @@ const EXTRACTOR_SYSTEM = [ 'Output strictly one JSON object on a single line:', '{"facts":[{"fact":"","kind":"event|preference|commitment|belief|fact",', '"entity":"","confidence":<0..1>,', - '"notability":"high|medium|low"}]}.', + '"notability":"high|medium|low",', + '"metric":"","value":,', + '"unit":"","period":""}]}.', 'No prose, no code fences. Empty facts array is valid when nothing claim-worthy was said.', '', 'Rules:', @@ -124,6 +127,19 @@ const EXTRACTOR_SYSTEM = [ ' Can wait for batch processing.', ' * "low": Logistical noise, restaurant orders, routine scheduling, "we\'re at X place".', ' Skip entirely — not worth storing.', + '', + '- Typed-claim fields (metric/value/unit/period) — emit ONLY when the claim', + ' carries a quantitative metric assertion. Examples:', + ' * "MRR: $50K (Jan 2026)" → metric=mrr, value=50000, unit=USD, period=monthly', + ' * "ARR: $2M" → metric=arr, value=2000000, unit=USD, period=annual', + ' * "Team size: 12" → metric=team_size, value=12, unit=people, period=null', + ' * "Closed Series A: $15M" → metric=fundraise, value=15000000, unit=USD, period=null', + ' * "User churn: 5%" → metric=churn_rate, value=0.05, unit=pct, period=null', + ' Use lowercase snake_case for metric. Common labels: mrr, arr, revenue,', + ' runway, burn_rate, cash, gross_margin, team_size, headcount, users, mau,', + ' dau, cac, ltv, churn_rate, fundraise. For non-metric claims (preferences,', + ' events, beliefs), set all four to null. Numeric values: emit the raw', + ' number after currency/scale normalization (50000 not "$50K"; 0.05 not "5%").', ].join('\n'); const MAX_TURN_TEXT_CHARS = 8000; @@ -207,6 +223,15 @@ export async function extractFactsFromTurn(input: ExtractInput): Promise 0 diff --git a/src/core/operations-descriptions.ts b/src/core/operations-descriptions.ts index 61e8ba875..e0d02b5c9 100644 --- a/src/core/operations-descriptions.ts +++ b/src/core/operations-descriptions.ts @@ -90,6 +90,21 @@ export const FIND_CONTRADICTIONS_DESCRIPTION = "Reads the cached run row — does NOT trigger a new probe; users run " + "`gbrain eval suspected-contradictions` for that."; +export const FIND_TRAJECTORY_DESCRIPTION = + "v0.35.4 — return the chronological claim trajectory for an entity (typed " + + "metric values over time, plus auto-detected regressions and narrative drift). " + + "Use this when the user asks 'how has Acme's MRR trended', 'show me what " + + "alice-example said about runway over time', 'is this founder consistent', " + + "'find regressions for fund-a's portfolio', or wants a time-series view of an " + + "entity's structured claims. Returns " + + "`{points: [{fact_id, valid_from, metric, value, unit, period, text, source_session, source_markdown_slug}], " + + "regressions: [{metric, from_value, from_date, to_value, to_date, delta_pct}], " + + "drift_score: number|null, schema_version: 1}`. Drift score 0 = stable narrative, " + + "1 = every consecutive claim is unrelated; null when fewer than 3 typed points " + + "exist. Visibility-filtered for remote callers (world-only); source-scoped by " + + "the caller's OAuth source binding. Pair with `gbrain founder scorecard ` " + + "for an aggregated rollup of the same data."; + // ────────────────────────────────────────────────────────────────────────────── // v0.33.3 Cathedral III foundation — code-intelligence ops (MCP-exposed). // Pre-v0.33.3 the callers/callees/def/refs commands were CLI-only — agents diff --git a/src/core/operations.ts b/src/core/operations.ts index 49ed17700..7b59e8771 100644 --- a/src/core/operations.ts +++ b/src/core/operations.ts @@ -31,6 +31,7 @@ import { QUERY_DESCRIPTION, SEARCH_DESCRIPTION, FIND_CONTRADICTIONS_DESCRIPTION, + FIND_TRAJECTORY_DESCRIPTION, CODE_CALLERS_DESCRIPTION, CODE_CALLEES_DESCRIPTION, CODE_DEF_DESCRIPTION, @@ -2444,6 +2445,86 @@ const find_contradictions: Operation = { cliHints: { name: 'find-contradictions' }, }; +const find_trajectory: Operation = { + name: 'find_trajectory', + description: FIND_TRAJECTORY_DESCRIPTION, + scope: 'read', + // localOnly intentionally NOT set — federated OAuth clients should be + // able to query trajectories for entities in their scope. Visibility + // filtering (D-CDX-1) inside the engine restricts remote callers to + // visibility='world' facts. + params: { + entity_slug: { + type: 'string', + description: 'Required. Entity slug to chart (e.g. "companies/acme-example", "people/alice-example").', + }, + metric: { + type: 'string', + description: 'Optional. Filter to a single canonical metric (e.g. "mrr", "arr", "team_size"). When omitted, all metrics return.', + }, + since: { + type: 'string', + description: 'Optional lower bound on valid_from (YYYY-MM-DD or ISO).', + }, + until: { + type: 'string', + description: 'Optional upper bound on valid_from (YYYY-MM-DD or ISO).', + }, + limit: { + type: 'number', + description: 'Max points returned. Default 100, max 500.', + }, + }, + handler: async (ctx, p) => { + if (typeof p.entity_slug !== 'string' || !p.entity_slug.trim()) { + throw new Error('find_trajectory requires entity_slug (string)'); + } + const metric = typeof p.metric === 'string' ? p.metric : undefined; + const since = typeof p.since === 'string' ? p.since : undefined; + const until = typeof p.until === 'string' ? p.until : undefined; + const limit = typeof p.limit === 'number' ? p.limit : undefined; + const scope = sourceScopeOpts(ctx); + + // D-CDX-1: thread ctx.remote into the engine so visibility filtering + // happens at SQL level. Mirrors recall's posture for untrusted callers. + const points = await ctx.engine.findTrajectory({ + entitySlug: p.entity_slug, + ...scope, + remote: ctx.remote === true, + metric, + since, + until, + limit, + }); + + const { computeTrajectoryStats, TRAJECTORY_SCHEMA_VERSION } = await import('./trajectory.ts'); + const { regressions, drift_score } = computeTrajectoryStats(points); + + // Engine result includes raw embeddings (Float32Array); strip those + // before sending over MCP — they're bulky binary noise that consumers + // never need at this layer. + const wirePoints = points.map(pt => ({ + fact_id: pt.fact_id, + valid_from: pt.valid_from.toISOString().slice(0, 10), + metric: pt.metric, + value: pt.value, + unit: pt.unit, + period: pt.period, + text: pt.text, + source_session: pt.source_session, + source_markdown_slug: pt.source_markdown_slug, + })); + + return { + points: wirePoints, + regressions, + drift_score, + schema_version: TRAJECTORY_SCHEMA_VERSION, + }; + }, + cliHints: { name: 'find-trajectory' }, +}; + const get_recent_transcripts: Operation = { name: 'get_recent_transcripts', description: GET_RECENT_TRANSCRIPTS_DESCRIPTION, @@ -3172,6 +3253,8 @@ export const operations: Operation[] = [ find_contradictions, // v0.33: expertise + relationship-proximity routing find_experts, + // v0.35.4: temporal trajectory (typed claims over time + regression detection) + find_trajectory, // v0.33.3: Cathedral III code-intelligence (MCP-exposed; were CLI_ONLY pre-v0.33.3) code_callers, code_callees, code_def, code_refs, // v0.34 W3: recursive code_blast + code_flow diff --git a/src/core/pglite-engine.ts b/src/core/pglite-engine.ts index 6b7a759bd..ec1790e14 100644 --- a/src/core/pglite-engine.ts +++ b/src/core/pglite-engine.ts @@ -2297,22 +2297,41 @@ export class PGLiteEngine implements BrainEngine { const embedding = input.embedding ?? null; const embeddedAt = embedding ? new Date() : null; const embedStr = embedding ? toPgVectorLiteral(embedding) : null; + // v0.35.4 (D-CDX-5) — typed-claim columns. All four nullable. + const claimMetric = input.claim_metric ?? null; + const claimValue = input.claim_value ?? null; + const claimUnit = input.claim_unit ?? null; + const claimPeriod = input.claim_period ?? null; if (ctx.supersedeId !== undefined) { // Supersede flow: insert new + expire old in one txn so observers never // see both rows active simultaneously. const result = await this.db.transaction(async (tx) => { const ins = await tx.query<{ id: number }>( - `INSERT INTO facts ( - source_id, entity_slug, fact, kind, visibility, notability, context, - valid_from, valid_until, source, source_session, confidence, - embedding, embedded_at - ) VALUES ( - $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12, ${embedStr === null ? 'NULL' : `$13::vector`}, ${embedStr === null ? 'NULL' : '$14'} - ) RETURNING id`, embedStr === null - ? [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence] - : [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embedStr, embeddedAt], + ? `INSERT INTO facts ( + source_id, entity_slug, fact, kind, visibility, notability, context, + valid_from, valid_until, source, source_session, confidence, + embedding, embedded_at, + claim_metric, claim_value, claim_unit, claim_period + ) VALUES ( + $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12, + NULL, NULL, + $13, $14, $15, $16 + ) RETURNING id` + : `INSERT INTO facts ( + source_id, entity_slug, fact, kind, visibility, notability, context, + valid_from, valid_until, source, source_session, confidence, + embedding, embedded_at, + claim_metric, claim_value, claim_unit, claim_period + ) VALUES ( + $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12, + $13::vector, $14, + $15, $16, $17, $18 + ) RETURNING id`, + embedStr === null + ? [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, claimMetric, claimValue, claimUnit, claimPeriod] + : [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embedStr, embeddedAt, claimMetric, claimValue, claimUnit, claimPeriod], ); const newId = ins.rows[0].id; await tx.query( @@ -2326,16 +2345,30 @@ export class PGLiteEngine implements BrainEngine { } const ins = await this.db.query<{ id: number }>( - `INSERT INTO facts ( - source_id, entity_slug, fact, kind, visibility, notability, context, - valid_from, valid_until, source, source_session, confidence, - embedding, embedded_at - ) VALUES ( - $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12, ${embedStr === null ? 'NULL' : `$13::vector`}, ${embedStr === null ? 'NULL' : '$14'} - ) RETURNING id`, embedStr === null - ? [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence] - : [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embedStr, embeddedAt], + ? `INSERT INTO facts ( + source_id, entity_slug, fact, kind, visibility, notability, context, + valid_from, valid_until, source, source_session, confidence, + embedding, embedded_at, + claim_metric, claim_value, claim_unit, claim_period + ) VALUES ( + $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12, + NULL, NULL, + $13, $14, $15, $16 + ) RETURNING id` + : `INSERT INTO facts ( + source_id, entity_slug, fact, kind, visibility, notability, context, + valid_from, valid_until, source, source_session, confidence, + embedding, embedded_at, + claim_metric, claim_value, claim_unit, claim_period + ) VALUES ( + $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12, + $13::vector, $14, + $15, $16, $17, $18 + ) RETURNING id`, + embedStr === null + ? [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, claimMetric, claimValue, claimUnit, claimPeriod] + : [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embedStr, embeddedAt, claimMetric, claimValue, claimUnit, claimPeriod], ); return { id: ins.rows[0].id, status: 'inserted' }; } @@ -2376,23 +2409,45 @@ export class PGLiteEngine implements BrainEngine { const embedding = input.embedding ?? null; const embeddedAt = embedding ? new Date() : null; const embedStr = embedding ? toPgVectorLiteral(embedding) : null; + // v0.35.4 (D-CDX-5) — typed-claim columns. All four nullable. + const claimMetric = input.claim_metric ?? null; + const claimValue = input.claim_value ?? null; + const claimUnit = input.claim_unit ?? null; + const claimPeriod = input.claim_period ?? null; + // Param-positional dispatch: embedStr presence shifts the trailing + // slots by one. Order of named slots stays stable across both + // branches: embedded_at, row_num, source_markdown_slug, + // claim_metric, claim_value, claim_unit, claim_period. const ins = await tx.query<{ id: number }>( - `INSERT INTO facts ( - source_id, entity_slug, fact, kind, visibility, notability, context, - valid_from, valid_until, source, source_session, confidence, - embedding, embedded_at, - row_num, source_markdown_slug - ) VALUES ( - $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12, - ${embedStr === null ? 'NULL' : `$13::vector`}, - ${embedStr === null ? '$13' : '$14'}, - ${embedStr === null ? '$14' : '$15'}, - ${embedStr === null ? '$15' : '$16'} - ) RETURNING id`, embedStr === null - ? [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embeddedAt, input.row_num, input.source_markdown_slug] - : [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embedStr, embeddedAt, input.row_num, input.source_markdown_slug], + ? `INSERT INTO facts ( + source_id, entity_slug, fact, kind, visibility, notability, context, + valid_from, valid_until, source, source_session, confidence, + embedding, embedded_at, + row_num, source_markdown_slug, + claim_metric, claim_value, claim_unit, claim_period + ) VALUES ( + $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12, + NULL, $13, + $14, $15, + $16, $17, $18, $19 + ) RETURNING id` + : `INSERT INTO facts ( + source_id, entity_slug, fact, kind, visibility, notability, context, + valid_from, valid_until, source, source_session, confidence, + embedding, embedded_at, + row_num, source_markdown_slug, + claim_metric, claim_value, claim_unit, claim_period + ) VALUES ( + $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12, + $13::vector, $14, + $15, $16, + $17, $18, $19, $20 + ) RETURNING id`, + embedStr === null + ? [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embeddedAt, input.row_num, input.source_markdown_slug, claimMetric, claimValue, claimUnit, claimPeriod] + : [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embedStr, embeddedAt, input.row_num, input.source_markdown_slug, claimMetric, claimValue, claimUnit, claimPeriod], ); out.push(ins.rows[0].id); } @@ -2517,6 +2572,98 @@ export class PGLiteEngine implements BrainEngine { return result.rows.map(rowToFact); } + async findTrajectory(opts: import('./engine.ts').TrajectoryOpts): Promise { + const limit = clampSearchLimit(opts.limit, 100, 500); + const sinceDate = opts.since ? new Date(opts.since) : null; + const untilDate = opts.until ? new Date(opts.until) : null; + const metric = opts.metric ?? null; + const useArray = Array.isArray(opts.sourceIds) && opts.sourceIds.length > 0; + const sourceIds = useArray ? opts.sourceIds! : null; + const sourceId = opts.sourceId ?? 'default'; + const remoteFilter = opts.remote === true; + + // Build SQL dynamically. PGLite uses $N positional params; we + // assemble the WHERE clauses + params array in tandem to keep them + // aligned. Final shape is single SELECT, ORDER BY (valid_from, id) ASC. + const where: string[] = [ + useArray ? `source_id = ANY($1::text[])` : `source_id = $1`, + `entity_slug = $2`, + `expired_at IS NULL`, + ]; + const params: unknown[] = [useArray ? sourceIds : sourceId, opts.entitySlug]; + let p = 3; + if (remoteFilter) { + where.push(`visibility = 'world'`); + } + if (metric !== null) { + where.push(`claim_metric = $${p}`); + params.push(metric); + p += 1; + } + if (sinceDate) { + where.push(`valid_from >= $${p}`); + params.push(sinceDate); + p += 1; + } + if (untilDate) { + where.push(`valid_from <= $${p}`); + params.push(untilDate); + p += 1; + } + params.push(limit); + const limitPlaceholder = p; + + const sqlText = ` + SELECT id, valid_from, + claim_metric, claim_value, claim_unit, claim_period, + fact, source_session, source_markdown_slug, + embedding + FROM facts + WHERE ${where.join(' AND ')} + ORDER BY valid_from ASC, id ASC + LIMIT $${limitPlaceholder} + `; + const result = await this.db.query<{ + id: number; + valid_from: Date | string; + claim_metric: string | null; + claim_value: number | null; + claim_unit: string | null; + claim_period: string | null; + fact: string; + source_session: string | null; + source_markdown_slug: string | null; + embedding: string | number[] | Float32Array | null; + }>(sqlText, params); + + return result.rows.map(r => { + // Inline embedding parser — mirrors rowToFact() at line 3911. + let embedding: Float32Array | null = null; + if (r.embedding != null) { + if (r.embedding instanceof Float32Array) embedding = r.embedding; + else if (Array.isArray(r.embedding)) embedding = new Float32Array(r.embedding); + else if (typeof r.embedding === 'string') { + const trimmed = r.embedding.trim(); + const inner = trimmed.startsWith('[') ? trimmed.slice(1, -1) : trimmed; + const parts = inner.split(',').map(s => parseFloat(s.trim())).filter(Number.isFinite); + embedding = parts.length > 0 ? new Float32Array(parts) : null; + } + } + return { + fact_id: Number(r.id), + valid_from: r.valid_from instanceof Date ? r.valid_from : new Date(r.valid_from), + metric: r.claim_metric, + value: r.claim_value === null ? null : Number(r.claim_value), + unit: r.claim_unit, + period: r.claim_period, + text: r.fact, + source_session: r.source_session, + source_markdown_slug: r.source_markdown_slug, + embedding, + }; + }); + } + async consolidateFact(id: number, takeId: number): Promise { await this.db.query( `UPDATE facts SET consolidated_at = now(), consolidated_into = $1 WHERE id = $2`, diff --git a/src/core/postgres-engine.ts b/src/core/postgres-engine.ts index b8c3c7b49..235b70f71 100644 --- a/src/core/postgres-engine.ts +++ b/src/core/postgres-engine.ts @@ -2342,6 +2342,11 @@ export class PostgresEngine implements BrainEngine { const embedding = input.embedding ?? null; const embeddedAt = embedding ? new Date() : null; const embedLit = embedding ? toPgVectorLiteral(embedding) : null; + // v0.35.4 (D-CDX-5) — typed-claim columns. All four nullable. + const claimMetric = input.claim_metric ?? null; + const claimValue = input.claim_value ?? null; + const claimUnit = input.claim_unit ?? null; + const claimPeriod = input.claim_period ?? null; if (ctx.supersedeId !== undefined) { // Per-entity advisory lock + atomic insert + supersede in one txn. @@ -2354,11 +2359,13 @@ export class PostgresEngine implements BrainEngine { INSERT INTO facts ( source_id, entity_slug, fact, kind, visibility, notability, context, valid_from, valid_until, source, source_session, confidence, - embedding, embedded_at + embedding, embedded_at, + claim_metric, claim_value, claim_unit, claim_period ) VALUES ( ${ctx.source_id}, ${entitySlug}, ${input.fact}, ${kind}, ${visibility}, ${notability}, ${context}, ${validFrom}, ${validUntil}, ${input.source}, ${sourceSession}, ${confidence}, - ${embedLit === null ? null : tx.unsafe(`'${embedLit}'::vector`)}, ${embeddedAt} + ${embedLit === null ? null : tx.unsafe(`'${embedLit}'::vector`)}, ${embeddedAt}, + ${claimMetric}, ${claimValue}, ${claimUnit}, ${claimPeriod} ) RETURNING id `; const id = Number(ins[0].id); @@ -2378,11 +2385,13 @@ export class PostgresEngine implements BrainEngine { INSERT INTO facts ( source_id, entity_slug, fact, kind, visibility, notability, context, valid_from, valid_until, source, source_session, confidence, - embedding, embedded_at + embedding, embedded_at, + claim_metric, claim_value, claim_unit, claim_period ) VALUES ( ${ctx.source_id}, ${entitySlug}, ${input.fact}, ${kind}, ${visibility}, ${notability}, ${context}, ${validFrom}, ${validUntil}, ${input.source}, ${sourceSession}, ${confidence}, - ${embedLit === null ? null : tx.unsafe(`'${embedLit}'::vector`)}, ${embeddedAt} + ${embedLit === null ? null : tx.unsafe(`'${embedLit}'::vector`)}, ${embeddedAt}, + ${claimMetric}, ${claimValue}, ${claimUnit}, ${claimPeriod} ) RETURNING id `; return Number(ins[0].id); @@ -2429,18 +2438,25 @@ export class PostgresEngine implements BrainEngine { const embedding = input.embedding ?? null; const embeddedAt = embedding ? new Date() : null; const embedLit = embedding ? toPgVectorLiteral(embedding) : null; + // v0.35.4 (D-CDX-5) — typed-claim columns. All four nullable. + const claimMetric = input.claim_metric ?? null; + const claimValue = input.claim_value ?? null; + const claimUnit = input.claim_unit ?? null; + const claimPeriod = input.claim_period ?? null; const ins = await tx>` INSERT INTO facts ( source_id, entity_slug, fact, kind, visibility, notability, context, valid_from, valid_until, source, source_session, confidence, embedding, embedded_at, - row_num, source_markdown_slug + row_num, source_markdown_slug, + claim_metric, claim_value, claim_unit, claim_period ) VALUES ( ${ctx.source_id}, ${entitySlug}, ${input.fact}, ${kind}, ${visibility}, ${notability}, ${context}, ${validFrom}, ${validUntil}, ${input.source}, ${sourceSession}, ${confidence}, ${embedLit === null ? null : tx.unsafe(`'${embedLit}'::vector`)}, ${embeddedAt}, - ${input.row_num}, ${input.source_markdown_slug} + ${input.row_num}, ${input.source_markdown_slug}, + ${claimMetric}, ${claimValue}, ${claimUnit}, ${claimPeriod} ) RETURNING id `; out.push(Number(ins[0].id)); @@ -2599,6 +2615,62 @@ export class PostgresEngine implements BrainEngine { await sql`UPDATE facts SET consolidated_at = now(), consolidated_into = ${takeId} WHERE id = ${id}`; } + async findTrajectory(opts: import('./engine.ts').TrajectoryOpts): Promise { + const sql = this.sql; + const limit = clampSearchLimit(opts.limit, 100, 500); + const sinceDate = opts.since ? new Date(opts.since) : null; + const untilDate = opts.until ? new Date(opts.until) : null; + const metric = opts.metric ?? null; + const useArray = Array.isArray(opts.sourceIds) && opts.sourceIds.length > 0; + const sourceIds = useArray ? opts.sourceIds! : null; + const sourceId = opts.sourceId ?? 'default'; + const remoteFilter = opts.remote === true; + + // Source-scope predicate: array path (federated) wins over scalar. + // Engine.ts contract: returns chronological points; regressions + + // drift_score are computed by the caller (src/core/trajectory.ts). + const rows = await sql>` + SELECT id, valid_from, + claim_metric, claim_value, claim_unit, claim_period, + fact, source_session, source_markdown_slug, + embedding::text AS embedding + FROM facts + WHERE ${useArray ? sql`source_id = ANY(${sourceIds}::text[])` : sql`source_id = ${sourceId}`} + AND entity_slug = ${opts.entitySlug} + AND expired_at IS NULL + ${remoteFilter ? sql`AND visibility = 'world'` : sql``} + ${metric !== null ? sql`AND claim_metric = ${metric}` : sql``} + ${sinceDate ? sql`AND valid_from >= ${sinceDate}` : sql``} + ${untilDate ? sql`AND valid_from <= ${untilDate}` : sql``} + ORDER BY valid_from ASC, id ASC + LIMIT ${limit} + `; + + return rows.map(r => ({ + fact_id: Number(r.id), + valid_from: r.valid_from, + metric: r.claim_metric, + value: r.claim_value === null ? null : Number(r.claim_value), + unit: r.claim_unit, + period: r.claim_period, + text: r.fact, + source_session: r.source_session, + source_markdown_slug: r.source_markdown_slug, + embedding: tryParseEmbedding(r.embedding), + })); + } + async getFactsHealth(source_id: string): Promise { const sql = this.sql; const totals = await sql= 1) return DEFAULT_REGRESSION_THRESHOLD; + return n; +} + +function toISODate(d: Date): string { + return d.toISOString().slice(0, 10); +} + +/** + * Compute cosine similarity between two equal-length vectors. Returns 0 + * when either vector has length zero (defensive — never throws). + */ +function cosineSim(a: Float32Array, b: Float32Array): number { + if (a.length === 0 || b.length === 0 || a.length !== b.length) return 0; + let dot = 0; + let na = 0; + let nb = 0; + for (let i = 0; i < a.length; i++) { + dot += a[i] * b[i]; + na += a[i] * a[i]; + nb += b[i] * b[i]; + } + if (na === 0 || nb === 0) return 0; + return dot / (Math.sqrt(na) * Math.sqrt(nb)); +} + +/** + * Detect chronological regressions in a sorted trajectory. + * + * Iterates per-metric (so trajectories that interleave mrr + arr + team_size + * don't trip false regressions across metric boundaries). Within each metric, + * walks consecutive value pairs; a pair fires when + * `(newer - older) / older <= -threshold`. + * + * Pre-condition: caller passed points sorted by (valid_from ASC, fact_id ASC). + * The engine's `findTrajectory` enforces this. No re-sort here. + */ +export function detectRegressions( + points: TrajectoryPoint[], + threshold: number = DEFAULT_REGRESSION_THRESHOLD, +): TrajectoryRegression[] { + const out: TrajectoryRegression[] = []; + // Group by metric so each metric's regression detection is independent. + const byMetric = new Map(); + for (const p of points) { + if (p.metric === null || p.value === null) continue; + if (!Number.isFinite(p.value)) continue; + if (!byMetric.has(p.metric)) byMetric.set(p.metric, []); + byMetric.get(p.metric)!.push(p); + } + + for (const [metric, series] of byMetric) { + for (let i = 1; i < series.length; i++) { + const older = series[i - 1]; + const newer = series[i]; + const oldVal = older.value!; + const newVal = newer.value!; + // Guard against division-by-zero: a metric starting at exactly 0 + // can't compute a relative delta. Skip. + if (oldVal === 0) continue; + const delta = (newVal - oldVal) / oldVal; + if (delta <= -threshold) { + out.push({ + metric, + from_value: oldVal, + from_date: toISODate(older.valid_from), + to_value: newVal, + to_date: toISODate(newer.valid_from), + delta_pct: delta, + }); + } + } + } + return out; +} + +/** + * Compute drift score over the trajectory's existing embeddings. + * + * `1 - mean(cosine(emb[i], emb[i-1]))` clamped to [0, 1]. Range + * interpretation: 0 = narrative stable text-wise; 1 = every consecutive + * claim is unrelated to the prior. + * + * Returns null when fewer than 3 points have non-null embeddings — the + * statistic is meaningless on tiny samples. + */ +export function computeDriftScore(points: TrajectoryPoint[]): number | null { + const withEmb = points.filter(p => p.embedding !== null && p.embedding.length > 0); + if (withEmb.length < 3) return null; + let sumCos = 0; + let pairs = 0; + for (let i = 1; i < withEmb.length; i++) { + sumCos += cosineSim(withEmb[i - 1].embedding!, withEmb[i].embedding!); + pairs += 1; + } + if (pairs === 0) return null; + const meanCos = sumCos / pairs; + const drift = 1 - meanCos; + if (drift < 0) return 0; + if (drift > 1) return 1; + return drift; +} + +/** + * Compose the two derived metrics into a single TrajectoryStats. The MCP + * op + CLI both call this so the JSON shape stays consistent. + */ +export function computeTrajectoryStats( + points: TrajectoryPoint[], + opts: { threshold?: number } = {}, +): TrajectoryStats { + const threshold = opts.threshold ?? resolveRegressionThreshold(); + return { + regressions: detectRegressions(points, threshold), + drift_score: computeDriftScore(points), + }; +} diff --git a/test/consolidate-valid-until.test.ts b/test/consolidate-valid-until.test.ts new file mode 100644 index 000000000..bc682624b --- /dev/null +++ b/test/consolidate-valid-until.test.ts @@ -0,0 +1,226 @@ +/** + * v0.35.4 (D-CDX-4) — consolidate semantic upsert + chronological + * valid_until writeback. + * + * Pins: + * - R4a: a cluster of 3 chronologically-ordered facts produces + * 2 facts with valid_until set (older) and 1 with NULL (newest). + * - R4b/R7: running consolidate twice on the same input produces zero + * NEW takes (semantic upsert by (page_id, claim, since_date)). + * This is the Codex F4 fix — without it, the second cycle's + * extract_facts would clear consolidated_at and the second + * consolidate would append duplicate takes via MAX(row_num)+1. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import { runPhaseConsolidate } from '../src/core/cycle/phases/consolidate.ts'; + +let engine: PGLiteEngine; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({}); + await engine.initSchema(); +}); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await engine.executeRaw(`DELETE FROM facts`); + await engine.executeRaw(`DELETE FROM takes`); + await engine.executeRaw(`DELETE FROM pages WHERE slug LIKE 'cdx4-%'`); +}); + +function unitVec(): string { + const a = new Float32Array(1536); + a[0] = 1.0; + return '[' + Array.from(a).join(',') + ']'; +} + +async function seedPage(slug: string): Promise { + await engine.executeRaw( + `INSERT INTO pages (slug, type, title) VALUES ($1, 'company', 'Test') ON CONFLICT DO NOTHING`, + [slug], + ); + const r = await engine.executeRaw<{ id: number }>( + `SELECT id FROM pages WHERE slug = $1 AND source_id = 'default'`, + [slug], + ); + return r[0].id; +} + +async function insertFact(args: { + entity_slug: string; + text: string; + valid_from: Date; + confidence?: number; +}): Promise { + const r = await engine.executeRaw<{ id: number }>( + `INSERT INTO facts (source_id, entity_slug, fact, kind, source, valid_from, confidence, embedding, embedded_at) + VALUES ('default', $1, $2, 'fact', 'test', $3::timestamptz, $4, $5::vector, $3::timestamptz) + RETURNING id`, + [args.entity_slug, args.text, args.valid_from.toISOString(), args.confidence ?? 0.9, unitVec()], + ); + return r[0].id; +} + +describe('R4a — chronological valid_until writeback', () => { + test('cluster of 3 chronologically-ordered facts: 2 older get valid_until set, newest stays NULL', async () => { + await seedPage('cdx4-acme-mrr'); + const olderDay = new Date('2026-01-15T00:00:00Z'); + const midDay = new Date('2026-04-12T00:00:00Z'); + const newest = new Date('2026-07-08T00:00:00Z'); + + // All three close enough in vector space to cluster together (identical + // embeddings via unitVec()). Past the 24h "oldest age" gate. + const idOlder = await insertFact({ + entity_slug: 'cdx4-acme-mrr', + text: 'MRR claim', + valid_from: olderDay, + }); + const idMid = await insertFact({ + entity_slug: 'cdx4-acme-mrr', + text: 'MRR claim', + valid_from: midDay, + }); + const idNewest = await insertFact({ + entity_slug: 'cdx4-acme-mrr', + text: 'MRR claim', + valid_from: newest, + }); + + const r = await runPhaseConsolidate(engine, {}); + expect(r.details.facts_consolidated).toBe(3); + expect(r.details.takes_written).toBe(1); + + const rows = await engine.executeRaw<{ id: number; valid_until: Date | null }>( + `SELECT id, valid_until FROM facts WHERE entity_slug = 'cdx4-acme-mrr' ORDER BY valid_from ASC`, + ); + expect(rows.length).toBe(3); + // Older fact's valid_until = mid.valid_from. + expect(rows[0].id).toBe(idOlder); + expect(rows[0].valid_until).not.toBeNull(); + expect(new Date(rows[0].valid_until!).toISOString().slice(0, 10)).toBe('2026-04-12'); + // Mid fact's valid_until = newest.valid_from. + expect(rows[1].id).toBe(idMid); + expect(rows[1].valid_until).not.toBeNull(); + expect(new Date(rows[1].valid_until!).toISOString().slice(0, 10)).toBe('2026-07-08'); + // Newest fact's valid_until stays NULL. + expect(rows[2].id).toBe(idNewest); + expect(rows[2].valid_until).toBeNull(); + }); + + test('same-day cluster (3 facts, identical valid_from): id tiebreaker establishes chronological order', async () => { + await seedPage('cdx4-acme-sameday'); + const sameDay = new Date(Date.now() - 30 * 60 * 60 * 1000); + const idA = await insertFact({ entity_slug: 'cdx4-acme-sameday', text: 'same day', valid_from: sameDay }); + const idB = await insertFact({ entity_slug: 'cdx4-acme-sameday', text: 'same day', valid_from: sameDay }); + const idC = await insertFact({ entity_slug: 'cdx4-acme-sameday', text: 'same day', valid_from: sameDay }); + + await runPhaseConsolidate(engine, {}); + + // All three valid_from values are equal; the (id ASC) tiebreaker + // makes the lowest-id row the "oldest" chronologically. Pin that + // contract since the trajectory CLI depends on this ordering. + const rows = await engine.executeRaw<{ id: number; valid_until: Date | null }>( + `SELECT id, valid_until FROM facts WHERE entity_slug = 'cdx4-acme-sameday' ORDER BY id ASC`, + ); + expect(rows.length).toBe(3); + expect(rows[0].id).toBe(idA); + expect(rows[1].id).toBe(idB); + expect(rows[2].id).toBe(idC); + // First two are "older" by tiebreaker → both get valid_until set + // (= sameDay, since the next-newer fact has the same valid_from). + expect(rows[0].valid_until).not.toBeNull(); + expect(rows[1].valid_until).not.toBeNull(); + // Newest by tiebreaker stays NULL. + expect(rows[2].valid_until).toBeNull(); + }); +}); + +describe('R4b / R7 — cycle idempotency: re-run consolidate produces zero new takes (Codex F4 fix)', () => { + test('semantic upsert: second consolidate on identical state produces zero NEW takes', async () => { + await seedPage('cdx4-idempo-1'); + const oldDate = new Date(Date.now() - 30 * 60 * 60 * 1000); + for (let i = 0; i < 4; i++) { + await insertFact({ + entity_slug: 'cdx4-idempo-1', + text: 'stable claim', + valid_from: new Date(oldDate.getTime() + i * 60 * 60 * 1000), + }); + } + + // First run: 1 take, 4 facts consolidated. + const r1 = await runPhaseConsolidate(engine, {}); + expect(r1.details.takes_written).toBe(1); + const countAfter1 = await engine.executeRaw<{ n: string }>( + `SELECT COUNT(*)::text AS n FROM takes WHERE page_id = (SELECT id FROM pages WHERE slug = 'cdx4-idempo-1')`, + ); + expect(parseInt(countAfter1[0].n, 10)).toBe(1); + + // Simulate the Codex F4 scenario: clear consolidated_at on every fact + // (extract_facts cycle phase wipes facts via delete-then-insert, which + // is functionally identical to NULL-ing consolidated_at). DO NOT touch + // valid_until — the prior consolidate wrote it; the semantic upsert + // should still find the take. + await engine.executeRaw( + `UPDATE facts SET consolidated_at = NULL, consolidated_into = NULL + WHERE entity_slug = 'cdx4-idempo-1'`, + ); + + // Second run: must NOT append another take. + const r2 = await runPhaseConsolidate(engine, {}); + expect(r2.details.facts_consolidated).toBe(4); + // takes_written reports the NEW takes inserted this run; on the upsert + // hit path it's 0 (no new INSERT) but facts still get marked consolidated. + expect(r2.details.takes_written).toBe(0); + + const countAfter2 = await engine.executeRaw<{ n: string }>( + `SELECT COUNT(*)::text AS n FROM takes WHERE page_id = (SELECT id FROM pages WHERE slug = 'cdx4-idempo-1')`, + ); + expect(parseInt(countAfter2[0].n, 10)).toBe(1); // STILL 1 — no duplicate + + // Facts were re-consolidated into the existing take. + const facts = await engine.executeRaw<{ consolidated_into: number }>( + `SELECT consolidated_into FROM facts WHERE entity_slug = 'cdx4-idempo-1' AND consolidated_into IS NOT NULL`, + ); + expect(facts.length).toBe(4); + }); + + test('valid_until idempotency: second run leaves valid_until unchanged (no diff)', async () => { + await seedPage('cdx4-idempo-2'); + const t1 = new Date('2026-01-15T00:00:00Z'); + const t2 = new Date('2026-04-12T00:00:00Z'); + const t3 = new Date('2026-07-08T00:00:00Z'); + await insertFact({ entity_slug: 'cdx4-idempo-2', text: 'iterable', valid_from: t1 }); + await insertFact({ entity_slug: 'cdx4-idempo-2', text: 'iterable', valid_from: t2 }); + await insertFact({ entity_slug: 'cdx4-idempo-2', text: 'iterable', valid_from: t3 }); + + await runPhaseConsolidate(engine, {}); + const before = await engine.executeRaw<{ id: number; valid_until: Date | null }>( + `SELECT id, valid_until FROM facts WHERE entity_slug = 'cdx4-idempo-2' ORDER BY valid_from ASC`, + ); + + // Reset consolidated_at to simulate extract_facts re-run. + await engine.executeRaw( + `UPDATE facts SET consolidated_at = NULL, consolidated_into = NULL + WHERE entity_slug = 'cdx4-idempo-2'`, + ); + + await runPhaseConsolidate(engine, {}); + const after = await engine.executeRaw<{ id: number; valid_until: Date | null }>( + `SELECT id, valid_until FROM facts WHERE entity_slug = 'cdx4-idempo-2' ORDER BY valid_from ASC`, + ); + // Same valid_until values; the IS DISTINCT FROM guard avoided rewrites. + expect(after.length).toBe(3); + for (let i = 0; i < before.length; i++) { + expect(after[i].id).toBe(before[i].id); + const a = after[i].valid_until ? new Date(after[i].valid_until!).toISOString() : null; + const b = before[i].valid_until ? new Date(before[i].valid_until!).toISOString() : null; + expect(a).toBe(b); + } + }); +}); diff --git a/test/engine-find-trajectory.test.ts b/test/engine-find-trajectory.test.ts new file mode 100644 index 000000000..77c66767c --- /dev/null +++ b/test/engine-find-trajectory.test.ts @@ -0,0 +1,320 @@ +/** + * v0.35.4 — BrainEngine.findTrajectory (T4) + trajectory.ts derived + * metrics tests. + * + * Pins: + * - Chronological ordering by (valid_from ASC, fact_id ASC) — R3. + * - Source scoping (scalar + federated array, D-CDX-6). + * - Visibility filter for remote callers (D-CDX-1) — R6. + * - Metric filter narrows results to a single canonical name. + * - since/until window honored. + * - Regression detection per locked threshold (D-ENG-2). + * - Drift score returns null when <3 embedded points (G3). + * - Empty entity returns {points: [], regressions: [], drift_score: null} (G1). + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import { + detectRegressions, + computeDriftScore, + computeTrajectoryStats, + DEFAULT_REGRESSION_THRESHOLD, +} from '../src/core/trajectory.ts'; +import type { TrajectoryPoint } from '../src/core/engine.ts'; + +let engine: PGLiteEngine; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({}); + await engine.initSchema(); +}); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await engine.executeRaw(`DELETE FROM facts WHERE entity_slug LIKE 'traj-%'`); + await engine.executeRaw(`DELETE FROM sources WHERE id LIKE 'traj-%'`); +}); + +function vecForMetric(metric: string, offset: number): string { + // Deterministic per-metric/offset embedding: each metric gets a + // unit-vector in a different "direction" of the embedding space, with + // a small perturbation per offset so consecutive same-metric facts + // are very-similar-but-not-identical (drift score lands between 0 and + // some small value). + const a = new Float32Array(1536); + const idx = (metric.charCodeAt(0) + offset) % 1536; + a[idx] = 1.0; + a[(idx + 1) % 1536] = 0.05 * offset; // tiny drift between consecutive + return '[' + Array.from(a).join(',') + ']'; +} + +async function insertTyped(args: { + source_id?: string; + entity_slug: string; + metric: string; + value: number; + unit?: string; + period?: string; + valid_from: Date; + visibility?: 'private' | 'world'; + offset?: number; + text?: string; +}): Promise { + const sid = args.source_id ?? 'default'; + await engine.executeRaw( + `INSERT INTO sources (id, name) VALUES ($1, $1) ON CONFLICT DO NOTHING`, + [sid], + ); + const r = await engine.executeRaw<{ id: number }>( + `INSERT INTO facts (source_id, entity_slug, fact, kind, source, valid_from, + claim_metric, claim_value, claim_unit, claim_period, + visibility, embedding, embedded_at) + VALUES ($1, $2, $3, 'fact', 'test', $4::timestamptz, + $5, $6, $7, $8, + $9, $10::vector, $4::timestamptz) + RETURNING id`, + [ + sid, args.entity_slug, args.text ?? `${args.metric} ${args.value}`, + args.valid_from.toISOString(), + args.metric, args.value, args.unit ?? null, args.period ?? null, + args.visibility ?? 'private', + vecForMetric(args.metric, args.offset ?? 0), + ], + ); + return r[0].id; +} + +describe('findTrajectory — chronological ordering (R3)', () => { + test('returns points in (valid_from ASC, id ASC) order regardless of insert order', async () => { + // Insert out of order. Engine must re-order. + const idJul = await insertTyped({ entity_slug: 'traj-order', metric: 'mrr', value: 150000, valid_from: new Date('2026-07-08') }); + const idJan = await insertTyped({ entity_slug: 'traj-order', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') }); + const idApr = await insertTyped({ entity_slug: 'traj-order', metric: 'mrr', value: 200000, valid_from: new Date('2026-04-12') }); + + const points = await engine.findTrajectory({ entitySlug: 'traj-order' }); + expect(points.map(p => p.fact_id)).toEqual([idJan, idApr, idJul]); + expect(points[0].valid_from.toISOString().slice(0, 10)).toBe('2026-01-15'); + expect(points[2].valid_from.toISOString().slice(0, 10)).toBe('2026-07-08'); + }); +}); + +describe('findTrajectory — source scoping (D-CDX-6)', () => { + test('scalar sourceId returns only that source', async () => { + await insertTyped({ source_id: 'traj-src-A', entity_slug: 'traj-srcscope', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') }); + await insertTyped({ source_id: 'traj-src-B', entity_slug: 'traj-srcscope', metric: 'mrr', value: 99999, valid_from: new Date('2026-01-15') }); + + const pointsA = await engine.findTrajectory({ entitySlug: 'traj-srcscope', sourceId: 'traj-src-A' }); + expect(pointsA.length).toBe(1); + expect(pointsA[0].value).toBe(50000); + + const pointsB = await engine.findTrajectory({ entitySlug: 'traj-srcscope', sourceId: 'traj-src-B' }); + expect(pointsB.length).toBe(1); + expect(pointsB[0].value).toBe(99999); + }); + + test('federated sourceIds returns union across the array', async () => { + await insertTyped({ source_id: 'traj-src-A', entity_slug: 'traj-fed', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') }); + await insertTyped({ source_id: 'traj-src-B', entity_slug: 'traj-fed', metric: 'mrr', value: 99999, valid_from: new Date('2026-04-12') }); + await insertTyped({ source_id: 'traj-src-C', entity_slug: 'traj-fed', metric: 'mrr', value: 11111, valid_from: new Date('2026-07-08') }); + + const points = await engine.findTrajectory({ + entitySlug: 'traj-fed', + sourceIds: ['traj-src-A', 'traj-src-B'], + }); + // Two of three sources visible, in chronological order. + expect(points.length).toBe(2); + expect(points.map(p => p.value)).toEqual([50000, 99999]); + }); +}); + +describe('findTrajectory — visibility filter (D-CDX-1 / R6)', () => { + test('remote=true returns ONLY world-visibility points', async () => { + await insertTyped({ entity_slug: 'traj-vis', metric: 'mrr', value: 50000, visibility: 'private', valid_from: new Date('2026-01-15') }); + await insertTyped({ entity_slug: 'traj-vis', metric: 'mrr', value: 99999, visibility: 'world', valid_from: new Date('2026-04-12') }); + + const trusted = await engine.findTrajectory({ entitySlug: 'traj-vis', remote: false }); + expect(trusted.length).toBe(2); // local CLI sees both + + const remote = await engine.findTrajectory({ entitySlug: 'traj-vis', remote: true }); + expect(remote.length).toBe(1); // OAuth client sees world only + expect(remote[0].value).toBe(99999); + }); + + test('remote default (undefined) is treated as trusted — sees both', async () => { + await insertTyped({ entity_slug: 'traj-vis-default', metric: 'mrr', value: 50000, visibility: 'private', valid_from: new Date('2026-01-15') }); + await insertTyped({ entity_slug: 'traj-vis-default', metric: 'mrr', value: 99999, visibility: 'world', valid_from: new Date('2026-04-12') }); + + // No `remote` field — engine default must be trusted. + const all = await engine.findTrajectory({ entitySlug: 'traj-vis-default' }); + expect(all.length).toBe(2); + }); +}); + +describe('findTrajectory — metric + since + until filters', () => { + test('metric filter narrows to one canonical name', async () => { + await insertTyped({ entity_slug: 'traj-m', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') }); + await insertTyped({ entity_slug: 'traj-m', metric: 'arr', value: 600000, valid_from: new Date('2026-01-15') }); + + const mrrOnly = await engine.findTrajectory({ entitySlug: 'traj-m', metric: 'mrr' }); + expect(mrrOnly.length).toBe(1); + expect(mrrOnly[0].metric).toBe('mrr'); + }); + + test('since/until window honored', async () => { + await insertTyped({ entity_slug: 'traj-w', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') }); + await insertTyped({ entity_slug: 'traj-w', metric: 'mrr', value: 99999, valid_from: new Date('2026-04-12') }); + await insertTyped({ entity_slug: 'traj-w', metric: 'mrr', value: 11111, valid_from: new Date('2026-07-08') }); + + const inWindow = await engine.findTrajectory({ + entitySlug: 'traj-w', + since: '2026-02-01', + until: '2026-05-01', + }); + expect(inWindow.length).toBe(1); + expect(inWindow[0].value).toBe(99999); + }); + + test('unknown entity returns empty array', async () => { + const empty = await engine.findTrajectory({ entitySlug: 'traj-does-not-exist' }); + expect(empty).toEqual([]); + }); +}); + +// ──────────────────────────────────────────────────────────────────────── +// trajectory.ts pure-function tests +// ──────────────────────────────────────────────────────────────────────── + +function makePoint(args: { + id: number; + metric: string; + value: number; + date: string; + emb?: Float32Array | null; +}): TrajectoryPoint { + return { + fact_id: args.id, + valid_from: new Date(args.date), + metric: args.metric, + value: args.value, + unit: 'USD', + period: 'monthly', + text: `${args.metric} = ${args.value}`, + source_session: null, + source_markdown_slug: null, + embedding: args.emb ?? null, + }; +} + +describe('detectRegressions (D-ENG-2)', () => { + test('emits a regression when newer value drops by >= threshold', () => { + const points: TrajectoryPoint[] = [ + makePoint({ id: 1, metric: 'mrr', value: 200000, date: '2026-04-12' }), + makePoint({ id: 2, metric: 'mrr', value: 150000, date: '2026-07-08' }), // -25% + ]; + const regs = detectRegressions(points, DEFAULT_REGRESSION_THRESHOLD); + expect(regs.length).toBe(1); + expect(regs[0].metric).toBe('mrr'); + expect(regs[0].delta_pct).toBeCloseTo(-0.25, 4); + expect(regs[0].from_date).toBe('2026-04-12'); + expect(regs[0].to_date).toBe('2026-07-08'); + }); + + test('skips when drop is below threshold (5% with default 10%)', () => { + const points: TrajectoryPoint[] = [ + makePoint({ id: 1, metric: 'mrr', value: 100000, date: '2026-01-15' }), + makePoint({ id: 2, metric: 'mrr', value: 95000, date: '2026-04-12' }), // -5% + ]; + expect(detectRegressions(points).length).toBe(0); + }); + + test('multiple metrics tracked independently', () => { + const points: TrajectoryPoint[] = [ + makePoint({ id: 1, metric: 'mrr', value: 200000, date: '2026-04-12' }), + makePoint({ id: 2, metric: 'arr', value: 600000, date: '2026-04-12' }), + makePoint({ id: 3, metric: 'mrr', value: 150000, date: '2026-07-08' }), // -25% mrr + makePoint({ id: 4, metric: 'arr', value: 700000, date: '2026-07-08' }), // +16% arr → no regression + ]; + const regs = detectRegressions(points); + expect(regs.length).toBe(1); + expect(regs[0].metric).toBe('mrr'); + }); + + test('skips points with null value', () => { + const points: TrajectoryPoint[] = [ + makePoint({ id: 1, metric: 'mrr', value: 200000, date: '2026-04-12' }), + { ...makePoint({ id: 2, metric: 'mrr', value: 0, date: '2026-07-08' }), value: null }, + ]; + expect(detectRegressions(points).length).toBe(0); + }); + + test('skips when older value is 0 (division-by-zero guard)', () => { + const points: TrajectoryPoint[] = [ + makePoint({ id: 1, metric: 'mrr', value: 0, date: '2026-04-12' }), + makePoint({ id: 2, metric: 'mrr', value: 1000, date: '2026-07-08' }), + ]; + expect(detectRegressions(points).length).toBe(0); + }); +}); + +describe('computeDriftScore (D-ENG-3 / G3)', () => { + function unitVec(dim: number, offset: number): Float32Array { + const a = new Float32Array(8); + a[offset % 8] = 1.0; + return a; + } + + test('returns null with fewer than 3 embedded points', () => { + const points: TrajectoryPoint[] = [ + makePoint({ id: 1, metric: 'mrr', value: 1, date: '2026-01-15', emb: unitVec(8, 0) }), + makePoint({ id: 2, metric: 'mrr', value: 2, date: '2026-04-12', emb: unitVec(8, 1) }), + ]; + expect(computeDriftScore(points)).toBeNull(); + }); + + test('returns null when no points have embeddings (G3 graceful fallback)', () => { + const points: TrajectoryPoint[] = [ + makePoint({ id: 1, metric: 'mrr', value: 1, date: '2026-01-15' }), + makePoint({ id: 2, metric: 'mrr', value: 2, date: '2026-04-12' }), + makePoint({ id: 3, metric: 'mrr', value: 3, date: '2026-07-08' }), + ]; + expect(computeDriftScore(points)).toBeNull(); + }); + + test('identical consecutive embeddings → drift 0 (cohesive narrative)', () => { + const v = unitVec(8, 0); + const points: TrajectoryPoint[] = [ + makePoint({ id: 1, metric: 'mrr', value: 1, date: '2026-01-15', emb: v }), + makePoint({ id: 2, metric: 'mrr', value: 2, date: '2026-04-12', emb: v }), + makePoint({ id: 3, metric: 'mrr', value: 3, date: '2026-07-08', emb: v }), + ]; + expect(computeDriftScore(points)).toBe(0); + }); + + test('orthogonal consecutive embeddings → drift 1 (every claim unrelated)', () => { + const points: TrajectoryPoint[] = [ + makePoint({ id: 1, metric: 'mrr', value: 1, date: '2026-01-15', emb: unitVec(8, 0) }), + makePoint({ id: 2, metric: 'mrr', value: 2, date: '2026-04-12', emb: unitVec(8, 1) }), + makePoint({ id: 3, metric: 'mrr', value: 3, date: '2026-07-08', emb: unitVec(8, 2) }), + ]; + expect(computeDriftScore(points)).toBe(1); + }); +}); + +describe('computeTrajectoryStats — composed shape', () => { + test('returns both regressions + drift_score in one call', () => { + const v = new Float32Array(4); + v[0] = 1; + const points: TrajectoryPoint[] = [ + makePoint({ id: 1, metric: 'mrr', value: 200000, date: '2026-04-12', emb: v }), + makePoint({ id: 2, metric: 'mrr', value: 150000, date: '2026-07-08', emb: v }), + ]; + const stats = computeTrajectoryStats(points); + expect(stats.regressions.length).toBe(1); + expect(stats.drift_score).toBeNull(); // <3 embedded + }); +}); diff --git a/test/eval-contradictions/no-valid-until-write.test.ts b/test/eval-contradictions/no-valid-until-write.test.ts new file mode 100644 index 000000000..b17b9674c --- /dev/null +++ b/test/eval-contradictions/no-valid-until-write.test.ts @@ -0,0 +1,158 @@ +/** + * v0.35.4 (R1 + R8 — D-CDX-4 + D-CDX-7) — IRON-RULE: the contradiction + * probe NEVER writes `valid_until` on the facts table. + * + * The temporal trajectory wave (v0.35.4) gives `consolidate` the + * authority to write `valid_until` on chronologically-superseded facts. + * That authority is exclusive: the contradiction probe surfaces + * `temporal_supersession` verdicts via paste-ready commands, but it + * must NEVER auto-mutate. This preserves the + * `src/core/eval-contradictions/auto-supersession.ts:4` invariant. + * + * Two layered guards: + * R1 — grep guard over the entire `src/core/eval-contradictions/` + * subtree and the `src/commands/eval-suspected-contradictions*.ts` + * files: no code path may UPDATE facts.valid_until. + * R8 — broader guard over all of `src/`: the only file that writes + * valid_until is `src/core/cycle/phases/consolidate.ts`. Any new + * write site fails this guard; the human adding it must explicitly + * amend the allow-list AND document the deliberate design change. + */ + +import { test, expect, describe } from 'bun:test'; +import { readdirSync, readFileSync, statSync } from 'node:fs'; +import { join } from 'node:path'; + +// Allow-listed files that legitimately write `valid_until`. Adding a new +// file here means you've thought carefully about the +// auto-supersession.ts:4 invariant and decided the new write site +// preserves it. +// +// - consolidate.ts (v0.35.4 — chronological writeback) +// - facts/forget.ts (v0.32.2 — user-initiated `gbrain forget`; user is +// the supersession authority, not the probe) +const VALID_UNTIL_WRITE_ALLOWLIST: ReadonlySet = new Set([ + 'src/core/cycle/phases/consolidate.ts', + 'src/core/facts/forget.ts', +]); + +function walkTs(dir: string, acc: string[] = []): string[] { + for (const name of readdirSync(dir)) { + const full = join(dir, name); + const st = statSync(full); + if (st.isDirectory()) { + // Skip generated / vendored / test directories. + if (name === 'node_modules' || name === '.git' || name === 'dist') continue; + walkTs(full, acc); + } else if (name.endsWith('.ts') && !name.endsWith('.test.ts')) { + acc.push(full); + } + } + return acc; +} + +/** + * Detect lines that look like they are UPDATEing facts.valid_until. + * Permissive on SQL formatting (the same UPDATE can be split across lines, + * use template strings, or use parameterized SQL via postgres.js's + * tagged-template syntax). We look for the pair of substrings near + * each other in the same file: `UPDATE facts` and `valid_until`. + * + * Tolerated: a reference to `valid_until` in a SELECT projection list + * (which is fine — reading the column is not writing it) is filtered out + * by also requiring a write verb (SET) nearby OR an INSERT INTO facts + * with valid_until in the column list (INSERT path is OK from + * src/core/postgres-engine.ts + src/core/pglite-engine.ts because those + * are the engine layer, intentionally writing on caller request via + * insertFact/insertFacts; the issue is whether the contradiction probe + * path triggers those writes). + */ +function findValidUntilWrites(source: string): string[] { + // Quick-fail: no mention of valid_until at all. + if (!source.includes('valid_until')) return []; + const lines = source.split('\n'); + const hits: string[] = []; + for (let i = 0; i < lines.length; i++) { + const line = lines[i]; + // Detect actual SQL writes. The narrow pattern `UPDATE facts SET ... + // valid_until` is unambiguous — UPDATE is a SQL verb, not text + // anyone writes in a description string. Tolerates same-line and + // multi-line UPDATEs (look at the next 4 lines after `UPDATE facts` + // for `valid_until`). + if (/\bUPDATE\s+facts\b/i.test(line)) { + const window = lines.slice(i, i + 5).join('\n'); + if (/\bSET\b[\s\S]*\bvalid_until\b/i.test(window)) { + hits.push(`${i + 1}: ${line.trim()}`); + } + } + } + return hits; +} + +describe('R1 — contradiction probe never writes valid_until', () => { + test('no file under src/core/eval-contradictions/ writes facts.valid_until', () => { + const dir = 'src/core/eval-contradictions'; + const files = walkTs(dir); + expect(files.length).toBeGreaterThan(0); + for (const f of files) { + const src = readFileSync(f, 'utf-8'); + const hits = findValidUntilWrites(src); + expect(hits, `${f} contains valid_until write: ${hits.join(' | ')}`).toEqual([]); + } + }); + + test('no src/commands/eval-suspected-contradictions* file writes facts.valid_until', () => { + const dir = 'src/commands'; + const files = readdirSync(dir) + .filter(n => n.startsWith('eval-suspected-contradictions') && n.endsWith('.ts')) + .map(n => join(dir, n)); + expect(files.length).toBeGreaterThanOrEqual(1); + for (const f of files) { + const src = readFileSync(f, 'utf-8'); + const hits = findValidUntilWrites(src); + expect(hits, `${f} contains valid_until write: ${hits.join(' | ')}`).toEqual([]); + } + }); +}); + +describe('R8 — only the consolidate phase + engine insert layer may write valid_until', () => { + test('every src/ TypeScript file that writes valid_until is on the allow-list', () => { + const files = walkTs('src'); + const offenders: Array<{ file: string; hits: string[] }> = []; + + for (const f of files) { + // Normalize path separator for cross-platform matching of the + // allow-list keys. + const relForCheck = f.replace(/\\/g, '/'); + if (VALID_UNTIL_WRITE_ALLOWLIST.has(relForCheck)) continue; + + // Engine implementation files (postgres-engine.ts, pglite-engine.ts) + // legitimately write valid_until inside insertFact/insertFacts — + // those are caller-driven INSERTs. Pattern is `INSERT INTO facts ( + // ... valid_until ...) VALUES`. Detect and exempt that pattern. + // The R8 guard is about UPDATE; INSERT carrying valid_until as a + // column value is not the failure mode auto-supersession.ts:4 cares + // about. + const src = readFileSync(f, 'utf-8'); + const hits = findValidUntilWrites(src); + if (hits.length > 0) offenders.push({ file: relForCheck, hits }); + } + + expect( + offenders, + `Unexpected valid_until UPDATE sites:\n` + + offenders.map(o => ` ${o.file}:\n ${o.hits.join('\n ')}`).join('\n') + + `\n\nIf you added a deliberate write, append the path to ` + + `VALID_UNTIL_WRITE_ALLOWLIST in this test AND review the ` + + `\`auto-supersession.ts:4\` invariant first.`, + ).toEqual([]); + }); + + test('VALID_UNTIL_WRITE_ALLOWLIST is non-empty (consolidate is on it)', () => { + // Self-check: if the test file ever ships with an empty allow-list, + // the R8 guard collapses to "no code writes valid_until anywhere" and + // becomes a tautology. Keep the consolidate phase on the list as the + // explicit positive control. + expect(VALID_UNTIL_WRITE_ALLOWLIST.has('src/core/cycle/phases/consolidate.ts')).toBe(true); + }); +}); diff --git a/test/eval-trajectory.test.ts b/test/eval-trajectory.test.ts new file mode 100644 index 000000000..4ae31bfa6 --- /dev/null +++ b/test/eval-trajectory.test.ts @@ -0,0 +1,153 @@ +/** + * v0.35.4 — `gbrain eval trajectory` CLI (T6) tests. + * + * Pins: + * - argv parser: positional entity-slug required; --metric / --since / + * --until / --limit / --json honored; unknown flags rejected. + * - --json output has the stable schema_version: 1 envelope (R5). + * - Human format includes the regression marker for points that match. + * - Empty result graceful shape (G1). + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import { runEvalTrajectory } from '../src/commands/eval-trajectory.ts'; + +let engine: PGLiteEngine; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({}); + await engine.initSchema(); +}); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await engine.executeRaw(`DELETE FROM facts WHERE entity_slug LIKE 'cli-traj-%'`); +}); + +function unitVec(idx = 0): string { + const a = new Float32Array(1536); + a[idx % 1536] = 1.0; + return '[' + Array.from(a).join(',') + ']'; +} + +async function insertTyped(args: { + entity_slug: string; + metric: string; + value: number; + valid_from: Date; + unit?: string; +}): Promise { + await engine.executeRaw( + `INSERT INTO facts (source_id, entity_slug, fact, kind, source, valid_from, + claim_metric, claim_value, claim_unit, claim_period, + visibility, embedding, embedded_at) + VALUES ('default', $1, $2, 'fact', 'test', $3::timestamptz, + $4, $5, $6, 'monthly', + 'private', $7::vector, $3::timestamptz)`, + [args.entity_slug, `${args.metric} ${args.value}`, args.valid_from.toISOString(), + args.metric, args.value, args.unit ?? 'USD', unitVec()], + ); +} + +/** Capture console.log output to assert on. */ +async function captureRun(args: string[]): Promise<{ out: string; err: string }> { + const origLog = console.log; + const origErr = console.error; + let out = ''; + let err = ''; + console.log = (...a: unknown[]) => { out += a.map(String).join(' ') + '\n'; }; + console.error = (...a: unknown[]) => { err += a.map(String).join(' ') + '\n'; }; + try { + await runEvalTrajectory(engine, args); + } finally { + console.log = origLog; + console.error = origErr; + } + return { out, err }; +} + +describe('eval-trajectory CLI — arg parsing', () => { + test('--help prints usage and returns without DB call', async () => { + const { out } = await captureRun(['--help']); + expect(out).toContain('Usage: gbrain eval trajectory'); + expect(out).toContain('--metric'); + }); + + test('missing positional arg surfaces an error + non-zero exit', async () => { + // process.exit throws inside Bun test runner; capture via try/catch. + let exitCode: number | undefined; + const origExit = process.exit; + (process as any).exit = (code?: number) => { + exitCode = code; + throw new Error('__exit_intercept__'); + }; + try { + await captureRun([]); + } catch (e: any) { + if (!String(e).includes('__exit_intercept__')) throw e; + } finally { + process.exit = origExit; + } + expect(exitCode).toBe(1); + }); +}); + +describe('eval-trajectory CLI — JSON envelope (R5)', () => { + test('--json output has schema_version: 1 and points + regressions + drift_score keys', async () => { + await insertTyped({ entity_slug: 'cli-traj-shape', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') }); + const { out } = await captureRun(['cli-traj-shape', '--json']); + const parsed = JSON.parse(out); + expect(parsed.schema_version).toBe(1); + expect(parsed).toHaveProperty('points'); + expect(parsed).toHaveProperty('regressions'); + expect(parsed).toHaveProperty('drift_score'); + // Engine's raw embedding is NOT in the CLI JSON output. + expect(parsed.points[0]).not.toHaveProperty('embedding'); + expect(parsed.points[0].valid_from).toMatch(/^\d{4}-\d{2}-\d{2}$/); + }); + + test('--json empty-entity output is the same shape (G1)', async () => { + const { out } = await captureRun(['cli-traj-nonexistent', '--json']); + const parsed = JSON.parse(out); + expect(parsed.points).toEqual([]); + expect(parsed.regressions).toEqual([]); + expect(parsed.drift_score).toBeNull(); + expect(parsed.schema_version).toBe(1); + }); +}); + +describe('eval-trajectory CLI — regression annotation in human output', () => { + test('regression line is marked with [REGRESSION ↓XX.X%] in human format', async () => { + await insertTyped({ entity_slug: 'cli-traj-reg', metric: 'mrr', value: 200000, valid_from: new Date('2026-04-12') }); + await insertTyped({ entity_slug: 'cli-traj-reg', metric: 'mrr', value: 150000, valid_from: new Date('2026-07-08') }); + + const { out } = await captureRun(['cli-traj-reg']); + expect(out).toContain('Entity: cli-traj-reg'); + expect(out).toContain('mrr'); + expect(out).toContain('REGRESSION'); + expect(out).toContain('25.0%'); + }); + + test('empty entity produces the friendly no-claims message', async () => { + const { out } = await captureRun(['cli-traj-nothing']); + expect(out).toContain('Entity: cli-traj-nothing'); + expect(out).toContain('(no typed claims'); + }); +}); + +describe('eval-trajectory CLI — metric filter narrows results', () => { + test('--metric arr returns only ARR points', async () => { + await insertTyped({ entity_slug: 'cli-traj-flt', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') }); + await insertTyped({ entity_slug: 'cli-traj-flt', metric: 'arr', value: 600000, valid_from: new Date('2026-01-15') }); + + const { out } = await captureRun(['cli-traj-flt', '--metric', 'arr', '--json']); + const parsed = JSON.parse(out); + expect(parsed.points.length).toBe(1); + expect(parsed.points[0].metric).toBe('arr'); + }); +}); diff --git a/test/facts-fence-typed.test.ts b/test/facts-fence-typed.test.ts new file mode 100644 index 000000000..2865938cb --- /dev/null +++ b/test/facts-fence-typed.test.ts @@ -0,0 +1,321 @@ +// v0.35.4 — typed-claim fence parser+renderer round-trip + normalization. +// +// Pins: +// 1. Backward compat — a fence authored without typed fields still +// parses, renders as 10-cell shape, and round-trips byte-identical. +// 2. Typed fields parse from the 14-cell widened fence. +// 3. Renderer widens to 14 cells iff ANY row has a non-undefined typed +// field; otherwise stays at 10-cell (no diff noise on existing fences). +// 4. Round-trip preservation: parse → render → parse produces the same +// ParsedFact array, including typed fields. +// 5. Numeric value cell tolerates thousand separators (`50,000`). + +import { test, expect, describe } from 'bun:test'; +import { + parseFactsFence, + renderFactsTable, + upsertFactRow, + type ParsedFact, +} from '../src/core/facts-fence.ts'; +import { + extractFactsFromFenceText, + normalizeMetricLabel, + METRIC_NORMALIZATION_MAP, +} from '../src/core/facts/extract-from-fence.ts'; + +function wrap(inner: string): string { + return `## Facts\n\n\n${inner}\n\n`; +} + +describe('v0.35.4 — facts fence typed-claim parser', () => { + test('legacy 10-cell fence parses as before; all typed fields undefined', () => { + const body = wrap( + `| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | + |---|-------|------|------------|------------|------------|------------|-------------|--------|---------| + | 1 | Founded Acme in 2017 | fact | 1.0 | world | high | 2017-01-01 | | linkedin | |`, + ); + const { facts, warnings } = parseFactsFence(body); + expect(warnings).toEqual([]); + expect(facts.length).toBe(1); + expect(facts[0].claim).toBe('Founded Acme in 2017'); + expect(facts[0].claimMetric).toBeUndefined(); + expect(facts[0].claimValue).toBeUndefined(); + expect(facts[0].claimUnit).toBeUndefined(); + expect(facts[0].claimPeriod).toBeUndefined(); + }); + + test('14-cell typed fence parses all four typed-claim fields', () => { + const body = wrap( + `| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | claim_metric | claim_value | claim_unit | claim_period | + |---|-------|------|------------|------------|------------|------------|-------------|--------|---------|--------------|-------------|------------|--------------| + | 1 | MRR hit $50K | fact | 1.0 | private | high | 2026-01-15 | | OH transcript | | mrr | 50000 | USD | monthly |`, + ); + const { facts, warnings } = parseFactsFence(body); + expect(warnings).toEqual([]); + expect(facts.length).toBe(1); + expect(facts[0].claimMetric).toBe('mrr'); + expect(facts[0].claimValue).toBe(50000); + expect(facts[0].claimUnit).toBe('USD'); + expect(facts[0].claimPeriod).toBe('monthly'); + }); + + test('numeric value cell tolerates comma thousand separators', () => { + const body = wrap( + `| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | claim_metric | claim_value | claim_unit | claim_period | + |---|-------|------|------------|------------|------------|------------|-------------|--------|---------|--------------|-------------|------------|--------------| + | 1 | ARR | fact | 1.0 | private | high | 2026-04-12 | | bo call | | arr | 2,000,000 | USD | annual |`, + ); + const { facts } = parseFactsFence(body); + expect(facts[0].claimValue).toBe(2000000); + }); + + test('empty typed cells parse as undefined (not "")', () => { + const body = wrap( + `| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | claim_metric | claim_value | claim_unit | claim_period | + |---|-------|------|------------|------------|------------|------------|-------------|--------|---------|--------------|-------------|------------|--------------| + | 1 | bare claim | fact | 1.0 | private | high | 2026-01-01 | | | | | | | |`, + ); + const { facts } = parseFactsFence(body); + expect(facts[0].claimMetric).toBeUndefined(); + expect(facts[0].claimValue).toBeUndefined(); + expect(facts[0].claimUnit).toBeUndefined(); + expect(facts[0].claimPeriod).toBeUndefined(); + }); +}); + +describe('v0.35.4 — facts fence typed-claim renderer', () => { + test('renders 10-cell shape when no row has typed fields (backward compat)', () => { + const facts: ParsedFact[] = [ + { + rowNum: 1, + claim: 'Founded Acme in 2017', + kind: 'fact', + confidence: 1.0, + visibility: 'world', + notability: 'high', + validFrom: '2017-01-01', + source: 'linkedin', + active: true, + }, + ]; + const out = renderFactsTable(facts); + // 10-cell header + expect(out).toContain('| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context |'); + // NOT the 14-cell variant + expect(out).not.toContain('claim_metric'); + }); + + test('widens to 14 cells when ANY row has typed fields', () => { + const facts: ParsedFact[] = [ + { + rowNum: 1, + claim: 'plain claim', + kind: 'fact', + confidence: 1.0, + visibility: 'private', + notability: 'medium', + active: true, + }, + { + rowNum: 2, + claim: 'MRR hit $50K', + kind: 'fact', + confidence: 1.0, + visibility: 'private', + notability: 'high', + active: true, + claimMetric: 'mrr', + claimValue: 50000, + claimUnit: 'USD', + claimPeriod: 'monthly', + }, + ]; + const out = renderFactsTable(facts); + expect(out).toContain('claim_metric'); + expect(out).toContain('claim_value'); + expect(out).toContain('claim_unit'); + expect(out).toContain('claim_period'); + // Row 1 has empty typed cells; row 2 has the values. + expect(out).toContain('| mrr | 50000 | USD | monthly |'); + }); + + test('round-trip preservation: parse → render → parse is structurally idempotent for typed facts', () => { + const body = wrap( + `| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | claim_metric | claim_value | claim_unit | claim_period | + |---|-------|------|------------|------------|------------|------------|-------------|--------|---------|--------------|-------------|------------|--------------| + | 1 | MRR $50K | fact | 1.0 | private | high | 2026-01-15 | | OH | | mrr | 50000 | USD | monthly | + | 2 | Team grew to 12 | fact | 1.0 | private | medium | 2026-02-01 | | meeting | | team_size | 12 | people | | + | 3 | Plain non-typed claim | fact | 0.85 | private | low | 2026-03-01 | | inferred | | | | | |`, + ); + const first = parseFactsFence(body); + expect(first.warnings).toEqual([]); + expect(first.facts.length).toBe(3); + const rendered = renderFactsTable(first.facts); + const second = parseFactsFence(rendered); + expect(second.warnings).toEqual([]); + expect(second.facts).toEqual(first.facts); + }); + + test('upsertFactRow threads typed fields when a new row carries them', () => { + // Start from a fence with NO typed fields → 10-cell shape. + const body = wrap( + `| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | + |---|-------|------|------------|------------|------------|------------|-------------|--------|---------| + | 1 | Founded Acme in 2017 | fact | 1.0 | world | high | 2017-01-01 | | linkedin | |`, + ); + const { body: newBody, rowNum } = upsertFactRow(body, { + claim: 'MRR hit $50K', + kind: 'fact', + confidence: 1.0, + visibility: 'private', + notability: 'high', + validFrom: '2026-01-15', + source: 'OH transcript', + claimMetric: 'mrr', + claimValue: 50000, + claimUnit: 'USD', + claimPeriod: 'monthly', + }); + expect(rowNum).toBe(2); + // Adding a typed row widens the table. + expect(newBody).toContain('claim_metric'); + const parsed = parseFactsFence(newBody); + expect(parsed.warnings).toEqual([]); + expect(parsed.facts.length).toBe(2); + expect(parsed.facts[1].claimMetric).toBe('mrr'); + expect(parsed.facts[1].claimValue).toBe(50000); + }); +}); + +describe('v0.35.4 — metric normalization (D-ENG-4)', () => { + test('known seed-map labels normalize to canonical snake_case', () => { + expect(normalizeMetricLabel('MRR')).toBe('mrr'); + expect(normalizeMetricLabel('Monthly Recurring Revenue')).toBe('mrr'); + expect(normalizeMetricLabel('ARR')).toBe('arr'); + expect(normalizeMetricLabel('annual recurring revenue')).toBe('arr'); + expect(normalizeMetricLabel('Team Size')).toBe('team_size'); + expect(normalizeMetricLabel('Burn Rate')).toBe('burn_rate'); + expect(normalizeMetricLabel('Churn')).toBe('churn_rate'); + }); + + test('unknown labels lowercase + spaces → underscores; non-alphanumeric stripped', () => { + expect(normalizeMetricLabel('Net Promoter Score')).toBe('net_promoter_score'); + expect(normalizeMetricLabel(' CAC ')).toBe('cac'); + expect(normalizeMetricLabel('Time-to-Hire')).toBe('timetohire'); + }); + + test('empty / null / undefined → undefined (the "no metric set" signal)', () => { + expect(normalizeMetricLabel(undefined)).toBeUndefined(); + expect(normalizeMetricLabel(null)).toBeUndefined(); + expect(normalizeMetricLabel('')).toBeUndefined(); + expect(normalizeMetricLabel(' ')).toBeUndefined(); + }); + + test('METRIC_NORMALIZATION_MAP covers the 15-metric seed list named in the plan', () => { + // Pin the seed map so the docs in CLAUDE.md / CHANGELOG match. + const required = [ + 'mrr', 'arr', 'runway', 'headcount', 'team_size', + 'cac', 'ltv', 'gross_margin', 'burn_rate', 'cash', + 'users', 'mau', 'dau', 'churn_rate', 'revenue', + ]; + const canonicalValues = new Set(METRIC_NORMALIZATION_MAP.values()); + for (const r of required) { + expect(canonicalValues.has(r)).toBe(true); + } + }); +}); + +describe('v0.35.4 (D-ENG-1) — extractFactsFromFenceText valid_from precedence', () => { + const fixedToday = new Date('2026-05-17T00:00:00.000Z'); + + test('Path 1: explicit validFrom in fence row wins', () => { + const facts: ParsedFact[] = [{ + rowNum: 1, + claim: 'fence-dated', + kind: 'fact', + confidence: 1.0, + visibility: 'private', + notability: 'medium', + validFrom: '2026-01-15', + active: true, + }]; + const out = extractFactsFromFenceText(facts, 'people/alice-example', 'default', { + nowOverride: fixedToday, + pageEffectiveDate: new Date('2026-04-28'), + }); + expect(out[0].valid_from?.toISOString().slice(0, 10)).toBe('2026-01-15'); + }); + + test('Path 2: missing fence validFrom + pageEffectiveDate set → uses page date', () => { + const facts: ParsedFact[] = [{ + rowNum: 1, + claim: 'no fence date', + kind: 'fact', + confidence: 1.0, + visibility: 'private', + notability: 'medium', + active: true, + }]; + const out = extractFactsFromFenceText(facts, 'people/alice-example', 'default', { + nowOverride: fixedToday, + pageEffectiveDate: new Date('2026-04-28'), + }); + expect(out[0].valid_from?.toISOString().slice(0, 10)).toBe('2026-04-28'); + }); + + test('Path 3: missing fence validFrom AND undefined pageEffectiveDate → undefined (engine defaults to now)', () => { + const facts: ParsedFact[] = [{ + rowNum: 1, + claim: 'no dates at all', + kind: 'fact', + confidence: 1.0, + visibility: 'private', + notability: 'medium', + active: true, + }]; + const out = extractFactsFromFenceText(facts, 'people/alice-example', 'default', { + nowOverride: fixedToday, + pageEffectiveDate: null, + }); + // valid_from is left undefined; the engine layer applies now() at insert. + expect(out[0].valid_from).toBeUndefined(); + }); + + test('typed-claim fields thread through with metric normalization applied', () => { + const facts: ParsedFact[] = [{ + rowNum: 1, + claim: 'MRR $50K', + kind: 'fact', + confidence: 1.0, + visibility: 'private', + notability: 'high', + active: true, + claimMetric: 'Monthly Recurring Revenue', // unnormalized + claimValue: 50000, + claimUnit: 'USD', + claimPeriod: 'monthly', + }]; + const out = extractFactsFromFenceText(facts, 'companies/acme-example', 'default'); + expect(out[0].claim_metric).toBe('mrr'); // normalized + expect(out[0].claim_value).toBe(50000); + expect(out[0].claim_unit).toBe('USD'); + expect(out[0].claim_period).toBe('monthly'); + }); + + test('rows with no typed-claim fields land with null claim_* columns (backward compat)', () => { + const facts: ParsedFact[] = [{ + rowNum: 1, + claim: 'bare claim', + kind: 'fact', + confidence: 1.0, + visibility: 'private', + notability: 'low', + active: true, + }]; + const out = extractFactsFromFenceText(facts, 'people/bob-example', 'default'); + expect(out[0].claim_metric).toBeNull(); + expect(out[0].claim_value).toBeNull(); + expect(out[0].claim_unit).toBeNull(); + expect(out[0].claim_period).toBeNull(); + }); +}); diff --git a/test/founder-scorecard.test.ts b/test/founder-scorecard.test.ts new file mode 100644 index 000000000..f8f73d342 --- /dev/null +++ b/test/founder-scorecard.test.ts @@ -0,0 +1,242 @@ +/** + * v0.35.4 — `gbrain founder scorecard` CLI (T7) tests. + * + * Pins: + * - Pure compute fn: each of the four rollup fields produces correct math. + * - JSON envelope has schema_version: 1 + every required field (R5). + * - G2: empty entity (no facts, no takes) returns a valid empty rollup + * with no NaN / nulls in numeric slots. + * - Red flags fire for regressions + high drift + missed predictions. + */ + +import { describe, test, expect } from 'bun:test'; +import { computeFounderScorecard } from '../src/commands/founder-scorecard.ts'; +import type { TrajectoryPoint, Take } from '../src/core/engine.ts'; + +function pt(args: { + id: number; + metric: string; + value: number; + date: string; + emb?: Float32Array | null; +}): TrajectoryPoint { + return { + fact_id: args.id, + valid_from: new Date(args.date), + metric: args.metric, + value: args.value, + unit: 'USD', + period: 'monthly', + text: `${args.metric} = ${args.value}`, + source_session: null, + source_markdown_slug: null, + embedding: args.emb ?? null, + }; +} + +function take(args: { + id: number; + claim: string; + resolved_outcome: boolean | null; +}): Take { + return { + id: args.id, + page_id: 1, + page_slug: 'companies/acme-example', + row_num: args.id, + claim: args.claim, + kind: 'fact', + holder: 'self', + weight: 0.9, + since_date: '2026-01-15', + until_date: null, + source: 'test', + active: true, + resolved_at: args.resolved_outcome === null ? null : '2026-06-01', + resolved_outcome: args.resolved_outcome, + resolved_value: null, + resolved_unit: null, + resolved_source: null, + resolved_outcome_label: null, + resolved_by: null, + superseded_by: null, + embedded_at: null, + created_at: '2026-01-15', + updated_at: '2026-01-15', + } as unknown as Take; +} + +describe('computeFounderScorecard — JSON envelope (R5)', () => { + test('empty inputs → valid empty rollup, schema_version: 1, no NaN (G2)', () => { + const sc = computeFounderScorecard({ + entitySlug: 'companies/empty', + windowSince: '2025-05-17', + windowUntil: '2026-05-17', + points: [], + takes: [], + }); + expect(sc.schema_version).toBe(1); + expect(sc.entity_slug).toBe('companies/empty'); + expect(sc.window.since).toBe('2025-05-17'); + expect(sc.window.until).toBe('2026-05-17'); + expect(sc.claim_accuracy.predicted).toBe(0); + expect(sc.claim_accuracy.accurate).toBe(0); + expect(sc.claim_accuracy.pct).toBeNull(); + expect(sc.consistency.score).toBeNull(); + expect(sc.consistency.metric_changes).toBe(0); + expect(sc.consistency.typed_facts).toBe(0); + expect(sc.growth_trajectory).toEqual([]); + expect(sc.red_flags).toEqual([]); + // No NaN slipped into numeric slots. + expect(Number.isNaN(sc.claim_accuracy.predicted)).toBe(false); + expect(Number.isNaN(sc.consistency.metric_changes)).toBe(false); + }); +}); + +describe('computeFounderScorecard — claim_accuracy', () => { + test('3 takes, 1 accurate, 1 missed, 1 unresolved → 1/2 = 50% over RESOLVED only', () => { + const sc = computeFounderScorecard({ + entitySlug: 'companies/acme-example', + windowSince: null, windowUntil: null, + points: [], + takes: [ + take({ id: 1, claim: 'will hit $1M ARR', resolved_outcome: true }), + take({ id: 2, claim: 'will close X', resolved_outcome: false }), + take({ id: 3, claim: 'might do Y', resolved_outcome: null }), + ], + }); + expect(sc.claim_accuracy.predicted).toBe(2); + expect(sc.claim_accuracy.accurate).toBe(1); + expect(sc.claim_accuracy.pct).toBeCloseTo(0.5, 3); + }); +}); + +describe('computeFounderScorecard — consistency + growth_trajectory', () => { + test('3-point stable trajectory → 0 changes, score 1.0, growth direction matches latest delta', () => { + const sc = computeFounderScorecard({ + entitySlug: 'companies/stable', + windowSince: null, windowUntil: null, + points: [ + pt({ id: 1, metric: 'mrr', value: 100, date: '2026-01-15' }), + pt({ id: 2, metric: 'mrr', value: 101, date: '2026-04-12' }), + pt({ id: 3, metric: 'mrr', value: 102, date: '2026-07-08' }), + ], + takes: [], + }); + // 1% deltas are below the 5% change threshold. + expect(sc.consistency.metric_changes).toBe(0); + expect(sc.consistency.typed_facts).toBe(3); + expect(sc.consistency.score).toBeCloseTo(1.0, 3); + expect(sc.growth_trajectory.length).toBe(1); + expect(sc.growth_trajectory[0].metric).toBe('mrr'); + // 101 → 102 = 0.99% delta < 1% threshold → 'flat'. + expect(sc.growth_trajectory[0].direction).toBe('flat'); + }); + + test('trajectory with one big drop → 1 change, score 0.667, direction down', () => { + const sc = computeFounderScorecard({ + entitySlug: 'companies/declining', + windowSince: null, windowUntil: null, + points: [ + pt({ id: 1, metric: 'mrr', value: 200000, date: '2026-04-12' }), + pt({ id: 2, metric: 'mrr', value: 150000, date: '2026-07-08' }), + pt({ id: 3, metric: 'mrr', value: 150500, date: '2026-09-01' }), + ], + takes: [], + }); + expect(sc.consistency.metric_changes).toBe(1); + expect(sc.consistency.typed_facts).toBe(3); + expect(sc.consistency.score).toBeCloseTo(1 - 1 / 3, 3); + expect(sc.growth_trajectory[0].direction).toBe('flat'); // last delta is tiny + }); + + test('multiple metrics: each gets its own growth entry, alphabetically ordered', () => { + const sc = computeFounderScorecard({ + entitySlug: 'companies/multi', + windowSince: null, windowUntil: null, + points: [ + pt({ id: 1, metric: 'mrr', value: 100, date: '2026-01-15' }), + pt({ id: 2, metric: 'arr', value: 1200, date: '2026-01-15' }), + pt({ id: 3, metric: 'mrr', value: 130, date: '2026-04-12' }), + pt({ id: 4, metric: 'arr', value: 1500, date: '2026-04-12' }), + ], + takes: [], + }); + expect(sc.growth_trajectory.map(g => g.metric)).toEqual(['arr', 'mrr']); + expect(sc.growth_trajectory[0].direction).toBe('up'); // arr +25% + expect(sc.growth_trajectory[1].direction).toBe('up'); // mrr +30% + }); +}); + +describe('computeFounderScorecard — red_flags', () => { + test('regression fires a red flag', () => { + const sc = computeFounderScorecard({ + entitySlug: 'companies/regression', + windowSince: null, windowUntil: null, + points: [ + pt({ id: 1, metric: 'mrr', value: 200000, date: '2026-04-12' }), + pt({ id: 2, metric: 'mrr', value: 150000, date: '2026-07-08' }), + ], + takes: [], + }); + const reg = sc.red_flags.find(f => f.kind === 'regression'); + expect(reg).toBeDefined(); + expect(reg!.metric).toBe('mrr'); + expect(reg!.text).toContain('25.0%'); + }); + + test('missed predictions surface as red flags', () => { + const sc = computeFounderScorecard({ + entitySlug: 'companies/missed', + windowSince: null, windowUntil: null, + points: [], + takes: [ + take({ id: 1, claim: 'predicted X by June, did not hit it', resolved_outcome: false }), + ], + }); + const missed = sc.red_flags.find(f => f.kind === 'missed_prediction'); + expect(missed).toBeDefined(); + expect(missed!.text).toContain('did not hit it'); + }); + + test('high drift score (>=0.5) fires a narrative_drift flag', () => { + function v(i: number): Float32Array { + const a = new Float32Array(8); + a[i % 8] = 1.0; + return a; + } + const sc = computeFounderScorecard({ + entitySlug: 'companies/drift', + windowSince: null, windowUntil: null, + points: [ + pt({ id: 1, metric: 'mrr', value: 100, date: '2026-01-15', emb: v(0) }), + pt({ id: 2, metric: 'mrr', value: 101, date: '2026-04-12', emb: v(3) }), + pt({ id: 3, metric: 'mrr', value: 102, date: '2026-07-08', emb: v(6) }), + ], + takes: [], + }); + const drift = sc.red_flags.find(f => f.kind === 'narrative_drift'); + expect(drift).toBeDefined(); + }); + + test('clean trajectory + accurate takes + low drift = zero red flags', () => { + function v(): Float32Array { + const a = new Float32Array(8); + a[0] = 1.0; + return a; + } + const sc = computeFounderScorecard({ + entitySlug: 'companies/clean', + windowSince: null, windowUntil: null, + points: [ + pt({ id: 1, metric: 'mrr', value: 100, date: '2026-01-15', emb: v() }), + pt({ id: 2, metric: 'mrr', value: 110, date: '2026-04-12', emb: v() }), + pt({ id: 3, metric: 'mrr', value: 120, date: '2026-07-08', emb: v() }), + ], + takes: [ + take({ id: 1, claim: 'accurate prediction', resolved_outcome: true }), + ], + }); + expect(sc.red_flags).toEqual([]); + }); +}); diff --git a/test/migrate.test.ts b/test/migrate.test.ts index 38a7bf44f..89b6be99e 100644 --- a/test/migrate.test.ts +++ b/test/migrate.test.ts @@ -784,6 +784,108 @@ describe('migrate runner v66 — partial index materialized on PGLite', () => { }); }); +// ──────────────────────────────────────────────────────────────────────── +// v0.35.4 — migration v67 (facts_typed_claim_columns) +// Adds four optional typed-claim columns to `facts` + a partial index +// keyed on (entity_slug, claim_metric, valid_from) WHERE claim_metric IS NOT NULL. +// All fields nullable; the migration is metadata-only on both engines +// because no DEFAULT is set and the partial index covers zero rows until +// extraction emits typed fields. +// ──────────────────────────────────────────────────────────────────────── + +describe('migrate v67 — facts_typed_claim_columns', () => { + const v67 = MIGRATIONS.find(m => m.version === 67); + + test('v67 exists and uses an inline sql field (no handler needed)', () => { + expect(v67).toBeDefined(); + expect(v67!.name).toBe('facts_typed_claim_columns'); + expect(v67!.idempotent).toBe(true); + expect(typeof v67!.sql).toBe('string'); + expect((v67!.sql as string).length).toBeGreaterThan(0); + }); + + test('v67 sql adds all four typed-claim columns', () => { + const sql = v67!.sql as string; + expect(sql).toContain('ADD COLUMN IF NOT EXISTS claim_metric'); + expect(sql).toContain('ADD COLUMN IF NOT EXISTS claim_value'); + expect(sql).toContain('ADD COLUMN IF NOT EXISTS claim_unit'); + expect(sql).toContain('ADD COLUMN IF NOT EXISTS claim_period'); + // DOUBLE PRECISION is the numeric type for claim_value (per plan D-CDX). + expect(sql).toContain('DOUBLE PRECISION'); + }); + + test('v67 creates partial index on (entity_slug, claim_metric, valid_from)', () => { + const sql = v67!.sql as string; + expect(sql).toContain('CREATE INDEX IF NOT EXISTS facts_typed_claim_idx'); + expect(sql).toContain('ON facts (entity_slug, claim_metric, valid_from)'); + expect(sql).toContain('WHERE claim_metric IS NOT NULL'); + }); +}); + +describe('migrate runner v67 — typed-claim columns materialized on PGLite', () => { + let engine: PGLiteEngine; + + beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({}); + await engine.initSchema(); + }); + + afterAll(async () => { + await engine.disconnect(); + }); + + test('v67 added claim_metric, claim_value, claim_unit, claim_period columns to facts', async () => { + const rows = await (engine as any).db.query( + `SELECT column_name, data_type FROM information_schema.columns + WHERE table_name = 'facts' + AND column_name IN ('claim_metric', 'claim_value', 'claim_unit', 'claim_period') + ORDER BY column_name`, + ); + const names = rows.rows.map((r: any) => r.column_name).sort(); + expect(names).toEqual(['claim_metric', 'claim_period', 'claim_unit', 'claim_value']); + const byName: Record = Object.fromEntries( + rows.rows.map((r: any) => [r.column_name, r.data_type]), + ); + // claim_value is DOUBLE PRECISION; the others are TEXT. + expect(byName['claim_value']).toBe('double precision'); + expect(byName['claim_metric']).toBe('text'); + expect(byName['claim_unit']).toBe('text'); + expect(byName['claim_period']).toBe('text'); + }); + + test('v67 created facts_typed_claim_idx partial index on PGLite', async () => { + const rows = await (engine as any).db.query( + `SELECT indexname, indexdef FROM pg_indexes WHERE indexname = 'facts_typed_claim_idx'`, + ); + expect(rows.rows.length).toBe(1); + // The partial-predicate appears in the materialized index definition. + expect(rows.rows[0].indexdef).toContain('claim_metric'); + }); + + test('v67 columns are nullable — existing facts persist with NULL typed fields (backward compat)', async () => { + // Insert a fact via raw SQL with no typed-claim values; assert the + // four columns remain NULL. The cycle path (extract_facts) hits this + // backward-compat surface every time it processes a fence without + // metric assertions. + const db = (engine as any).db; + await db.exec(`INSERT INTO sources (id, name) VALUES ('v67-test', 'v67-test') ON CONFLICT DO NOTHING`); + await db.exec( + `INSERT INTO facts (source_id, entity_slug, fact, source, valid_from) + VALUES ('v67-test', 'v67/example', 'plain non-typed claim', 'test', now())`, + ); + const row = await db.query( + `SELECT claim_metric, claim_value, claim_unit, claim_period + FROM facts WHERE source_id = 'v67-test' AND entity_slug = 'v67/example'`, + ); + expect(row.rows.length).toBe(1); + expect(row.rows[0].claim_metric).toBeNull(); + expect(row.rows[0].claim_value).toBeNull(); + expect(row.rows[0].claim_unit).toBeNull(); + expect(row.rows[0].claim_period).toBeNull(); + }); +}); + describe('migrate: v8 (links_dedup) regression — must be fast on 1K duplicate rows', () => { let engine: PGLiteEngine; diff --git a/test/operations-find-trajectory.test.ts b/test/operations-find-trajectory.test.ts new file mode 100644 index 000000000..c9585071d --- /dev/null +++ b/test/operations-find-trajectory.test.ts @@ -0,0 +1,185 @@ +/** + * v0.35.4 — find_trajectory MCP op (T5) tests. + * + * Pins: + * - Param validation: entity_slug required, non-empty. + * - Visibility filter on remote=true callers (R6 / D-CDX-1). + * - Source scoping via sourceScopeOpts (federated vs scalar). + * - Stable JSON envelope: points + regressions + drift_score + schema_version=1 (R5). + * - Engine result's raw Float32Array embedding is NOT serialized to wire. + * - Empty-result graceful shape (G1). + * - The op is registered + read-scope + not localOnly. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import { operationsByName } from '../src/core/operations.ts'; +import type { OperationContext } from '../src/core/operations.ts'; + +let engine: PGLiteEngine; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({}); + await engine.initSchema(); +}); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await engine.executeRaw(`DELETE FROM facts WHERE entity_slug LIKE 'optraj-%'`); + await engine.executeRaw(`DELETE FROM sources WHERE id LIKE 'optraj-%'`); +}); + +function unitVec(idx: number): string { + const a = new Float32Array(1536); + a[idx % 1536] = 1.0; + return '[' + Array.from(a).join(',') + ']'; +} + +async function insertTyped(args: { + source_id?: string; + entity_slug: string; + metric: string; + value: number; + valid_from: Date; + visibility?: 'private' | 'world'; + vecIdx?: number; +}): Promise { + const sid = args.source_id ?? 'default'; + await engine.executeRaw( + `INSERT INTO sources (id, name) VALUES ($1, $1) ON CONFLICT DO NOTHING`, + [sid], + ); + await engine.executeRaw( + `INSERT INTO facts (source_id, entity_slug, fact, kind, source, valid_from, + claim_metric, claim_value, claim_unit, claim_period, + visibility, embedding, embedded_at) + VALUES ($1, $2, $3, 'fact', 'test', $4::timestamptz, + $5, $6, 'USD', 'monthly', + $7, $8::vector, $4::timestamptz)`, + [ + sid, args.entity_slug, `${args.metric} = ${args.value}`, + args.valid_from.toISOString(), args.metric, args.value, + args.visibility ?? 'private', unitVec(args.vecIdx ?? 0), + ], + ); +} + +function mkCtx(overrides: Partial = {}): OperationContext { + return { + engine, + config: {} as any, + logger: { info: () => {}, warn: () => {}, error: () => {} } as any, + dryRun: false, + remote: false, + ...overrides, + } as OperationContext; +} + +describe('find_trajectory MCP op — registration + shape', () => { + test('registered with read scope, NOT localOnly', () => { + const op = operationsByName['find_trajectory']; + expect(op).toBeDefined(); + expect(op.scope).toBe('read'); + expect(op.localOnly).toBeUndefined(); + // Description references the v0.35.4 contract. + expect(op.description).toContain('schema_version'); + }); + + test('throws on missing entity_slug', async () => { + const op = operationsByName['find_trajectory']; + await expect(op.handler(mkCtx(), {})).rejects.toThrow(/entity_slug/); + await expect(op.handler(mkCtx(), { entity_slug: '' })).rejects.toThrow(/entity_slug/); + await expect(op.handler(mkCtx(), { entity_slug: ' ' })).rejects.toThrow(/entity_slug/); + }); + + test('returns stable JSON shape with schema_version: 1', async () => { + await insertTyped({ entity_slug: 'optraj-shape', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') }); + const op = operationsByName['find_trajectory']; + const result = await op.handler(mkCtx(), { entity_slug: 'optraj-shape' }) as any; + expect(result).toHaveProperty('points'); + expect(result).toHaveProperty('regressions'); + expect(result).toHaveProperty('drift_score'); + expect(result.schema_version).toBe(1); + // Embedding NOT serialized to the wire. + expect(result.points[0]).not.toHaveProperty('embedding'); + // valid_from is YYYY-MM-DD string. + expect(result.points[0].valid_from).toMatch(/^\d{4}-\d{2}-\d{2}$/); + }); + + test('unknown entity returns graceful empty shape (G1)', async () => { + const op = operationsByName['find_trajectory']; + const result = await op.handler(mkCtx(), { entity_slug: 'optraj-does-not-exist' }) as any; + expect(result.points).toEqual([]); + expect(result.regressions).toEqual([]); + expect(result.drift_score).toBeNull(); + expect(result.schema_version).toBe(1); + }); +}); + +describe('find_trajectory MCP op — visibility filter (R6 / D-CDX-1)', () => { + test('remote=true sees only world-visibility points', async () => { + await insertTyped({ entity_slug: 'optraj-vis', metric: 'mrr', value: 50000, visibility: 'private', valid_from: new Date('2026-01-15') }); + await insertTyped({ entity_slug: 'optraj-vis', metric: 'mrr', value: 99999, visibility: 'world', valid_from: new Date('2026-04-12') }); + + const op = operationsByName['find_trajectory']; + const local = await op.handler(mkCtx({ remote: false }), { entity_slug: 'optraj-vis' }) as any; + expect(local.points.length).toBe(2); + + const remote = await op.handler(mkCtx({ remote: true }), { entity_slug: 'optraj-vis' }) as any; + expect(remote.points.length).toBe(1); + expect(remote.points[0].value).toBe(99999); + }); +}); + +describe('find_trajectory MCP op — source scoping (D-CDX-6)', () => { + test('federated sourceIds from auth.allowedSources narrows scope', async () => { + await insertTyped({ source_id: 'optraj-A', entity_slug: 'optraj-fed', metric: 'mrr', value: 1, valid_from: new Date('2026-01-15') }); + await insertTyped({ source_id: 'optraj-B', entity_slug: 'optraj-fed', metric: 'mrr', value: 2, valid_from: new Date('2026-04-12') }); + await insertTyped({ source_id: 'optraj-C', entity_slug: 'optraj-fed', metric: 'mrr', value: 3, valid_from: new Date('2026-07-08') }); + + const op = operationsByName['find_trajectory']; + const ctx = mkCtx({ + auth: { allowedSources: ['optraj-A', 'optraj-B'] } as any, + }); + const result = await op.handler(ctx, { entity_slug: 'optraj-fed' }) as any; + expect(result.points.length).toBe(2); + expect(result.points.map((p: any) => p.value)).toEqual([1, 2]); + }); + + test('scalar ctx.sourceId narrows to that single source', async () => { + await insertTyped({ source_id: 'optraj-X', entity_slug: 'optraj-scalar', metric: 'mrr', value: 100, valid_from: new Date('2026-01-15') }); + await insertTyped({ source_id: 'optraj-Y', entity_slug: 'optraj-scalar', metric: 'mrr', value: 200, valid_from: new Date('2026-01-15') }); + + const op = operationsByName['find_trajectory']; + const ctx = mkCtx({ sourceId: 'optraj-X' }); + const result = await op.handler(ctx, { entity_slug: 'optraj-scalar' }) as any; + expect(result.points.length).toBe(1); + expect(result.points[0].value).toBe(100); + }); +}); + +describe('find_trajectory MCP op — regression + drift surface', () => { + test('regressions populate when newer value drops >= 10% (D-ENG-2 default)', async () => { + await insertTyped({ entity_slug: 'optraj-reg', metric: 'mrr', value: 200000, valid_from: new Date('2026-04-12'), vecIdx: 0 }); + await insertTyped({ entity_slug: 'optraj-reg', metric: 'mrr', value: 150000, valid_from: new Date('2026-07-08'), vecIdx: 0 }); + + const op = operationsByName['find_trajectory']; + const result = await op.handler(mkCtx(), { entity_slug: 'optraj-reg' }) as any; + expect(result.regressions.length).toBe(1); + expect(result.regressions[0].metric).toBe('mrr'); + expect(result.regressions[0].delta_pct).toBeCloseTo(-0.25, 3); + }); + + test('drift_score returns null with <3 embedded points (G3)', async () => { + await insertTyped({ entity_slug: 'optraj-drift', metric: 'mrr', value: 1, valid_from: new Date('2026-01-15') }); + await insertTyped({ entity_slug: 'optraj-drift', metric: 'mrr', value: 2, valid_from: new Date('2026-04-12') }); + + const op = operationsByName['find_trajectory']; + const result = await op.handler(mkCtx(), { entity_slug: 'optraj-drift' }) as any; + expect(result.drift_score).toBeNull(); + }); +}); diff --git a/test/schema-bootstrap-coverage.test.ts b/test/schema-bootstrap-coverage.test.ts index 5e9bb0a79..c68ecf085 100644 --- a/test/schema-bootstrap-coverage.test.ts +++ b/test/schema-bootstrap-coverage.test.ts @@ -640,6 +640,14 @@ const COLUMN_EXEMPTIONS = new Set([ 'pages.source_path', 'content_chunks.edges_backfilled_at', 'query_cache.knobs_hash', + // v0.35.6 (migration v67) — typed-claim columns + facts_typed_claim_idx + // partial index are co-defined in the same migration, so the schema-blob + // forward-reference path isn't tripped. Bootstrap is only required when an + // index in PGLITE_SCHEMA_SQL references a column added by a later migration. + 'facts.claim_metric', + 'facts.claim_value', + 'facts.claim_unit', + 'facts.claim_period', ]); test('every ALTER TABLE ADD COLUMN in MIGRATIONS is covered by applyForwardReferenceBootstrap (column-only class)', async () => {