mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 21:19:18 +00:00
v0.35.7.0 feat: temporal trajectory + founder scorecard (Phases 2-4) (#1131)
* feat(facts): typed-claim substrate + cycle correctness fixes (v0.35.6 wave 1/3) Schema (migration v67): - Add four optional typed-claim columns to facts: claim_metric TEXT, claim_value DOUBLE PRECISION, claim_unit TEXT, claim_period TEXT - Partial index facts_typed_claim_idx ON (entity_slug, claim_metric, valid_from) WHERE claim_metric IS NOT NULL - All nullable, metadata-only on both engines Fence layer: - ParsedFact (facts-fence.ts) gains optional claimMetric/Value/Unit/Period - Parser tolerates both 10-cell (legacy) and 14-cell (widened) rows - Renderer emits 14 cells iff any row has typed data; otherwise stays 10-cell so existing fences don't widen on unrelated edits - Numeric value cell tolerates comma thousand separators (50,000 -> 50000) Extract pipeline (D-CDX-2, D-ENG-1): - src/core/facts/extract.ts (the actual Haiku call site, NOT extract-facts.ts cycle phase) extends its system prompt to emit typed fields for metric-shaped claims - extractFactsFromFenceText gains optional pageEffectiveDate. Precedence: fence-row validFrom > pageEffectiveDate > undefined (engine defaults to now) - normalizeMetricLabel: 15-entry seed map for common founder metrics (mrr, arr, runway, headcount, team_size, cac, ltv, gross_margin, burn_rate, cash, users, mau, dau, churn_rate, revenue); unknown labels lowercase + space->_ Engine extensions: - NewFact + insertFact + insertFacts in both engines accept the four typed columns (all nullable) - Cycle phase extract-facts.ts threads page.effective_date through AND batch-embeds via gateway.embed() before insertFacts (D-CDX-3 fix for cycle-inserted facts arriving with embedding=NULL) Consolidate fix (D-CDX-4 — Codex F4): - Replace MAX(row_num)+1 INSERT with semantic upsert on (page_id, claim, since_date). Re-running the full cycle on stable input produces zero new takes — fixes the pre-existing duplicate-takes bug after extract_facts wipes consolidated_at - Chronological valid_until writeback per cluster: sort by (valid_from ASC, id ASC), walk pairs, set older.valid_until = newer.valid_from Tests: - test/migrate.test.ts +6 cases for v67 shape + materialization + nullable backward compat - test/facts-fence-typed.test.ts (new, 17 cases): parser+renderer round-trip, normalization seed map coverage, valid_from precedence three-branch - test/consolidate-valid-until.test.ts (new, 4 cases): chronological writeback (R4a), same-day id tiebreaker, cycle re-run zero duplicates (R4b/R7), valid_until idempotency - test/schema-bootstrap-coverage.test.ts: add four typed-claim columns to COLUMN_EXEMPTIONS (migration co-defines the partial index, no forward reference to bootstrap) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(trajectory): find_trajectory MCP op + eval/founder CLIs (v0.35.6 wave 2/3) Engine method (D-CDX-1, D-CDX-6): - BrainEngine.findTrajectory(opts) on both Postgres and PGLite - TrajectoryOpts: scalar sourceId fast path + sourceIds federated array (mirrors v0.34.1.0 search* dual pattern) - opts.remote: when true, SQL adds AND visibility='world' so OAuth read clients see only world-visibility facts (mirrors recall's posture — closes the F7 privacy regression Codex caught in plan review) - Single SQL query, ORDER BY valid_from ASC, id ASC for deterministic output (R3 pin). Returns TrajectoryPoint[] including raw embedding so the caller can compute drift without a second round-trip Pure function library (src/core/trajectory.ts, new): - detectRegressions(points, threshold): walks consecutive (metric, value) pairs per metric; emits when newer drops >= threshold below older. 10% default, override via GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD - computeDriftScore(points): 1 - mean(cosine(emb[i], emb[i-1])) over embedded points; clamped [0,1]; null when <3 embedded points (D-ENG-3 graceful degradation) - computeTrajectoryStats(points): composed shape returning both - TRAJECTORY_SCHEMA_VERSION = 1 — additive-only across releases (R5) MCP op (src/core/operations.ts): - find_trajectory: scope read, NOT localOnly. Routes through sourceScopeOpts(ctx) for federated isolation AND threads ctx.remote for visibility filtering. Strips raw Float32Array embeddings from the wire shape; converts valid_from to YYYY-MM-DD string - Registered in operations array after find_experts - FIND_TRAJECTORY_DESCRIPTION in operations-descriptions.ts CLIs: - gbrain eval trajectory <entity> [--metric M] [--since D] [--until D] [--limit N] [--json] — chronological human view with [REGRESSION] inline annotation; thin-client routing via callRemoteTool(find_trajectory). Dispatched in src/commands/eval.ts sub-subcommand block - gbrain founder scorecard <entity> [--since D] [--until D] [--json] — pure aggregation over Phase 2's substrate. Four signals: claim_accuracy (over resolved takes), consistency, growth_trajectory, red_flags. computeFounderScorecard exported for tests. Registered as top-level command in cli.ts; added to CLI_ONLY set Tests (45 cases across 5 files): - test/engine-find-trajectory.test.ts: 18 cases — chronological order, source scoping (scalar + federated), visibility filter on remote=true, metric + since/until filters, regression detection at threshold boundaries, drift score with various embedding states - test/operations-find-trajectory.test.ts: 9 cases — op registration, param validation, JSON envelope shape, R5 schema_version: 1, embedding stripped from wire, R6 visibility filter, source scoping - test/eval-trajectory.test.ts: 7 cases — arg parsing, --help, --json envelope, regression annotation, --metric filter, empty entity - test/founder-scorecard.test.ts: 9 cases — empty inputs no-NaN (G2), claim_accuracy math, consistency math, growth_trajectory math, red_flags fire for regression / narrative_drift / missed_prediction - test/eval-contradictions/no-valid-until-write.test.ts: 4 cases — R1 (probe never writes valid_until under eval-contradictions/) + R8 (only allow-listed files write valid_until anywhere in src/) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: v0.35.6.0 — CHANGELOG + VERSION + docs + migration note Bumps to v0.35.6.0 (next-minor after master's v0.35.5.1 — typed-claim substrate + trajectory + founder scorecard is a new user-facing feature surface, not a fix). - VERSION + package.json synced - CHANGELOG.md release-summary block in the wave-style voice, lead with what the user can now DO. Sections: typed metric claims in the fence, chronological metric trajectories, founder scorecard, MCP find_trajectory op, cycle re-run idempotency fix, embedding-on-insert fix, valid_from precedence fix. To-take-advantage-of block with verification + opt-in fence syntax example - CLAUDE.md Key Files entry consolidating the wave across eval-trajectory.ts + founder-scorecard.ts + trajectory.ts. Names every D-ENG / D-CDX decision and the Codex outside-voice F-numbers - skills/migrations/v0.35.6.md agent-readable migration note. Includes fence-syntax example for typed-claim rows so downstream agents start emitting them. Iron-rule contracts called out (R1 + R8 + R7 + visibility) - llms-full.txt regenerated to reflect the new CLAUDE.md entry Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: post-ship sync for v0.35.7.0 — trajectory + founder scorecard - README.md: add `gbrain eval trajectory` to EVAL section, add new TEMPORAL block covering `gbrain founder scorecard` + the GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD env override; add v0.35.7 "What's new" paragraph below the v0.28.8 LongMemEval blurb - AGENTS.md: new bullet under Common tasks teaching agents to reach for `gbrain eval trajectory` / `gbrain founder scorecard` / the `find_trajectory` MCP op when asked to evaluate a founder/company over time - docs/contradictions.md: append "Temporal axis follow-on (v0.35.3.1 + v0.35.7)" subsection under See also, cross-linking the trajectory substrate and naming the auto-supersession.ts:4 invariant preserved by both the verdict enum (probe side) and consolidate's valid_until writeback (cycle side) - CLAUDE.md: fix stale (v0.35.4) tag on the trajectory entry to (v0.35.7) — version got rebumped twice during the merge wave - skills/migrations/v0.35.7.md renamed to v0.35.7.0.md for consistency with the v0.35.0.0.md / v0.14.0.md / etc naming convention - llms-full.txt regenerated to reflect the CLAUDE.md edit Coverage map (Diataxis): /eval trajectory CLI ✅ ref (README, AGENTS) ✅ how-to (CHANGELOG) ❌ tutorial /founder scorecard CLI ✅ ref (README, AGENTS) ✅ how-to (CHANGELOG) ❌ tutorial find_trajectory MCP op ✅ ref (CLAUDE.md, AGENTS, contradictions.md) typed-claim fence cols ✅ ref (skills/migrations/v0.35.7.0.md, CHANGELOG) Migration v67 ✅ ref (CLAUDE.md, CHANGELOG) No tutorial / explanation gaps worth filling in this PR — the migration note's fence-syntax example already covers the "first typed claim" walkthrough. ARCHITECTURE diagrams not drifted (the trajectory work extends existing facts/takes infrastructure; no new component boxes). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
af7e5379c2
commit
1dadd9ed71
@@ -59,6 +59,14 @@ writing or reviewing an operation, consult `src/core/operations.ts` for the cont
|
||||
<dataset.jsonl>` (v0.28.8) runs against an isolated in-memory PGLite
|
||||
per question — your `~/.gbrain` is never opened. Full guide:
|
||||
[`docs/eval-bench.md`](./docs/eval-bench.md).
|
||||
- **Track a founder/company over time (v0.35.7):** when an entity has
|
||||
typed metric claims in its `## Facts` fence (`metric: mrr`, `value: 50000`,
|
||||
`unit: USD`, `period: monthly` columns), run
|
||||
`gbrain eval trajectory <entity-slug>` for the chronological history
|
||||
with regressions auto-flagged, or `gbrain founder scorecard <entity-slug>`
|
||||
for a four-signal JSON rollup (claim_accuracy / consistency /
|
||||
growth_trajectory / red_flags). MCP op `find_trajectory` exposes the
|
||||
same data — read scope, visibility-filtered for remote callers.
|
||||
- **Everything else:** [`./llms.txt`](./llms.txt) is the full documentation map.
|
||||
[`./llms-full.txt`](./llms-full.txt) is the same map with core docs inlined for
|
||||
single-fetch ingestion.
|
||||
|
||||
@@ -2,6 +2,80 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.35.7.0] - 2026-05-17
|
||||
|
||||
**The contradiction probe grew up. Typed claims over time, regressions detected automatically, founder scorecards as a one-liner.**
|
||||
|
||||
v0.35.3.1 taught the probe to see dates. v0.35.7 turns dates into a proper time-series substrate. Your `## Facts` fences can now carry typed metric assertions (mrr=50000, arr=2000000, team_size=12), the consolidate cycle phase writes `valid_until` on chronologically-superseded facts, and two new commands turn that data into operator signal: `gbrain eval trajectory <entity>` shows the sorted claim history with regressions flagged inline, and `gbrain founder scorecard <entity>` rolls up claim accuracy, consistency, growth direction, and red flags into one JSON payload.
|
||||
|
||||
This is the wave the original temporal-contradiction RFC deferred to "Phases 2-4." Three rounds of review (CEO, eng, codex outside-voice) caught a security regression in the planned API, a pre-existing cycle-idempotency bug that would have poisoned trajectory data on every dream cycle re-run, and a misidentified file path for the LLM extraction prompt — all fixed before any code landed.
|
||||
|
||||
### What you can now do
|
||||
|
||||
**Author typed metric claims in the `## Facts` fence.** The fence widens from 10 to 14 columns when a row carries `claim_metric`, `claim_value`, `claim_unit`, `claim_period`. Mixed fences (some typed rows, some not) work fine — the renderer stays at 10 cells when every row's typed fields are empty, so existing brains don't see diff churn. Metric labels normalize to lowercase snake_case (`MRR` → `mrr`, `Monthly Recurring Revenue` → `mrr`) so trajectory queries don't fragment across capitalization variants. Fifteen common founder metrics have canonical names in the seed map; unknown labels lowercase + underscore-collapse and pass through.
|
||||
|
||||
**Get chronological metric trajectories on any entity.** `gbrain eval trajectory companies/acme-example` prints a sorted history with auto-detected regressions flagged inline. `--metric mrr` filters to a single metric. `--since 2026-01-01 --until 2026-07-31` narrows the window. `--json` returns the stable `schema_version: 1` envelope `{points, regressions, drift_score, schema_version}`. Regression threshold is 10% by default, override via `GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD`. The drift_score is `1 - mean(cosine(emb[i], emb[i-1]))` over the trajectory's existing embeddings: 0 means narrative stable, 1 means every claim is unrelated; null when fewer than 3 typed points have embeddings.
|
||||
|
||||
**Roll up an entity into a founder scorecard.** `gbrain founder scorecard companies/acme-example` produces four signals: claim_accuracy (over resolved takes), consistency (1 minus the fraction of metric value-changes), growth_trajectory (direction + delta per metric), and red_flags (regressions + high drift + missed predictions). `--json` for a stable contract that any downstream system can consume. Default window is the last year; `--since` / `--until` to narrow.
|
||||
|
||||
**Reach trajectories via the MCP `find_trajectory` op** — read scope, not localOnly. OAuth clients query other agents' brains directly. Visibility filtering matches `recall`'s posture: remote callers see only `visibility='world'` facts; local CLI sees everything. Source-scoped via the existing `sourceScopeOpts` v0.34.1.0 pattern — federated `allowedSources` clients get the union; scalar `sourceId` clients see their one source.
|
||||
|
||||
**Run the dream cycle without poisoning trajectory data.** v0.35.7 fixes a pre-existing cycle idempotency bug: before this wave, running the full cycle twice in a row would append duplicate takes via `MAX(row_num)+1` after `extract_facts` wiped `consolidated_at` on every fact. The fix is a semantic upsert keyed on `(page_id, claim, since_date)` so re-promotion of a stable cluster reuses the existing take. The autopilot can now run on a cron without silently doubling your facts table's promotion count.
|
||||
|
||||
**Cycle-inserted facts now arrive with embeddings.** Before this wave, the `extract_facts` cycle phase inserted fence-derived facts with `embedding = NULL`, which broke consolidate's cosine clustering and any downstream embedding-dependent feature (drift score, semantic similarity in recall). v0.35.7 batches `gateway.embed()` after fence parse and threads embeddings into `insertFacts`. ~$0.02 per 1K facts at OpenAI 3-large pricing; falls open when the embedding gateway is unavailable.
|
||||
|
||||
**Fact valid_from now reflects claim dates, not import timestamps.** Eng review caught that `extractFactsFromFenceText` returned `valid_from: undefined` when a fence row lacked an explicit `validFrom:`, which fell through to `insertFacts`'s `now()` default. So a meeting page dated 2026-04-28 used to land its facts as claimed-on-today instead of claimed-on-the-meeting-date — and every trajectory query against existing fences showed import dates instead of claim dates. The fix threads `pages.effective_date` (v0.29.1+) as the fallback. Precedence chain: explicit fence `validFrom:` > `pages.effective_date` > `now()`.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
- **Schema migration v67** (`facts_typed_claim_columns`) adds four optional columns to `facts`: `claim_metric TEXT`, `claim_value DOUBLE PRECISION`, `claim_unit TEXT`, `claim_period TEXT`. Plus partial index `facts_typed_claim_idx ON facts (entity_slug, claim_metric, valid_from) WHERE claim_metric IS NOT NULL`. All nullable; metadata-only on both engines.
|
||||
- `ParsedFact` (`src/core/facts-fence.ts`) gains optional `claimMetric`, `claimValue`, `claimUnit`, `claimPeriod`. Parser tolerates both 10-cell (legacy) and 14-cell (widened) row shapes. Renderer emits 14 cells iff any row has a non-undefined typed field; otherwise stays at 10-cell for backward compat. Numeric `claimValue` cell tolerates comma thousand separators (`50,000` → `50000`).
|
||||
- `extractFactsFromFenceText` (`src/core/facts/extract-from-fence.ts`) gains optional `pageEffectiveDate` in opts. Three-branch precedence: fence row > pageEffectiveDate > undefined (engine defaults to `now()`). Metric normalization via `normalizeMetricLabel` with a 15-entry seed map for common founder metrics.
|
||||
- `src/core/facts/extract.ts` (the MCP put_page / backstop Haiku call site, NOT the cycle phase — Codex F2 fix) extends its system prompt to optionally emit `metric/value/unit/period` for metric-shaped claims. Backward compatible — non-metric claims emit nulls.
|
||||
- `NewFact` interface (`src/core/engine.ts`) gains `claim_metric`, `claim_value`, `claim_unit`, `claim_period` (all nullable). Postgres + PGLite `insertFact` / `insertFacts` SQL extended.
|
||||
- `BrainEngine.findTrajectory` method added to both engines. Source-scoped (`sourceId` scalar fast path + `sourceIds` federated array), visibility-filtered when `remote=true`. Single SQL query, `ORDER BY valid_from ASC, id ASC` for deterministic ordering (R3 pin).
|
||||
- `src/core/trajectory.ts` (new file) — pure functions `detectRegressions`, `computeDriftScore`, `computeTrajectoryStats`. Exported `TRAJECTORY_SCHEMA_VERSION = 1`. Threshold knob via `GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD` env var.
|
||||
- `find_trajectory` MCP op (`src/core/operations.ts`) — scope: read, not localOnly. Routes through `sourceScopeOpts(ctx)` + threads `ctx.remote` for visibility filtering. Strips raw `Float32Array` embeddings from the wire response.
|
||||
- `gbrain eval trajectory <entity>` CLI (`src/commands/eval-trajectory.ts`). Pure data fn + JSON formatter + human formatter + thin-client routing seam.
|
||||
- `gbrain founder scorecard <entity>` CLI (`src/commands/founder-scorecard.ts`). Pure aggregation function `computeFounderScorecard` exported for tests. Thin-client routing falls back to trajectory-only when remote.
|
||||
- `consolidate` cycle phase (`src/core/cycle/phases/consolidate.ts`) gains semantic upsert keyed on `(page_id, claim, since_date)` + chronological `valid_until` writeback on each cluster. Re-running the full cycle on stable input produces zero new takes (R7 pin) and zero `valid_until` diffs (idempotent).
|
||||
- `extract_facts` cycle phase (`src/core/cycle/extract-facts.ts`) batch-embeds via `gateway.embed()` before `insertFacts`. Threads `page.effective_date` as the `pageEffectiveDate` fallback.
|
||||
- New tests: `test/facts-fence-typed.test.ts` (17 cases — round-trip, normalization, valid_from precedence), `test/consolidate-valid-until.test.ts` (4 cases — R4a chronological writeback, R4b/R7 cycle idempotency), `test/engine-find-trajectory.test.ts` (18 cases — engine + trajectory.ts pure functions), `test/operations-find-trajectory.test.ts` (9 cases — MCP op shape + visibility + source scoping), `test/eval-trajectory.test.ts` (7 cases — CLI), `test/founder-scorecard.test.ts` (9 cases — rollup math), `test/eval-contradictions/no-valid-until-write.test.ts` (4 cases — R1+R8 grep guards).
|
||||
- Migration v67 test added to `test/migrate.test.ts` (6 cases — shape, source-shape, idempotency, runtime materialization, backward compat with nullable columns).
|
||||
|
||||
## To take advantage of v0.35.7.0
|
||||
|
||||
`gbrain upgrade` should do this automatically. The wave ships migration v67 (`facts_typed_claim_columns`); `apply-migrations` runs it transparently.
|
||||
|
||||
1. **Verify the migration applied:**
|
||||
```bash
|
||||
gbrain doctor --json
|
||||
```
|
||||
Look for `schema_version: 67`. If it's still lower, run `gbrain apply-migrations --yes`.
|
||||
2. **Try the new commands on an existing entity:**
|
||||
```bash
|
||||
gbrain eval trajectory <some-entity-slug>
|
||||
gbrain founder scorecard <some-entity-slug>
|
||||
```
|
||||
Without any typed-claim data yet, `eval trajectory` will say "(no typed claims for this entity in the window)". That's normal — the substrate is now ready; the data lands as you author typed fence rows or as the next dream cycle's Haiku extraction populates them on conversation-derived facts.
|
||||
3. **Author a typed-claim fence row (optional):**
|
||||
In any entity's `## Facts` fence, extend a row with typed columns:
|
||||
```markdown
|
||||
| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | claim_metric | claim_value | claim_unit | claim_period |
|
||||
|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|--------------|-------------|------------|--------------|
|
||||
| 1 | MRR hit $50K | fact | 1.0 | private | high | 2026-01-15 | | OH transcript | | mrr | 50000 | USD | monthly |
|
||||
```
|
||||
On next `gbrain sync` + dream cycle, the fact lands with typed fields. `gbrain eval trajectory` picks it up immediately.
|
||||
4. **(Optional) Tighten the regression threshold:**
|
||||
```bash
|
||||
export GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD=0.05 # fire on 5%+ drops instead of 10%
|
||||
```
|
||||
5. **If any step fails or the numbers look wrong,** please file an issue:
|
||||
https://github.com/garrytan/gbrain/issues with:
|
||||
- output of `gbrain doctor`
|
||||
- contents of `~/.gbrain/upgrade-errors.jsonl` if it exists
|
||||
- which step broke
|
||||
|
||||
## [0.35.6.0] - 2026-05-17
|
||||
|
||||
**Your search results stop letting weak pages climb to the top just because they have a lot of links pointing at them.** Off by default; turn it on with one config key.
|
||||
|
||||
@@ -84,6 +84,7 @@ strict behavior when unset.
|
||||
- `src/commands/eval-export.ts` (v0.25.0) — streams `eval_candidates` rows as NDJSON to stdout with `schema_version: 1` prefix on every line. EPIPE-safe, progress heartbeats on stderr, stable id-desc tiebreaker so `--since` windows never dupe/miss rows.
|
||||
- `src/commands/eval-prune.ts` (v0.25.0) — explicit retention cleanup. Requires `--older-than DUR`. `--dry-run` reports would-delete count.
|
||||
- `src/commands/eval-replay.ts` (v0.25.0) — contributor-facing replay tool. Reads NDJSON from `gbrain eval export`, re-runs each captured `query` / `search` op against the current brain, computes set-Jaccard@k between captured + current `retrieved_slugs`, top-1 stability rate, and latency Δ. Stable JSON shape (`schema_version: 1`) for CI gating; human mode prints a regression table. Pure Bun, zero new deps. The dev-loop half of BrainBench-Real that closes the gap between "data captured" and "data used to gate a PR." See `docs/eval-bench.md` for the workflow.
|
||||
- `src/commands/eval-trajectory.ts` + `src/commands/founder-scorecard.ts` + `src/core/trajectory.ts` (v0.35.7) — temporal trajectory + founder scorecard. The wave that turns the v0.35.3.1 date-aware contradiction probe into a useful temporal substrate. `gbrain eval trajectory <entity>` shows the chronological typed-claim history (mrr/arr/team_size/etc) with regressions auto-flagged inline; `gbrain founder scorecard <entity>` rolls up claim_accuracy / consistency / growth_trajectory / red_flags into one JSON. Pure-function math lives in `trajectory.ts`: `detectRegressions(points, threshold)` walks consecutive metric-value pairs per metric (10% drop default, env override `GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD`); `computeDriftScore(points)` returns `1 - mean(cosine(emb[i], emb[i-1]))` over existing embeddings (null when <3 embedded points). Backed by `BrainEngine.findTrajectory(opts)` — both Postgres and PGLite, single SQL query, deterministic `ORDER BY valid_from ASC, id ASC` (R3). Source-scoped via the v0.34.1.0 `sourceId` scalar / `sourceIds` array dual pattern (D-CDX-6); visibility-filtered for remote callers (D-CDX-1) — `recall`-equivalent posture. MCP op `find_trajectory` (read scope, NOT localOnly) registered after `find_experts`. Migration v67 adds four optional typed-claim columns (`claim_metric`, `claim_value`, `claim_unit`, `claim_period`) + a partial index on `(entity_slug, claim_metric, valid_from) WHERE claim_metric IS NOT NULL`. Fence widens from 10 to 14 cells when any row has typed data; renderer stays at 10 cells when none do (no churn diff on existing fences). Metric labels normalize to lowercase snake_case via `normalizeMetricLabel` (15-entry seed map for common founder metrics). The `consolidate` cycle phase gains semantic upsert keyed on `(page_id, claim, since_date)` — fixes the pre-existing F4 duplicate-takes bug where re-running the full cycle after `extract_facts` cleared `consolidated_at` would silently append duplicate takes via `MAX(row_num)+1`. Also writes chronological `valid_until` on each cluster's older facts. The `extract_facts` cycle phase batch-embeds via `gateway.embed()` before insert AND threads `pages.effective_date` as the `pageEffectiveDate` fallback for `valid_from` (precedence chain: fence-row > pageEffectiveDate > now()). The contradiction probe MUST NOT write `valid_until` — R1+R8 grep guard at `test/eval-contradictions/no-valid-until-write.test.ts` pins this. Codex outside-voice round caught F1 (v66 collision → v67), F2 (Haiku lives in `facts/extract.ts` not `extract-facts.ts` cycle phase), F3 (cycle didn't embed before insert), F4 (idempotency bug), F5+F6 (missed `fence-write.ts` caller + no Page object there → pageEffectiveDate is OPTIONAL), F7 (privacy regression — visibility filter added), F8 (ParsedFact needed typed-field extension for markdown system-of-record), F9 (dual scalar+federated sourceId). Plan: `~/.claude/plans/system-instruction-you-are-working-curious-jellyfish.md`. Tests: 258 across 12 files.
|
||||
- `src/commands/eval-suspected-contradictions.ts` + `src/core/eval-contradictions/{judge,runner,types,date-filter,cost-tracker,cache,severity-classify,cross-source,trends,calibration,judge-errors,auto-supersession,fixture-redact}.ts` (v0.32.6) — `gbrain eval suspected-contradictions [run|trend|review]`. Probe samples top-K retrieval pairs per query (cross-slug + intra-page chunk-vs-take), date pre-filters (3-rule layered — same-paragraph-dual-date overrides separation rule), LLM judge (query-conditioned per Codex; UTF-8-safe truncation; C1 confidence-floor double-enforcement; resolution_kind output drives M7 paste-ready commands), persistent cache keyed on `(chunk_a_hash, chunk_b_hash, model_id, prompt_version, truncation_policy)` (Codex outside-voice fix — prompt edits cleanly invalidate prior verdicts), Wilson 95% CI calibration on the headline percentage with `small_sample_note` when n<30, judge_errors as first-class typed counters (parse_fail/refusal/timeout/http_5xx/unknown — Codex fix to bias from silent skip), M5 trend writes to `eval_contradictions_runs`, M6 source-tier breakdown reuses `DEFAULT_SOURCE_BOOSTS` prefix logic, deterministic sampling (combined_score DESC + lex tiebreaker — stable cache hit-rate across re-runs). Hermetic via `judgeFn` + `searchFn` DI in the runner; never touches the real gateway in tests. Engine surface: `BrainEngine.listActiveTakesForPages` (P1 batched), `writeContradictionsRun` + `loadContradictionsTrend` (M5), `getContradictionCacheEntry` + `putContradictionCacheEntry` + `sweepContradictionCache` (P2). Schema migrations v51 + v52. MCP op `find_contradictions` (read scope, NOT localOnly, NOT in subagent allowlist — user-initiated only). M1 doctor check surfaces high-severity findings with paste-ready resolution commands. M2 synthesize phase pre-fetches latest probe's top-5-by-severity findings and threads them into `buildSynthesisPrompt` as an informational block. 226 hermetic unit tests + 12 real-Postgres E2E. Plan: `~/.claude/plans/system-instruction-you-are-working-hashed-dewdrop.md`. Architecture doc: `docs/contradictions.md`.
|
||||
- `src/core/think/index.ts` (v0.35.5.0 — gateway adapter) — `runThink` no longer instantiates `new Anthropic()` directly. The internal `LLMClient` instance is now built by a small adapter that wraps `gateway.chat()` from `src/core/ai/gateway.ts`, the canonical AI seam v0.31.12 established for chat/embed/expansion. Closes #952: stdio MCP launches (Claude Desktop, Cursor) don't inherit shell env, so the Anthropic SDK's env-only key resolution lost the key any user had set via `gbrain config set anthropic_api_key`. The gateway reads from `~/.gbrain/config.json` AND from env, so both paths work. Test seam preserved: `opts.client?: ThinkLLMClient` injection still works for the 12+ existing tests (`test/think-pipeline.serial.test.ts`, `test/think-gateway-adapter.test.ts`, etc.); `opts.stubResponse` continues to short-circuit before any LLM call. When neither key nor client is available, the graceful "no LLM available" stub still fires with the same `NO_ANTHROPIC_API_KEY` warning. v0.36.x TODO: drop `ThinkLLMClient` indirection entirely, migrate tests to `__setChatTransportForTests` seam from `src/core/ai/gateway.ts`.
|
||||
- `src/core/operations.ts` extension (v0.35.5.0 orphans fix) — `findOrphanPages` (both engines) now filters `p.deleted_at IS NULL` on the candidate side AND adds `JOIN pages src ON src.id = l.from_page_id WHERE src.deleted_at IS NULL` to the EXISTS subquery on the link-source side. Pre-v0.35.5 the query filtered nothing on `deleted_at`, so soft-deleted pages (v0.26.5 soft-delete shipped without updating this query) appeared as orphans AND links from soft-deleted source pages still suppressed live pages from orphan results. Closes #1021. Pinned by `test/orphans.test.ts`'s soft-delete cases.
|
||||
|
||||
@@ -12,6 +12,8 @@ GBrain is those patterns, generalized. 34 skills. Install in 30 minutes. Your ag
|
||||
|
||||
**New in v0.28.8 — LongMemEval in the box:** `gbrain eval longmemeval <dataset.jsonl>` runs the public [LongMemEval](https://huggingface.co/datasets/xiaowu0162/longmemeval) benchmark against gbrain's hybrid retrieval. One in-memory PGLite per run, `TRUNCATE` between questions (runtime-enumerated tables, schema-migration-safe), 25.9ms p50 per question on Apple Silicon. Your `~/.gbrain` brain is never touched. Retrieved chat content is sanitized with the same `INJECTION_PATTERNS` that protect takes — one source of truth for prompt-injection defense. Hand the JSONL output to LongMemEval's `evaluate_qa.py` to score.
|
||||
|
||||
**New in v0.35.7 — Temporal trajectory + founder scorecard.** Author typed metric assertions in the `## Facts` fence (`mrr=50000`, `arr=2000000`, `team_size=12`) and gbrain stores them as first-class typed columns. `gbrain eval trajectory companies/acme-example` prints the chronological history with regressions auto-flagged inline. `gbrain founder scorecard companies/acme-example` rolls up claim accuracy, consistency, growth direction, and red flags into a stable `schema_version: 1` JSON contract. New MCP op `find_trajectory` exposes the same data to agents (read scope, visibility-filtered for remote callers). The `consolidate` cycle phase now writes `valid_until` on chronologically-superseded facts AND uses semantic upsert on `(page_id, claim, since_date)` — re-running the dream cycle on stable input is now a true no-op (fixed a pre-existing duplicate-takes bug from prior versions).
|
||||
|
||||
> **~30 minutes to a fully working brain.** Database ready in 2 seconds (PGLite, no server). You just answer questions about API keys.
|
||||
|
||||
> **LLMs:** fetch [`llms.txt`](llms.txt) for the documentation map, or [`llms-full.txt`](llms-full.txt) for the same map with core docs inlined in one fetch. **Agents:** start with [`AGENTS.md`](AGENTS.md) (or [`CLAUDE.md`](CLAUDE.md) if you're Claude Code).
|
||||
@@ -770,6 +772,21 @@ EVAL
|
||||
gbrain eval longmemeval <dataset> Run public LongMemEval against gbrain hybrid retrieval (v0.28.8)
|
||||
[--limit N] [--retrieval-only] [--keyword-only] [--expansion]
|
||||
[--top-k K] [--model M] [--output FILE]
|
||||
gbrain eval suspected-contradictions Cached contradiction probe (v0.32.6, 6-verdict enum v0.35.3.1)
|
||||
[run|trend|review] [--severity S] [--budget-usd N]
|
||||
gbrain eval trajectory <entity> Chronological typed-claim trajectory + regressions (v0.35.7)
|
||||
[--metric M] [--since YYYY-MM-DD] [--until YYYY-MM-DD]
|
||||
[--limit N] [--json]
|
||||
|
||||
TEMPORAL (v0.35.7)
|
||||
gbrain founder scorecard <entity> Roll up an entity's typed claims + resolved-take outcomes into
|
||||
four signals: claim_accuracy / consistency / growth_trajectory /
|
||||
red_flags. Stable JSON contract (schema_version: 1).
|
||||
[--since YYYY-MM-DD] [--until YYYY-MM-DD] [--json]
|
||||
Backed by the find_trajectory MCP op (read scope, federated
|
||||
source-scoped, visibility-filtered for remote callers).
|
||||
Override regression threshold:
|
||||
export GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD=0.05
|
||||
|
||||
ADMIN
|
||||
gbrain doctor [--json] [--fast] Health checks (resolver, skills, DB, embeddings)
|
||||
|
||||
@@ -148,3 +148,19 @@ pay near-zero on re-runs (until you bump PROMPT_VERSION).
|
||||
- CHANGELOG: `## [0.32.6]` entry covers the whole release.
|
||||
- Cost discipline: `docs/eval-bench.md` for the recommended nightly cadence
|
||||
+ trend-tracking workflow.
|
||||
- **Temporal axis follow-on (v0.35.3.1 + v0.35.7):** v0.35.3.1 added a
|
||||
six-member verdict enum (`no_contradiction | contradiction |
|
||||
temporal_supersession | temporal_regression | temporal_evolution |
|
||||
negation_artifact`) and threaded `pages.effective_date` into the judge
|
||||
prompt so the probe stops crying wolf on legitimate change-over-time.
|
||||
v0.35.7 lands the trajectory substrate the probe pointed at:
|
||||
`gbrain eval trajectory <entity>` shows the chronological typed-claim
|
||||
history with regressions flagged inline; `gbrain founder scorecard
|
||||
<entity>` rolls up four signals (accuracy, consistency, growth
|
||||
direction, red flags) into a stable JSON contract. MCP op
|
||||
`find_trajectory` (read scope, visibility-filtered for remote callers)
|
||||
exposes the same data to agents. The probe's `temporal_supersession`
|
||||
verdict and the consolidate phase's `valid_until` writeback both
|
||||
preserve the `auto-supersession.ts:4` "NEVER auto-applies" invariant
|
||||
— the probe still emits paste-ready commands, only `consolidate`
|
||||
writes `valid_until` (R1+R8 grep guard pins this).
|
||||
|
||||
@@ -72,6 +72,14 @@ writing or reviewing an operation, consult `src/core/operations.ts` for the cont
|
||||
<dataset.jsonl>` (v0.28.8) runs against an isolated in-memory PGLite
|
||||
per question — your `~/.gbrain` is never opened. Full guide:
|
||||
[`docs/eval-bench.md`](./docs/eval-bench.md).
|
||||
- **Track a founder/company over time (v0.35.7):** when an entity has
|
||||
typed metric claims in its `## Facts` fence (`metric: mrr`, `value: 50000`,
|
||||
`unit: USD`, `period: monthly` columns), run
|
||||
`gbrain eval trajectory <entity-slug>` for the chronological history
|
||||
with regressions auto-flagged, or `gbrain founder scorecard <entity-slug>`
|
||||
for a four-signal JSON rollup (claim_accuracy / consistency /
|
||||
growth_trajectory / red_flags). MCP op `find_trajectory` exposes the
|
||||
same data — read scope, visibility-filtered for remote callers.
|
||||
- **Everything else:** [`./llms.txt`](./llms.txt) is the full documentation map.
|
||||
[`./llms-full.txt`](./llms-full.txt) is the same map with core docs inlined for
|
||||
single-fetch ingestion.
|
||||
@@ -192,6 +200,7 @@ strict behavior when unset.
|
||||
- `src/commands/eval-export.ts` (v0.25.0) — streams `eval_candidates` rows as NDJSON to stdout with `schema_version: 1` prefix on every line. EPIPE-safe, progress heartbeats on stderr, stable id-desc tiebreaker so `--since` windows never dupe/miss rows.
|
||||
- `src/commands/eval-prune.ts` (v0.25.0) — explicit retention cleanup. Requires `--older-than DUR`. `--dry-run` reports would-delete count.
|
||||
- `src/commands/eval-replay.ts` (v0.25.0) — contributor-facing replay tool. Reads NDJSON from `gbrain eval export`, re-runs each captured `query` / `search` op against the current brain, computes set-Jaccard@k between captured + current `retrieved_slugs`, top-1 stability rate, and latency Δ. Stable JSON shape (`schema_version: 1`) for CI gating; human mode prints a regression table. Pure Bun, zero new deps. The dev-loop half of BrainBench-Real that closes the gap between "data captured" and "data used to gate a PR." See `docs/eval-bench.md` for the workflow.
|
||||
- `src/commands/eval-trajectory.ts` + `src/commands/founder-scorecard.ts` + `src/core/trajectory.ts` (v0.35.7) — temporal trajectory + founder scorecard. The wave that turns the v0.35.3.1 date-aware contradiction probe into a useful temporal substrate. `gbrain eval trajectory <entity>` shows the chronological typed-claim history (mrr/arr/team_size/etc) with regressions auto-flagged inline; `gbrain founder scorecard <entity>` rolls up claim_accuracy / consistency / growth_trajectory / red_flags into one JSON. Pure-function math lives in `trajectory.ts`: `detectRegressions(points, threshold)` walks consecutive metric-value pairs per metric (10% drop default, env override `GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD`); `computeDriftScore(points)` returns `1 - mean(cosine(emb[i], emb[i-1]))` over existing embeddings (null when <3 embedded points). Backed by `BrainEngine.findTrajectory(opts)` — both Postgres and PGLite, single SQL query, deterministic `ORDER BY valid_from ASC, id ASC` (R3). Source-scoped via the v0.34.1.0 `sourceId` scalar / `sourceIds` array dual pattern (D-CDX-6); visibility-filtered for remote callers (D-CDX-1) — `recall`-equivalent posture. MCP op `find_trajectory` (read scope, NOT localOnly) registered after `find_experts`. Migration v67 adds four optional typed-claim columns (`claim_metric`, `claim_value`, `claim_unit`, `claim_period`) + a partial index on `(entity_slug, claim_metric, valid_from) WHERE claim_metric IS NOT NULL`. Fence widens from 10 to 14 cells when any row has typed data; renderer stays at 10 cells when none do (no churn diff on existing fences). Metric labels normalize to lowercase snake_case via `normalizeMetricLabel` (15-entry seed map for common founder metrics). The `consolidate` cycle phase gains semantic upsert keyed on `(page_id, claim, since_date)` — fixes the pre-existing F4 duplicate-takes bug where re-running the full cycle after `extract_facts` cleared `consolidated_at` would silently append duplicate takes via `MAX(row_num)+1`. Also writes chronological `valid_until` on each cluster's older facts. The `extract_facts` cycle phase batch-embeds via `gateway.embed()` before insert AND threads `pages.effective_date` as the `pageEffectiveDate` fallback for `valid_from` (precedence chain: fence-row > pageEffectiveDate > now()). The contradiction probe MUST NOT write `valid_until` — R1+R8 grep guard at `test/eval-contradictions/no-valid-until-write.test.ts` pins this. Codex outside-voice round caught F1 (v66 collision → v67), F2 (Haiku lives in `facts/extract.ts` not `extract-facts.ts` cycle phase), F3 (cycle didn't embed before insert), F4 (idempotency bug), F5+F6 (missed `fence-write.ts` caller + no Page object there → pageEffectiveDate is OPTIONAL), F7 (privacy regression — visibility filter added), F8 (ParsedFact needed typed-field extension for markdown system-of-record), F9 (dual scalar+federated sourceId). Plan: `~/.claude/plans/system-instruction-you-are-working-curious-jellyfish.md`. Tests: 258 across 12 files.
|
||||
- `src/commands/eval-suspected-contradictions.ts` + `src/core/eval-contradictions/{judge,runner,types,date-filter,cost-tracker,cache,severity-classify,cross-source,trends,calibration,judge-errors,auto-supersession,fixture-redact}.ts` (v0.32.6) — `gbrain eval suspected-contradictions [run|trend|review]`. Probe samples top-K retrieval pairs per query (cross-slug + intra-page chunk-vs-take), date pre-filters (3-rule layered — same-paragraph-dual-date overrides separation rule), LLM judge (query-conditioned per Codex; UTF-8-safe truncation; C1 confidence-floor double-enforcement; resolution_kind output drives M7 paste-ready commands), persistent cache keyed on `(chunk_a_hash, chunk_b_hash, model_id, prompt_version, truncation_policy)` (Codex outside-voice fix — prompt edits cleanly invalidate prior verdicts), Wilson 95% CI calibration on the headline percentage with `small_sample_note` when n<30, judge_errors as first-class typed counters (parse_fail/refusal/timeout/http_5xx/unknown — Codex fix to bias from silent skip), M5 trend writes to `eval_contradictions_runs`, M6 source-tier breakdown reuses `DEFAULT_SOURCE_BOOSTS` prefix logic, deterministic sampling (combined_score DESC + lex tiebreaker — stable cache hit-rate across re-runs). Hermetic via `judgeFn` + `searchFn` DI in the runner; never touches the real gateway in tests. Engine surface: `BrainEngine.listActiveTakesForPages` (P1 batched), `writeContradictionsRun` + `loadContradictionsTrend` (M5), `getContradictionCacheEntry` + `putContradictionCacheEntry` + `sweepContradictionCache` (P2). Schema migrations v51 + v52. MCP op `find_contradictions` (read scope, NOT localOnly, NOT in subagent allowlist — user-initiated only). M1 doctor check surfaces high-severity findings with paste-ready resolution commands. M2 synthesize phase pre-fetches latest probe's top-5-by-severity findings and threads them into `buildSynthesisPrompt` as an informational block. 226 hermetic unit tests + 12 real-Postgres E2E. Plan: `~/.claude/plans/system-instruction-you-are-working-hashed-dewdrop.md`. Architecture doc: `docs/contradictions.md`.
|
||||
- `src/core/think/index.ts` (v0.35.5.0 — gateway adapter) — `runThink` no longer instantiates `new Anthropic()` directly. The internal `LLMClient` instance is now built by a small adapter that wraps `gateway.chat()` from `src/core/ai/gateway.ts`, the canonical AI seam v0.31.12 established for chat/embed/expansion. Closes #952: stdio MCP launches (Claude Desktop, Cursor) don't inherit shell env, so the Anthropic SDK's env-only key resolution lost the key any user had set via `gbrain config set anthropic_api_key`. The gateway reads from `~/.gbrain/config.json` AND from env, so both paths work. Test seam preserved: `opts.client?: ThinkLLMClient` injection still works for the 12+ existing tests (`test/think-pipeline.serial.test.ts`, `test/think-gateway-adapter.test.ts`, etc.); `opts.stubResponse` continues to short-circuit before any LLM call. When neither key nor client is available, the graceful "no LLM available" stub still fires with the same `NO_ANTHROPIC_API_KEY` warning. v0.36.x TODO: drop `ThinkLLMClient` indirection entirely, migrate tests to `__setChatTransportForTests` seam from `src/core/ai/gateway.ts`.
|
||||
- `src/core/operations.ts` extension (v0.35.5.0 orphans fix) — `findOrphanPages` (both engines) now filters `p.deleted_at IS NULL` on the candidate side AND adds `JOIN pages src ON src.id = l.from_page_id WHERE src.deleted_at IS NULL` to the EXISTS subquery on the link-source side. Pre-v0.35.5 the query filtered nothing on `deleted_at`, so soft-deleted pages (v0.26.5 soft-delete shipped without updating this query) appeared as orphans AND links from soft-deleted source pages still suppressed live pages from orphan results. Closes #1021. Pinned by `test/orphans.test.ts`'s soft-delete cases.
|
||||
@@ -2134,6 +2143,8 @@ GBrain is those patterns, generalized. 34 skills. Install in 30 minutes. Your ag
|
||||
|
||||
**New in v0.28.8 — LongMemEval in the box:** `gbrain eval longmemeval <dataset.jsonl>` runs the public [LongMemEval](https://huggingface.co/datasets/xiaowu0162/longmemeval) benchmark against gbrain's hybrid retrieval. One in-memory PGLite per run, `TRUNCATE` between questions (runtime-enumerated tables, schema-migration-safe), 25.9ms p50 per question on Apple Silicon. Your `~/.gbrain` brain is never touched. Retrieved chat content is sanitized with the same `INJECTION_PATTERNS` that protect takes — one source of truth for prompt-injection defense. Hand the JSONL output to LongMemEval's `evaluate_qa.py` to score.
|
||||
|
||||
**New in v0.35.7 — Temporal trajectory + founder scorecard.** Author typed metric assertions in the `## Facts` fence (`mrr=50000`, `arr=2000000`, `team_size=12`) and gbrain stores them as first-class typed columns. `gbrain eval trajectory companies/acme-example` prints the chronological history with regressions auto-flagged inline. `gbrain founder scorecard companies/acme-example` rolls up claim accuracy, consistency, growth direction, and red flags into a stable `schema_version: 1` JSON contract. New MCP op `find_trajectory` exposes the same data to agents (read scope, visibility-filtered for remote callers). The `consolidate` cycle phase now writes `valid_until` on chronologically-superseded facts AND uses semantic upsert on `(page_id, claim, since_date)` — re-running the dream cycle on stable input is now a true no-op (fixed a pre-existing duplicate-takes bug from prior versions).
|
||||
|
||||
> **~30 minutes to a fully working brain.** Database ready in 2 seconds (PGLite, no server). You just answer questions about API keys.
|
||||
|
||||
> **LLMs:** fetch [`llms.txt`](llms.txt) for the documentation map, or [`llms-full.txt`](llms-full.txt) for the same map with core docs inlined in one fetch. **Agents:** start with [`AGENTS.md`](AGENTS.md) (or [`CLAUDE.md`](CLAUDE.md) if you're Claude Code).
|
||||
@@ -2892,6 +2903,21 @@ EVAL
|
||||
gbrain eval longmemeval <dataset> Run public LongMemEval against gbrain hybrid retrieval (v0.28.8)
|
||||
[--limit N] [--retrieval-only] [--keyword-only] [--expansion]
|
||||
[--top-k K] [--model M] [--output FILE]
|
||||
gbrain eval suspected-contradictions Cached contradiction probe (v0.32.6, 6-verdict enum v0.35.3.1)
|
||||
[run|trend|review] [--severity S] [--budget-usd N]
|
||||
gbrain eval trajectory <entity> Chronological typed-claim trajectory + regressions (v0.35.7)
|
||||
[--metric M] [--since YYYY-MM-DD] [--until YYYY-MM-DD]
|
||||
[--limit N] [--json]
|
||||
|
||||
TEMPORAL (v0.35.7)
|
||||
gbrain founder scorecard <entity> Roll up an entity's typed claims + resolved-take outcomes into
|
||||
four signals: claim_accuracy / consistency / growth_trajectory /
|
||||
red_flags. Stable JSON contract (schema_version: 1).
|
||||
[--since YYYY-MM-DD] [--until YYYY-MM-DD] [--json]
|
||||
Backed by the find_trajectory MCP op (read scope, federated
|
||||
source-scoped, visibility-filtered for remote callers).
|
||||
Override regression threshold:
|
||||
export GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD=0.05
|
||||
|
||||
ADMIN
|
||||
gbrain doctor [--json] [--fast] Health checks (resolver, skills, DB, embeddings)
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gbrain",
|
||||
"version": "0.35.6.0",
|
||||
"version": "0.35.7.0",
|
||||
"description": "Postgres-native personal knowledge brain with hybrid RAG search",
|
||||
"type": "module",
|
||||
"main": "src/core/index.ts",
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
---
|
||||
version: v0.35.7.0
|
||||
date: 2026-05-17
|
||||
feature_pitch: >
|
||||
Typed metric claims, chronological trajectory queries, and a founder
|
||||
scorecard CLI. The temporal axis the v0.35.3.1 contradiction probe
|
||||
needed in order to be useful, finally landed.
|
||||
---
|
||||
|
||||
# v0.35.7.0 — Temporal trajectory + founder scorecard
|
||||
|
||||
## What changed
|
||||
|
||||
- **Migration v67** (`facts_typed_claim_columns`) adds four optional
|
||||
columns to `facts`: `claim_metric`, `claim_value`, `claim_unit`,
|
||||
`claim_period`. All nullable — existing facts persist identically.
|
||||
- The `## Facts` fence widens from 10 to 14 columns when a row carries
|
||||
typed-claim data. Backward compat: fences without typed data stay at
|
||||
10 cells.
|
||||
- New MCP op: `find_trajectory` (read scope, not localOnly). Federated
|
||||
source-scoped, visibility-filtered for remote callers.
|
||||
- New CLI commands: `gbrain eval trajectory <entity>` and
|
||||
`gbrain founder scorecard <entity>`.
|
||||
- The `extract_facts` cycle phase now batch-embeds facts before insert
|
||||
AND threads `pages.effective_date` as the fallback `valid_from`.
|
||||
- The `consolidate` cycle phase now uses semantic upsert on `(page_id,
|
||||
claim, since_date)` so re-running the full cycle on stable input
|
||||
produces zero duplicate takes. Also writes chronological `valid_until`
|
||||
on each cluster's older facts.
|
||||
|
||||
## What agents need to do
|
||||
|
||||
Most of this is automatic on upgrade. Three things worth knowing:
|
||||
|
||||
### 1. Author typed-claim rows in `## Facts` fences
|
||||
|
||||
When you write facts about a company or person that include metric
|
||||
assertions, add the four typed columns to the fence row:
|
||||
|
||||
```markdown
|
||||
| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | claim_metric | claim_value | claim_unit | claim_period |
|
||||
|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|--------------|-------------|------------|--------------|
|
||||
| 1 | MRR hit $50K (Jan 2026) | fact | 1.0 | private | high | 2026-01-15 | | OH transcript | | mrr | 50000 | USD | monthly |
|
||||
| 2 | ARR target | commitment | 0.6 | private | high | 2026-01-15 | 2026-12-31 | OH transcript | predicted $1M ARR by Q4 | arr | 1000000 | USD | annual |
|
||||
```
|
||||
|
||||
The `claim_metric` column normalizes to lowercase snake_case (mrr, arr,
|
||||
team_size, runway, burn_rate, cac, ltv, mau, dau, churn_rate, revenue,
|
||||
fundraise, headcount, gross_margin, users). Unknown labels lowercase +
|
||||
underscore-collapse and pass through verbatim.
|
||||
|
||||
### 2. When the user asks "how has X trended"
|
||||
|
||||
Use `gbrain eval trajectory`:
|
||||
|
||||
```bash
|
||||
gbrain eval trajectory companies/acme-example
|
||||
gbrain eval trajectory companies/acme-example --metric mrr
|
||||
gbrain eval trajectory companies/acme-example --since 2026-01-01 --json
|
||||
```
|
||||
|
||||
Or the MCP op `find_trajectory` with `{entity_slug, metric?, since?,
|
||||
until?, limit?}`.
|
||||
|
||||
Regressions are auto-flagged at 10% drops by default. Drift score
|
||||
indicates narrative stability (0 = stable, 1 = every claim is unrelated;
|
||||
null when <3 embedded points).
|
||||
|
||||
### 3. When the user asks "is this founder consistent / reliable"
|
||||
|
||||
Use `gbrain founder scorecard`:
|
||||
|
||||
```bash
|
||||
gbrain founder scorecard companies/acme-example --json
|
||||
```
|
||||
|
||||
Returns four signals: `claim_accuracy` (over resolved takes),
|
||||
`consistency` (1 minus the fraction of metric value-changes),
|
||||
`growth_trajectory` (direction + delta per metric), `red_flags`
|
||||
(regressions + high drift + missed predictions).
|
||||
|
||||
JSON schema is `schema_version: 1`, additive-only across releases.
|
||||
|
||||
## Iron-rule contracts to preserve
|
||||
|
||||
- The contradiction probe MUST NOT write `valid_until` on facts.
|
||||
`consolidate` writes it; the probe emits paste-ready commands instead.
|
||||
This invariant is pinned by
|
||||
`test/eval-contradictions/no-valid-until-write.test.ts` (R1 + R8 grep
|
||||
guards).
|
||||
- `find_trajectory` MUST visibility-filter remote callers (`ctx.remote ===
|
||||
true` → SQL adds `AND visibility = 'world'`). Mirrors `recall`'s
|
||||
posture.
|
||||
- `consolidate`'s semantic upsert MUST be keyed on `(page_id, claim,
|
||||
since_date)`, NOT `MAX(row_num)+1`. Pinned by
|
||||
`test/consolidate-valid-until.test.ts` (R4b/R7).
|
||||
|
||||
## Where to look
|
||||
|
||||
- Cycle phase: `src/core/cycle/extract-facts.ts`,
|
||||
`src/core/cycle/phases/consolidate.ts`
|
||||
- Engine method: `BrainEngine.findTrajectory` in
|
||||
`src/core/engine.ts`, `src/core/postgres-engine.ts`,
|
||||
`src/core/pglite-engine.ts`
|
||||
- Trajectory math: `src/core/trajectory.ts`
|
||||
- MCP op: `src/core/operations.ts` (search for `find_trajectory`)
|
||||
- CLIs: `src/commands/eval-trajectory.ts`,
|
||||
`src/commands/founder-scorecard.ts`
|
||||
- Schema: `src/core/migrate.ts` (migration v67)
|
||||
+10
-1
@@ -27,7 +27,7 @@ for (const op of operations) {
|
||||
}
|
||||
|
||||
// CLI-only commands that bypass the operation layer
|
||||
const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache']);
|
||||
const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache', 'founder']);
|
||||
// CLI-only commands whose handlers print their own --help text. These are
|
||||
// excluded from the generic short-circuit so detailed per-command and
|
||||
// per-subcommand usage stays reachable.
|
||||
@@ -1165,6 +1165,15 @@ async function handleCliOnly(command: string, args: string[]) {
|
||||
await runTakes(engine, args);
|
||||
break;
|
||||
}
|
||||
case 'founder': {
|
||||
// v0.35.4 (T7) — founder scorecard. `gbrain founder scorecard <slug>`
|
||||
// rolls up Phase 2's typed-claim substrate into the four scorecard
|
||||
// metrics (claim accuracy, consistency, growth trajectory, red flags).
|
||||
// Thin-client routing handled inside the command file.
|
||||
const { runFounder } = await import('./commands/founder-scorecard.ts');
|
||||
await runFounder(engine, args);
|
||||
break;
|
||||
}
|
||||
case 'think': {
|
||||
const { runThinkCli } = await import('./commands/think.ts');
|
||||
await runThinkCli(engine, args);
|
||||
|
||||
@@ -0,0 +1,221 @@
|
||||
/**
|
||||
* gbrain eval trajectory <entity-slug> — chronological typed-claim
|
||||
* trajectory + regression detection + narrative drift score.
|
||||
*
|
||||
* v0.35.4 (T6) — pure data fn + JSON formatter + human formatter +
|
||||
* thin-client routing seam. Mirrors `gbrain salience` / `gbrain anomalies`
|
||||
* shape so the four temporal-axis read CLIs feel consistent.
|
||||
*
|
||||
* Usage:
|
||||
* gbrain eval trajectory companies/acme-example
|
||||
* gbrain eval trajectory companies/acme-example --metric mrr
|
||||
* gbrain eval trajectory companies/acme-example --since 2026-01-01 --until 2026-07-31
|
||||
* gbrain eval trajectory companies/acme-example --json
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import { loadConfig, isThinClient } from '../core/config.ts';
|
||||
import { callRemoteTool, unpackToolResult } from '../core/mcp-client.ts';
|
||||
import {
|
||||
computeTrajectoryStats,
|
||||
TRAJECTORY_SCHEMA_VERSION,
|
||||
type TrajectoryRegression,
|
||||
} from '../core/trajectory.ts';
|
||||
|
||||
interface RunOpts {
|
||||
entitySlug: string;
|
||||
metric?: string;
|
||||
since?: string;
|
||||
until?: string;
|
||||
limit?: number;
|
||||
json?: boolean;
|
||||
}
|
||||
|
||||
interface WireTrajectoryResult {
|
||||
points: Array<{
|
||||
fact_id: number;
|
||||
valid_from: string;
|
||||
metric: string | null;
|
||||
value: number | null;
|
||||
unit: string | null;
|
||||
period: string | null;
|
||||
text: string;
|
||||
source_session: string | null;
|
||||
source_markdown_slug: string | null;
|
||||
}>;
|
||||
regressions: TrajectoryRegression[];
|
||||
drift_score: number | null;
|
||||
schema_version: number;
|
||||
}
|
||||
|
||||
const HELP = `Usage: gbrain eval trajectory <entity-slug> [options]
|
||||
|
||||
Show the chronological claim trajectory for an entity (typed metric values
|
||||
over time, plus regressions and narrative drift score).
|
||||
|
||||
Examples:
|
||||
gbrain eval trajectory companies/acme-example
|
||||
gbrain eval trajectory companies/acme-example --metric mrr
|
||||
gbrain eval trajectory companies/acme-example --since 2026-01-01 --until 2026-07-31
|
||||
gbrain eval trajectory companies/acme-example --json
|
||||
|
||||
Options:
|
||||
--metric M Filter to a single canonical metric (mrr, arr, team_size, …)
|
||||
--since YYYY-MM-DD Lower bound on valid_from
|
||||
--until YYYY-MM-DD Upper bound on valid_from
|
||||
--limit N Max points (default 100, max 500)
|
||||
--json JSON output for agents (stable schema_version: 1)
|
||||
--help, -h Show this help
|
||||
`;
|
||||
|
||||
function parseArgs(args: string[]): RunOpts | { help: true } | { error: string } {
|
||||
const opts: Partial<RunOpts> = {};
|
||||
const positional: string[] = [];
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const a = args[i];
|
||||
if (a === '--help' || a === '-h') return { help: true };
|
||||
if (a === '--json') { opts.json = true; continue; }
|
||||
if (a === '--metric') { opts.metric = args[++i]; continue; }
|
||||
if (a === '--since') { opts.since = args[++i]; continue; }
|
||||
if (a === '--until') { opts.until = args[++i]; continue; }
|
||||
if (a === '--limit') {
|
||||
const n = parseInt(args[++i] ?? '', 10);
|
||||
if (Number.isFinite(n) && n > 0) opts.limit = n;
|
||||
continue;
|
||||
}
|
||||
if (a.startsWith('-')) {
|
||||
return { error: `Unknown flag: ${a}` };
|
||||
}
|
||||
positional.push(a);
|
||||
}
|
||||
if (positional.length !== 1) {
|
||||
return { error: 'Exactly one entity-slug positional argument is required.' };
|
||||
}
|
||||
return { ...(opts as RunOpts), entitySlug: positional[0] };
|
||||
}
|
||||
|
||||
export async function runEvalTrajectory(engine: BrainEngine, args: string[]): Promise<void> {
|
||||
const parsed = parseArgs(args);
|
||||
if ('help' in parsed) {
|
||||
console.log(HELP);
|
||||
return;
|
||||
}
|
||||
if ('error' in parsed) {
|
||||
console.error(parsed.error);
|
||||
console.error('');
|
||||
console.error(HELP);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
let result: WireTrajectoryResult;
|
||||
const cfg = loadConfig();
|
||||
if (isThinClient(cfg)) {
|
||||
// Thin-client install: route through the remote find_trajectory MCP op.
|
||||
const raw = await callRemoteTool(cfg!, 'find_trajectory', {
|
||||
entity_slug: parsed.entitySlug,
|
||||
metric: parsed.metric,
|
||||
since: parsed.since,
|
||||
until: parsed.until,
|
||||
limit: parsed.limit,
|
||||
}, { timeoutMs: 30_000 });
|
||||
result = unpackToolResult<WireTrajectoryResult>(raw);
|
||||
} else {
|
||||
// Local: call engine.findTrajectory directly, then compute derived
|
||||
// metrics via trajectory.ts. ctx.remote is implicitly false here so
|
||||
// visibility filtering is OFF — trusted local caller sees all facts.
|
||||
const points = await engine.findTrajectory({
|
||||
entitySlug: parsed.entitySlug,
|
||||
metric: parsed.metric,
|
||||
since: parsed.since,
|
||||
until: parsed.until,
|
||||
limit: parsed.limit,
|
||||
});
|
||||
const { regressions, drift_score } = computeTrajectoryStats(points);
|
||||
result = {
|
||||
points: points.map(p => ({
|
||||
fact_id: p.fact_id,
|
||||
valid_from: p.valid_from.toISOString().slice(0, 10),
|
||||
metric: p.metric,
|
||||
value: p.value,
|
||||
unit: p.unit,
|
||||
period: p.period,
|
||||
text: p.text,
|
||||
source_session: p.source_session,
|
||||
source_markdown_slug: p.source_markdown_slug,
|
||||
})),
|
||||
regressions,
|
||||
drift_score,
|
||||
schema_version: TRAJECTORY_SCHEMA_VERSION,
|
||||
};
|
||||
}
|
||||
|
||||
if (parsed.json) {
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
return;
|
||||
}
|
||||
|
||||
// Human format. Title + per-point line + regression callouts + drift.
|
||||
console.log(`Entity: ${parsed.entitySlug}`);
|
||||
if (parsed.metric) console.log(`Metric: ${parsed.metric}`);
|
||||
if (parsed.since || parsed.until) {
|
||||
console.log(`Window: ${parsed.since ?? '(unbounded)'} → ${parsed.until ?? '(now)'}`);
|
||||
}
|
||||
console.log('');
|
||||
|
||||
if (result.points.length === 0) {
|
||||
console.log('(no typed claims for this entity in the window)');
|
||||
return;
|
||||
}
|
||||
|
||||
// Build a regression lookup so each point's row can be annotated.
|
||||
const regBy = new Map<string, TrajectoryRegression>();
|
||||
for (const r of result.regressions) {
|
||||
regBy.set(`${r.metric}|${r.to_date}|${r.to_value}`, r);
|
||||
}
|
||||
|
||||
for (const p of result.points) {
|
||||
const metricCell = p.metric ?? '-';
|
||||
const valueCell = p.value === null ? '-' : formatValue(p.value, p.unit);
|
||||
const sourceCell = p.source_session ?? p.source_markdown_slug ?? '';
|
||||
let line = ` ${p.valid_from} ${pad(metricCell, 14)} ${pad(valueCell, 12)} (${sourceCell})`;
|
||||
const reg = p.metric && p.value !== null
|
||||
? regBy.get(`${p.metric}|${p.valid_from}|${p.value}`)
|
||||
: undefined;
|
||||
if (reg) {
|
||||
const pct = Math.abs(reg.delta_pct * 100).toFixed(1);
|
||||
line += ` [REGRESSION ↓${pct}%]`;
|
||||
}
|
||||
console.log(line);
|
||||
}
|
||||
|
||||
console.log('');
|
||||
if (result.drift_score === null) {
|
||||
console.log('Drift score: (insufficient embedded points; need 3+)');
|
||||
} else {
|
||||
const score = result.drift_score;
|
||||
const tier =
|
||||
score < 0.15 ? 'stable narrative' :
|
||||
score < 0.35 ? 'mild drift' :
|
||||
score < 0.6 ? 'moderate drift' :
|
||||
'narrative changing fast';
|
||||
console.log(`Drift score: ${score.toFixed(2)} (${tier})`);
|
||||
}
|
||||
if (result.regressions.length > 0) {
|
||||
console.log(`Regressions detected: ${result.regressions.length}`);
|
||||
}
|
||||
}
|
||||
|
||||
function pad(s: string, n: number): string {
|
||||
return s.length >= n ? s : s + ' '.repeat(n - s.length);
|
||||
}
|
||||
|
||||
function formatValue(v: number, unit: string | null): string {
|
||||
// Currency-style display for USD; plain for everything else. Keeps the
|
||||
// output readable without locale assumptions.
|
||||
if (unit === 'USD') {
|
||||
if (Math.abs(v) >= 1_000_000) return `$${(v / 1_000_000).toFixed(1)}M`;
|
||||
if (Math.abs(v) >= 1_000) return `$${(v / 1_000).toFixed(0)}K`;
|
||||
return `$${v.toFixed(0)}`;
|
||||
}
|
||||
return String(v);
|
||||
}
|
||||
@@ -67,6 +67,12 @@ export async function runEvalCommand(engine: BrainEngine, args: string[]): Promi
|
||||
const { runEvalSuspectedContradictions } = await import('./eval-suspected-contradictions.ts');
|
||||
return runEvalSuspectedContradictions(engine, args.slice(1));
|
||||
}
|
||||
if (sub === 'trajectory') {
|
||||
// v0.35.4 (T6) — chronological claim trajectory for an entity. Engine
|
||||
// is connected; thin-client routing handled inside the command file.
|
||||
const { runEvalTrajectory } = await import('./eval-trajectory.ts');
|
||||
return runEvalTrajectory(engine, args.slice(1));
|
||||
}
|
||||
// v0.32.3 search-lite — per-mode orchestrator + comparison report.
|
||||
if (sub === 'run-all') {
|
||||
const { runEvalRunAll } = await import('./eval-run-all.ts');
|
||||
|
||||
@@ -0,0 +1,371 @@
|
||||
/**
|
||||
* gbrain founder scorecard <entity-slug> — rolls up Phase 2's typed-claim
|
||||
* substrate + the takes outcome columns into a four-metric founder
|
||||
* scorecard.
|
||||
*
|
||||
* v0.35.4 (T7). Pure aggregation over facts + takes + the trajectory
|
||||
* derived metrics. Zero new schema. Zero LLM calls.
|
||||
*
|
||||
* Output (schema_version: 1, additive-only across releases per R5):
|
||||
*
|
||||
* {
|
||||
* schema_version: 1,
|
||||
* entity_slug: "companies/acme-example",
|
||||
* window: { since, until },
|
||||
* claim_accuracy: {
|
||||
* predicted: <number of takes with resolved_outcome != null>,
|
||||
* accurate: <number where resolved_outcome === true>,
|
||||
* pct: <accurate / predicted, or null when predicted=0>,
|
||||
* },
|
||||
* consistency: {
|
||||
* score: <1 - (unique_metric_changes / typed_facts), clamped [0,1]>,
|
||||
* metric_changes: <consecutive metric-value changes across typed facts>,
|
||||
* typed_facts: <count of typed-claim facts in the window>,
|
||||
* },
|
||||
* growth_trajectory: [
|
||||
* { metric, direction: 'up'|'down'|'flat', latest_delta_pct },
|
||||
* ...
|
||||
* ],
|
||||
* red_flags: [
|
||||
* { kind: 'regression', metric, text },
|
||||
* { kind: 'narrative_drift', text },
|
||||
* { kind: 'missed_prediction', text },
|
||||
* ],
|
||||
* }
|
||||
*
|
||||
* Usage:
|
||||
* gbrain founder scorecard companies/acme-example
|
||||
* gbrain founder scorecard companies/acme-example --since 2025-05-17 --until 2026-05-17
|
||||
* gbrain founder scorecard companies/acme-example --json
|
||||
*/
|
||||
|
||||
import type { BrainEngine, Take, TrajectoryPoint } from '../core/engine.ts';
|
||||
import { loadConfig, isThinClient } from '../core/config.ts';
|
||||
import { callRemoteTool, unpackToolResult } from '../core/mcp-client.ts';
|
||||
import {
|
||||
computeTrajectoryStats,
|
||||
TRAJECTORY_SCHEMA_VERSION,
|
||||
} from '../core/trajectory.ts';
|
||||
|
||||
export interface FounderScorecard {
|
||||
schema_version: number;
|
||||
entity_slug: string;
|
||||
window: { since: string | null; until: string | null };
|
||||
claim_accuracy: {
|
||||
predicted: number;
|
||||
accurate: number;
|
||||
pct: number | null;
|
||||
};
|
||||
consistency: {
|
||||
score: number | null;
|
||||
metric_changes: number;
|
||||
typed_facts: number;
|
||||
};
|
||||
growth_trajectory: Array<{
|
||||
metric: string;
|
||||
direction: 'up' | 'down' | 'flat';
|
||||
latest_delta_pct: number;
|
||||
}>;
|
||||
red_flags: Array<{
|
||||
kind: 'regression' | 'narrative_drift' | 'missed_prediction';
|
||||
metric?: string;
|
||||
text: string;
|
||||
}>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Pure data function — given a sorted trajectory + takes window, compute
|
||||
* the scorecard. Exported for tests so the rollup math is exercised
|
||||
* without a DB round trip.
|
||||
*/
|
||||
export function computeFounderScorecard(args: {
|
||||
entitySlug: string;
|
||||
windowSince: string | null;
|
||||
windowUntil: string | null;
|
||||
points: TrajectoryPoint[];
|
||||
takes: Take[];
|
||||
driftThresholdRedFlag?: number;
|
||||
}): FounderScorecard {
|
||||
const driftRedFlag = args.driftThresholdRedFlag ?? 0.5;
|
||||
const { regressions, drift_score } = computeTrajectoryStats(args.points);
|
||||
|
||||
// claim_accuracy — over resolved takes only.
|
||||
const resolved = args.takes.filter(t => t.resolved_outcome !== null);
|
||||
const accurate = resolved.filter(t => t.resolved_outcome === true).length;
|
||||
const accuracyPct = resolved.length > 0 ? accurate / resolved.length : null;
|
||||
|
||||
// consistency — count consecutive value changes per metric. A 'change'
|
||||
// is a pair where the relative delta is >=5%. The score normalizes
|
||||
// by total typed facts so a long-stable trajectory scores 1.0.
|
||||
const byMetric = new Map<string, TrajectoryPoint[]>();
|
||||
for (const p of args.points) {
|
||||
if (p.metric === null || p.value === null) continue;
|
||||
if (!byMetric.has(p.metric)) byMetric.set(p.metric, []);
|
||||
byMetric.get(p.metric)!.push(p);
|
||||
}
|
||||
let metricChanges = 0;
|
||||
let typedFacts = 0;
|
||||
for (const series of byMetric.values()) {
|
||||
typedFacts += series.length;
|
||||
for (let i = 1; i < series.length; i++) {
|
||||
const a = series[i - 1].value!;
|
||||
const b = series[i].value!;
|
||||
if (a === 0) continue;
|
||||
if (Math.abs(b - a) / Math.abs(a) >= 0.05) metricChanges += 1;
|
||||
}
|
||||
}
|
||||
const consistencyScore = typedFacts > 0
|
||||
? Math.max(0, Math.min(1, 1 - metricChanges / typedFacts))
|
||||
: null;
|
||||
|
||||
// growth_trajectory — per metric, the most recent delta direction.
|
||||
const growth: FounderScorecard['growth_trajectory'] = [];
|
||||
for (const [metric, series] of byMetric) {
|
||||
if (series.length < 2) continue;
|
||||
const latest = series[series.length - 1].value!;
|
||||
const prior = series[series.length - 2].value!;
|
||||
if (prior === 0) continue;
|
||||
const delta = (latest - prior) / prior;
|
||||
const dir: 'up' | 'down' | 'flat' =
|
||||
Math.abs(delta) < 0.01 ? 'flat' : (delta > 0 ? 'up' : 'down');
|
||||
growth.push({ metric, direction: dir, latest_delta_pct: delta });
|
||||
}
|
||||
// Stable ordering: alphabetical by metric so the JSON is deterministic.
|
||||
growth.sort((a, b) => a.metric.localeCompare(b.metric));
|
||||
|
||||
// red_flags — surface regressions, big narrative drift, and missed
|
||||
// predictions (resolved=false on a take).
|
||||
const redFlags: FounderScorecard['red_flags'] = [];
|
||||
for (const r of regressions) {
|
||||
const pct = Math.abs(r.delta_pct * 100).toFixed(1);
|
||||
redFlags.push({
|
||||
kind: 'regression',
|
||||
metric: r.metric,
|
||||
text: `${r.metric} fell ${pct}% (${r.from_date} → ${r.to_date})`,
|
||||
});
|
||||
}
|
||||
if (drift_score !== null && drift_score >= driftRedFlag) {
|
||||
redFlags.push({
|
||||
kind: 'narrative_drift',
|
||||
text: `Narrative drift score ${drift_score.toFixed(2)} — claims are diverging rapidly`,
|
||||
});
|
||||
}
|
||||
const missed = args.takes.filter(t => t.resolved_outcome === false);
|
||||
for (const m of missed) {
|
||||
redFlags.push({
|
||||
kind: 'missed_prediction',
|
||||
text: `Missed prediction: ${truncate(m.claim, 200)}`,
|
||||
});
|
||||
}
|
||||
|
||||
return {
|
||||
schema_version: TRAJECTORY_SCHEMA_VERSION,
|
||||
entity_slug: args.entitySlug,
|
||||
window: { since: args.windowSince, until: args.windowUntil },
|
||||
claim_accuracy: {
|
||||
predicted: resolved.length,
|
||||
accurate,
|
||||
pct: accuracyPct,
|
||||
},
|
||||
consistency: {
|
||||
score: consistencyScore,
|
||||
metric_changes: metricChanges,
|
||||
typed_facts: typedFacts,
|
||||
},
|
||||
growth_trajectory: growth,
|
||||
red_flags: redFlags,
|
||||
};
|
||||
}
|
||||
|
||||
function truncate(s: string, n: number): string {
|
||||
return s.length <= n ? s : s.slice(0, n - 3) + '...';
|
||||
}
|
||||
|
||||
interface RunOpts {
|
||||
sub: 'scorecard';
|
||||
entitySlug: string;
|
||||
since?: string;
|
||||
until?: string;
|
||||
json?: boolean;
|
||||
}
|
||||
|
||||
const HELP = `Usage: gbrain founder scorecard <entity-slug> [options]
|
||||
|
||||
Rolls up the entity's typed-claim trajectory + resolved-take outcomes into
|
||||
the four founder-evaluation metrics: claim accuracy, consistency, growth
|
||||
trajectory, red flags.
|
||||
|
||||
Options:
|
||||
--since YYYY-MM-DD Lower bound on valid_from / take created_at (default: 1 year ago)
|
||||
--until YYYY-MM-DD Upper bound (default: today)
|
||||
--json JSON output (stable schema_version: 1)
|
||||
--help, -h Show this help
|
||||
|
||||
Examples:
|
||||
gbrain founder scorecard companies/acme-example
|
||||
gbrain founder scorecard companies/acme-example --since 2025-05-17 --json
|
||||
`;
|
||||
|
||||
function parseArgs(args: string[]): RunOpts | { help: true } | { error: string } {
|
||||
if (args[0] !== 'scorecard') {
|
||||
return { error: `Unknown founder subcommand: ${args[0] ?? '(none)'}. Did you mean "founder scorecard"?` };
|
||||
}
|
||||
const opts: Partial<RunOpts> = { sub: 'scorecard' };
|
||||
const positional: string[] = [];
|
||||
for (let i = 1; i < args.length; i++) {
|
||||
const a = args[i];
|
||||
if (a === '--help' || a === '-h') return { help: true };
|
||||
if (a === '--json') { opts.json = true; continue; }
|
||||
if (a === '--since') { opts.since = args[++i]; continue; }
|
||||
if (a === '--until') { opts.until = args[++i]; continue; }
|
||||
if (a.startsWith('-')) return { error: `Unknown flag: ${a}` };
|
||||
positional.push(a);
|
||||
}
|
||||
if (positional.length !== 1) {
|
||||
return { error: 'Exactly one entity-slug positional argument is required.' };
|
||||
}
|
||||
return { ...(opts as RunOpts), entitySlug: positional[0] };
|
||||
}
|
||||
|
||||
export async function runFounder(engine: BrainEngine, args: string[]): Promise<void> {
|
||||
const parsed = parseArgs(args);
|
||||
if ('help' in parsed) {
|
||||
console.log(HELP);
|
||||
return;
|
||||
}
|
||||
if ('error' in parsed) {
|
||||
console.error(parsed.error);
|
||||
console.error('');
|
||||
console.error(HELP);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Default window: last year.
|
||||
const now = new Date();
|
||||
const oneYearAgo = new Date(now);
|
||||
oneYearAgo.setFullYear(oneYearAgo.getFullYear() - 1);
|
||||
const windowSince = parsed.since ?? oneYearAgo.toISOString().slice(0, 10);
|
||||
const windowUntil = parsed.until ?? now.toISOString().slice(0, 10);
|
||||
|
||||
let scorecard: FounderScorecard;
|
||||
const cfg = loadConfig();
|
||||
if (isThinClient(cfg)) {
|
||||
// Thin-client path: server-side find_trajectory + listTakes computed
|
||||
// locally is too heavyweight for one HTTP call, so this path fetches
|
||||
// the trajectory via the remote MCP op and reads takes through a
|
||||
// future `takes_list` op. For now thin-client returns a degraded
|
||||
// scorecard built from trajectory only (no take outcomes).
|
||||
// TODO(v0.35.5): add `takes_list_by_entity` MCP op + thin-client wire-up.
|
||||
const raw = await callRemoteTool(cfg!, 'find_trajectory', {
|
||||
entity_slug: parsed.entitySlug,
|
||||
since: windowSince,
|
||||
until: windowUntil,
|
||||
}, { timeoutMs: 30_000 });
|
||||
const trajWire = unpackToolResult<{
|
||||
points: Array<{
|
||||
fact_id: number; valid_from: string;
|
||||
metric: string | null; value: number | null;
|
||||
unit: string | null; period: string | null;
|
||||
text: string; source_session: string | null; source_markdown_slug: string | null;
|
||||
}>;
|
||||
}>(raw);
|
||||
const points: TrajectoryPoint[] = trajWire.points.map(p => ({
|
||||
fact_id: p.fact_id,
|
||||
valid_from: new Date(p.valid_from),
|
||||
metric: p.metric,
|
||||
value: p.value,
|
||||
unit: p.unit,
|
||||
period: p.period,
|
||||
text: p.text,
|
||||
source_session: p.source_session,
|
||||
source_markdown_slug: p.source_markdown_slug,
|
||||
embedding: null,
|
||||
}));
|
||||
scorecard = computeFounderScorecard({
|
||||
entitySlug: parsed.entitySlug,
|
||||
windowSince,
|
||||
windowUntil,
|
||||
points,
|
||||
takes: [],
|
||||
});
|
||||
} else {
|
||||
// Local: full pipeline. Get the trajectory + the entity's resolved takes.
|
||||
const points = await engine.findTrajectory({
|
||||
entitySlug: parsed.entitySlug,
|
||||
since: windowSince,
|
||||
until: windowUntil,
|
||||
});
|
||||
let takes: Take[] = [];
|
||||
try {
|
||||
takes = await engine.listTakes({
|
||||
page_slug: parsed.entitySlug,
|
||||
active: true,
|
||||
resolved: true,
|
||||
limit: 100,
|
||||
});
|
||||
} catch {
|
||||
// Some entity pages don't exist; treat as no takes.
|
||||
takes = [];
|
||||
}
|
||||
scorecard = computeFounderScorecard({
|
||||
entitySlug: parsed.entitySlug,
|
||||
windowSince,
|
||||
windowUntil,
|
||||
points,
|
||||
takes,
|
||||
});
|
||||
}
|
||||
|
||||
if (parsed.json) {
|
||||
console.log(JSON.stringify(scorecard, null, 2));
|
||||
return;
|
||||
}
|
||||
|
||||
// Human format.
|
||||
console.log(`Entity: ${scorecard.entity_slug}`);
|
||||
console.log(`Window: ${scorecard.window.since} → ${scorecard.window.until}`);
|
||||
console.log('');
|
||||
|
||||
console.log('Claim accuracy:');
|
||||
if (scorecard.claim_accuracy.predicted === 0) {
|
||||
console.log(' (no resolved predictions in window)');
|
||||
} else {
|
||||
const pct = scorecard.claim_accuracy.pct === null
|
||||
? 'n/a'
|
||||
: `${(scorecard.claim_accuracy.pct * 100).toFixed(1)}%`;
|
||||
console.log(` ${scorecard.claim_accuracy.accurate}/${scorecard.claim_accuracy.predicted} predictions accurate (${pct})`);
|
||||
}
|
||||
console.log('');
|
||||
|
||||
console.log('Consistency:');
|
||||
if (scorecard.consistency.typed_facts === 0) {
|
||||
console.log(' (no typed claims in window)');
|
||||
} else {
|
||||
const score = scorecard.consistency.score === null
|
||||
? 'n/a'
|
||||
: scorecard.consistency.score.toFixed(2);
|
||||
console.log(` score ${score} (${scorecard.consistency.metric_changes} changes across ${scorecard.consistency.typed_facts} typed facts)`);
|
||||
}
|
||||
console.log('');
|
||||
|
||||
console.log('Growth trajectory:');
|
||||
if (scorecard.growth_trajectory.length === 0) {
|
||||
console.log(' (no metrics with 2+ data points)');
|
||||
} else {
|
||||
for (const g of scorecard.growth_trajectory) {
|
||||
const sign = g.direction === 'up' ? '↑' : g.direction === 'down' ? '↓' : '→';
|
||||
const pct = (g.latest_delta_pct * 100).toFixed(1);
|
||||
console.log(` ${g.metric}: ${sign} ${pct}%`);
|
||||
}
|
||||
}
|
||||
console.log('');
|
||||
|
||||
console.log('Red flags:');
|
||||
if (scorecard.red_flags.length === 0) {
|
||||
console.log(' (none)');
|
||||
} else {
|
||||
for (const r of scorecard.red_flags) {
|
||||
console.log(` [${r.kind}] ${r.text}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -34,6 +34,7 @@
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import { parseFactsFence } from '../facts-fence.ts';
|
||||
import { extractFactsFromFenceText } from '../facts/extract-from-fence.ts';
|
||||
import { embed, isAvailable } from '../ai/gateway.ts';
|
||||
|
||||
export interface ExtractFactsOpts {
|
||||
/** Subset of slugs to reconcile. undefined = walk every page in the brain. */
|
||||
@@ -136,7 +137,40 @@ export async function runExtractFacts(
|
||||
|
||||
if (parsed.facts.length === 0) continue;
|
||||
|
||||
const extracted = extractFactsFromFenceText(parsed.facts, slug, sourceId);
|
||||
// v0.35.4 (D-ENG-1) — thread page.effective_date as the fallback
|
||||
// valid_from. Without this, fence rows without explicit `validFrom:`
|
||||
// land with `valid_from = now()` (import timestamp) and every
|
||||
// trajectory query against the page returns import dates instead of
|
||||
// claim dates.
|
||||
const pageEffectiveDate = page.effective_date ? new Date(page.effective_date) : null;
|
||||
const extracted = extractFactsFromFenceText(parsed.facts, slug, sourceId, { pageEffectiveDate });
|
||||
|
||||
// v0.35.4 (D-CDX-3) — batch-embed before insert. Without this,
|
||||
// cycle-inserted facts land with `embedding = NULL`, which breaks
|
||||
// consolidate's cosine clustering AND the drift_score formula in
|
||||
// find_trajectory. Falls open: if the embedding gateway is
|
||||
// unavailable (no API key configured), facts still insert with
|
||||
// NULL embeddings — drift_score gracefully returns null and
|
||||
// clustering falls back to recency.
|
||||
if (isAvailable('embedding') && extracted.length > 0) {
|
||||
try {
|
||||
const texts = extracted.map(e => e.fact);
|
||||
const embeddings = await embed(texts);
|
||||
// Defensive: embed should return one vector per input; if the
|
||||
// gateway returns a partial array (provider partial-batch retry
|
||||
// returning fewer than requested), only fill what we have.
|
||||
for (let i = 0; i < extracted.length && i < embeddings.length; i++) {
|
||||
extracted[i].embedding = embeddings[i];
|
||||
}
|
||||
} catch (err) {
|
||||
// Embedding failure is non-fatal — facts still get inserted, just
|
||||
// without embeddings. Cycle phase status stays 'ok'.
|
||||
result.warnings.push(
|
||||
`${slug}: extract_facts batch embed failed: ${err instanceof Error ? err.message : String(err)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const inserted = await engine.insertFacts(extracted, { source_id: sourceId }); // gbrain-allow-direct-insert: extract_facts cycle phase reconciles fence → DB
|
||||
result.factsInserted += inserted.inserted;
|
||||
}
|
||||
|
||||
@@ -147,37 +147,95 @@ export async function runPhaseConsolidate(
|
||||
continue;
|
||||
}
|
||||
|
||||
const inserted = await engine.addTakesBatch([{
|
||||
page_id: pageId,
|
||||
row_num: nextRowNum,
|
||||
claim: best.fact,
|
||||
kind: 'fact',
|
||||
holder: 'self',
|
||||
weight: clamp01(avgWeight),
|
||||
since_date: sinceISO,
|
||||
source: sources.slice(0, 200),
|
||||
active: true,
|
||||
}]);
|
||||
if (inserted < 1) continue;
|
||||
|
||||
// Read back the new takes.id by (page_id, row_num).
|
||||
const idRows = await engine.executeRaw<{ id: number }>(
|
||||
`SELECT id FROM takes WHERE page_id = $1 AND row_num = $2`,
|
||||
[pageId, nextRowNum],
|
||||
// v0.35.4 (D-CDX-4) — semantic upsert. The full dream cycle runs
|
||||
// `extract_facts` BEFORE `consolidate`; `extract_facts` hard-deletes
|
||||
// and re-inserts page facts via deleteFactsForPage + insertFacts,
|
||||
// which clears `consolidated_at` on every fact. Without this lookup,
|
||||
// a second cycle run would re-INSERT a duplicate take via
|
||||
// `MAX(row_num)+1`, silently poisoning trajectory + scorecard data.
|
||||
// Match on (page_id, claim, since_date) — the natural identity of a
|
||||
// promoted take.
|
||||
const existing = await engine.executeRaw<{ id: number }>(
|
||||
`SELECT id FROM takes
|
||||
WHERE page_id = $1 AND claim = $2 AND since_date = $3
|
||||
LIMIT 1`,
|
||||
[pageId, best.fact, sinceISO],
|
||||
);
|
||||
if (idRows.length === 0) {
|
||||
|
||||
let takeId: number;
|
||||
if (existing.length > 0) {
|
||||
// Re-promotion of a cluster we already wrote a take for. Refresh
|
||||
// the source-aggregation string (new fact rows may carry new
|
||||
// source_session values that the prior run didn't see); leave
|
||||
// row_num + weight untouched to keep the take's identity stable.
|
||||
takeId = existing[0].id;
|
||||
await engine.executeRaw(
|
||||
`UPDATE takes SET source = $1, updated_at = now() WHERE id = $2`,
|
||||
[sources.slice(0, 200), takeId],
|
||||
);
|
||||
} else {
|
||||
const inserted = await engine.addTakesBatch([{
|
||||
page_id: pageId,
|
||||
row_num: nextRowNum,
|
||||
claim: best.fact,
|
||||
kind: 'fact',
|
||||
holder: 'self',
|
||||
weight: clamp01(avgWeight),
|
||||
since_date: sinceISO,
|
||||
source: sources.slice(0, 200),
|
||||
active: true,
|
||||
}]);
|
||||
if (inserted < 1) continue;
|
||||
|
||||
const idRows = await engine.executeRaw<{ id: number }>(
|
||||
`SELECT id FROM takes WHERE page_id = $1 AND row_num = $2`,
|
||||
[pageId, nextRowNum],
|
||||
);
|
||||
if (idRows.length === 0) {
|
||||
nextRowNum += 1;
|
||||
continue;
|
||||
}
|
||||
takeId = idRows[0].id;
|
||||
nextRowNum += 1;
|
||||
continue;
|
||||
takesWritten += 1;
|
||||
}
|
||||
const takeId = idRows[0].id;
|
||||
nextRowNum += 1;
|
||||
takesWritten += 1;
|
||||
|
||||
// Mark all contributing facts consolidated.
|
||||
for (const f of cluster) {
|
||||
await engine.consolidateFact(f.id, takeId);
|
||||
factsConsolidated += 1;
|
||||
}
|
||||
|
||||
// v0.35.4 (D-CDX-4 part 2) — chronological valid_until writeback.
|
||||
// Sort the cluster by (valid_from ASC, id ASC); walk consecutive
|
||||
// pairs; stamp the older fact's valid_until = next_newer.valid_from.
|
||||
// The newest fact keeps valid_until = NULL. This makes the facts
|
||||
// table a proper bitemporal record without the contradiction probe
|
||||
// having to mutate it (preserves auto-supersession.ts:4 invariant —
|
||||
// see also R8 test guard).
|
||||
//
|
||||
// Idempotent: re-running on the same cluster produces the same
|
||||
// chronological order and the same valid_until values. No-op if
|
||||
// valid_until is already correct.
|
||||
const chronological = [...cluster].sort((a, b) => {
|
||||
const t = a.valid_from.getTime() - b.valid_from.getTime();
|
||||
if (t !== 0) return t;
|
||||
return a.id - b.id;
|
||||
});
|
||||
for (let i = 0; i < chronological.length - 1; i++) {
|
||||
const older = chronological[i];
|
||||
const newer = chronological[i + 1];
|
||||
await engine.executeRaw(
|
||||
// Only UPDATE when the new value would actually change. Avoids
|
||||
// touching updated_at on no-op rewrites and keeps idempotency
|
||||
// observable in the DB (zero affected rows on stable re-run).
|
||||
`UPDATE facts
|
||||
SET valid_until = $1
|
||||
WHERE id = $2
|
||||
AND (valid_until IS DISTINCT FROM $1)`,
|
||||
[newer.valid_from, older.id],
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -369,6 +369,17 @@ export interface NewFact {
|
||||
confidence?: number; // [0,1], default 1.0
|
||||
notability?: 'high' | 'medium' | 'low'; // salience filter for extraction gate
|
||||
embedding?: Float32Array | null; // pre-computed; if null, insertFact computes via gateway
|
||||
/**
|
||||
* v0.35.4 (D-CDX-5) — typed-claim fields. Optional. When populated,
|
||||
* `gbrain eval trajectory` + `find_trajectory` MCP op consume them for
|
||||
* chronological regression detection and drift_score. `claim_metric` is
|
||||
* normalized to lowercase snake_case by the extraction layer before
|
||||
* this method sees it; the engine stores verbatim.
|
||||
*/
|
||||
claim_metric?: string | null;
|
||||
claim_value?: number | null;
|
||||
claim_unit?: string | null;
|
||||
claim_period?: string | null;
|
||||
}
|
||||
|
||||
/** Options shared by list-facts methods. */
|
||||
@@ -402,6 +413,58 @@ export interface FactsHealth {
|
||||
p99_latency_ms?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.35.4 (D-CDX-6) — Options for `BrainEngine.findTrajectory`.
|
||||
*
|
||||
* `sourceId` (scalar fast path) and `sourceIds` (federated array) follow
|
||||
* the v0.34.1.0 search* pattern: when `sourceIds` is set the engine
|
||||
* applies `WHERE source_id = ANY($N::text[])`; otherwise scalar predicate
|
||||
* with `sourceId ?? 'default'`.
|
||||
*
|
||||
* `remote` (D-CDX-1) gates the visibility filter: when true the engine
|
||||
* adds `AND visibility = 'world'`, mirroring `recall`'s posture for
|
||||
* untrusted callers. Local CLI keeps `remote: false` and sees both
|
||||
* private + world facts.
|
||||
*/
|
||||
export interface TrajectoryOpts {
|
||||
entitySlug: string;
|
||||
/** Single-source scope; default 'default' when both this and sourceIds are unset. */
|
||||
sourceId?: string;
|
||||
/** Federated array scope (mutually exclusive with sourceId; the array wins when set). */
|
||||
sourceIds?: string[];
|
||||
/** When true, filters to visibility='world' only. Set by MCP layer from ctx.remote. */
|
||||
remote?: boolean;
|
||||
/** Metric filter. When set, only facts with this canonical metric label participate. */
|
||||
metric?: string;
|
||||
/** Lower bound on valid_from (inclusive). YYYY-MM-DD or full ISO. */
|
||||
since?: string | Date;
|
||||
/** Upper bound on valid_from (inclusive). YYYY-MM-DD or full ISO. */
|
||||
until?: string | Date;
|
||||
/** Cap on points returned. Default 100, max 500. */
|
||||
limit?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* A single point in an entity's claim trajectory. Carries the typed-claim
|
||||
* fields when populated (drives regression detection), the underlying
|
||||
* fact text (for display), provenance (source_session, source_markdown_slug),
|
||||
* and the raw embedding so the caller can compute drift_score without a
|
||||
* second SQL round-trip.
|
||||
*/
|
||||
export interface TrajectoryPoint {
|
||||
fact_id: number;
|
||||
valid_from: Date;
|
||||
metric: string | null;
|
||||
value: number | null;
|
||||
unit: string | null;
|
||||
period: string | null;
|
||||
text: string;
|
||||
source_session: string | null;
|
||||
source_markdown_slug: string | null;
|
||||
/** Raw embedding for drift computation; null when the fact was inserted without one. */
|
||||
embedding: Float32Array | null;
|
||||
}
|
||||
|
||||
/** Maximum results returned by search operations. Internal bulk operations (listPages) are not clamped. */
|
||||
export const MAX_SEARCH_LIMIT = 100;
|
||||
|
||||
@@ -1134,6 +1197,21 @@ export interface BrainEngine {
|
||||
*/
|
||||
consolidateFact(id: number, takeId: number): Promise<void>;
|
||||
|
||||
/**
|
||||
* v0.35.4 (D-CDX-1 + D-CDX-6) — chronological fact trajectory for an
|
||||
* entity. Returns points ordered by (valid_from ASC, id ASC) so the
|
||||
* caller can compute regressions and drift_score deterministically.
|
||||
*
|
||||
* - Source-scoped via `sourceId` (scalar) OR `sourceIds` (federated array).
|
||||
* - Visibility-filtered: when `opts.remote=true`, only `visibility='world'`
|
||||
* facts are returned. Trusted local callers see both private + world.
|
||||
* - Optional metric filter restricts to a single normalized metric label.
|
||||
* - Active-only by default (expired_at IS NULL); soft-deleted entities
|
||||
* on the pages side are NOT filtered here — trajectory is a facts-table
|
||||
* query and doesn't JOIN pages.
|
||||
*/
|
||||
findTrajectory(opts: TrajectoryOpts): Promise<TrajectoryPoint[]>;
|
||||
|
||||
/** Per-source operational metrics for `gbrain doctor` facts_health check. */
|
||||
getFactsHealth(source_id: string): Promise<FactsHealth>;
|
||||
|
||||
|
||||
+73
-5
@@ -98,6 +98,25 @@ export interface ParsedFact {
|
||||
*/
|
||||
supersededBy?: number;
|
||||
forgotten?: boolean;
|
||||
/**
|
||||
* v0.35.4 typed-claim fields (D-CDX-5). Optional. When present, drives
|
||||
* `gbrain eval trajectory` + the `find_trajectory` MCP op chronological
|
||||
* regression detection. The fence layout widens from 10 to 14 columns
|
||||
* when any row in the table has a non-undefined typed field; otherwise
|
||||
* stays 10-cell for backward compat with existing fences.
|
||||
*
|
||||
* - `claimMetric`: lowercase snake_case after normalization
|
||||
* (`mrr`, `arr`, `team_size`, …). Free-text labels accepted; the
|
||||
* parser does not enforce the seed-map allow-list.
|
||||
* - `claimValue`: numeric, finite. Empty cell → undefined.
|
||||
* - `claimUnit`: free-form unit string (`USD`, `people`, `pct`, …).
|
||||
* - `claimPeriod`: free-form period string (`monthly`, `annual`, …)
|
||||
* or undefined for non-periodic metrics.
|
||||
*/
|
||||
claimMetric?: string;
|
||||
claimValue?: number;
|
||||
claimUnit?: string;
|
||||
claimPeriod?: string;
|
||||
}
|
||||
|
||||
export interface FactsFenceParseResult {
|
||||
@@ -112,6 +131,20 @@ function parseConfidenceCell(raw: string): number | undefined {
|
||||
return Number.isFinite(n) ? n : undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.35.4 — parse a free-form numeric cell for typed-claim values.
|
||||
* Empty / non-numeric → undefined (caller decides whether to drop or warn).
|
||||
* Tolerates plain numbers and standard scientific notation. Locale-dependent
|
||||
* thousand separators (`,`) are stripped so `50,000` parses to `50000`.
|
||||
*/
|
||||
function parseNumericCell(raw: string): number | undefined {
|
||||
const trimmed = raw.trim();
|
||||
if (!trimmed) return undefined;
|
||||
const stripped = trimmed.replace(/,/g, '');
|
||||
const n = parseFloat(stripped);
|
||||
return Number.isFinite(n) ? n : undefined;
|
||||
}
|
||||
|
||||
function parseSupersededByFromContext(context: string | undefined): number | undefined {
|
||||
if (!context) return undefined;
|
||||
const m = context.match(/superseded by #(\d+)/i);
|
||||
@@ -175,8 +208,10 @@ export function parseFactsFence(body: string): FactsFenceParseResult {
|
||||
// Separator row (just dashes/colons) — skip.
|
||||
if (isSeparatorRow(cells)) continue;
|
||||
|
||||
// Expect 10 cells: row_num, claim, kind, confidence, visibility,
|
||||
// notability, valid_from, valid_until, source, context.
|
||||
// Expect 10 cells (legacy 10-cell fence) OR 14 cells (v0.35.4
|
||||
// typed-claim wide fence): row_num, claim, kind, confidence,
|
||||
// visibility, notability, valid_from, valid_until, source, context,
|
||||
// [claim_metric, claim_value, claim_unit, claim_period].
|
||||
// Tolerate 9 (missing trailing context cell) — markdown editors often
|
||||
// drop empty trailing cells.
|
||||
if (cells.length < 9) {
|
||||
@@ -190,6 +225,10 @@ export function parseFactsFence(body: string): FactsFenceParseResult {
|
||||
validFromRaw, validUntilRaw,
|
||||
sourceRaw,
|
||||
contextRaw = '',
|
||||
claimMetricRaw = '',
|
||||
claimValueRaw = '',
|
||||
claimUnitRaw = '',
|
||||
claimPeriodRaw = '',
|
||||
] = cells;
|
||||
|
||||
const rowNum = parseInt(rowNumStr, 10);
|
||||
@@ -246,6 +285,11 @@ export function parseFactsFence(body: string): FactsFenceParseResult {
|
||||
active: !struck,
|
||||
supersededBy,
|
||||
forgotten: struck ? forgotten : false,
|
||||
// v0.35.4 — typed-claim fields, all optional.
|
||||
claimMetric: parseStringCell(claimMetricRaw),
|
||||
claimValue: parseNumericCell(claimValueRaw),
|
||||
claimUnit: parseStringCell(claimUnitRaw),
|
||||
claimPeriod: parseStringCell(claimPeriodRaw),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -273,11 +317,28 @@ function formatConfidence(c: number): string {
|
||||
* fence state survives unrelated edits to other rows.
|
||||
*/
|
||||
export function renderFactsTable(facts: ParsedFact[]): string {
|
||||
const header = `| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context |`;
|
||||
const separator = `|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|`;
|
||||
// v0.35.4 (D-CDX-5): widen to 14 cells when ANY row has a non-undefined
|
||||
// typed-claim field. Otherwise stay at the 10-cell legacy shape so
|
||||
// existing fences don't get widened on unrelated rewrites (no churn diff
|
||||
// noise).
|
||||
const anyTyped = facts.some(f =>
|
||||
f.claimMetric !== undefined ||
|
||||
f.claimValue !== undefined ||
|
||||
f.claimUnit !== undefined ||
|
||||
f.claimPeriod !== undefined,
|
||||
);
|
||||
const header = anyTyped
|
||||
? `| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | claim_metric | claim_value | claim_unit | claim_period |`
|
||||
: `| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context |`;
|
||||
const separator = anyTyped
|
||||
? `|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|--------------|-------------|------------|--------------|`
|
||||
: `|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|`;
|
||||
const rows = facts.map(f => {
|
||||
const claimCell = f.active ? f.claim : `~~${f.claim}~~`;
|
||||
return `| ${f.rowNum} | ${escapeFenceCell(claimCell)} | ${f.kind} | ${formatConfidence(f.confidence)} | ${f.visibility} | ${f.notability} | ${escapeFenceCell(f.validFrom ?? '')} | ${escapeFenceCell(f.validUntil ?? '')} | ${escapeFenceCell(f.source ?? '')} | ${escapeFenceCell(f.context ?? '')} |`;
|
||||
const base = `| ${f.rowNum} | ${escapeFenceCell(claimCell)} | ${f.kind} | ${formatConfidence(f.confidence)} | ${f.visibility} | ${f.notability} | ${escapeFenceCell(f.validFrom ?? '')} | ${escapeFenceCell(f.validUntil ?? '')} | ${escapeFenceCell(f.source ?? '')} | ${escapeFenceCell(f.context ?? '')} |`;
|
||||
if (!anyTyped) return base;
|
||||
const valueCell = f.claimValue === undefined ? '' : String(f.claimValue);
|
||||
return `${base} ${escapeFenceCell(f.claimMetric ?? '')} | ${escapeFenceCell(valueCell)} | ${escapeFenceCell(f.claimUnit ?? '')} | ${escapeFenceCell(f.claimPeriod ?? '')} |`;
|
||||
});
|
||||
const inner = ['', header, separator, ...rows, ''].join('\n');
|
||||
return `${FACTS_FENCE_BEGIN}${inner}${FACTS_FENCE_END}`;
|
||||
@@ -317,6 +378,13 @@ export function upsertFactRow(
|
||||
source: newRow.source,
|
||||
context: newRow.context,
|
||||
active: newRow.active ?? true,
|
||||
// v0.35.4 — typed-claim pass-through. When undefined the renderer
|
||||
// stays at the 10-cell shape so unrelated edits don't widen the
|
||||
// fence.
|
||||
claimMetric: newRow.claimMetric,
|
||||
claimValue: newRow.claimValue,
|
||||
claimUnit: newRow.claimUnit,
|
||||
claimPeriod: newRow.claimPeriod,
|
||||
},
|
||||
];
|
||||
|
||||
|
||||
@@ -82,6 +82,83 @@ export interface ExtractFromFenceOpts {
|
||||
* the mapper uses real UTC midnight today.
|
||||
*/
|
||||
nowOverride?: Date;
|
||||
/**
|
||||
* v0.35.4 (D-ENG-1 + D-CDX-5) — optional fallback for `valid_from` when
|
||||
* the fence row lacks an explicit `validFrom:`. Threaded by the
|
||||
* `extract-facts` cycle phase from `engine.getPage(slug).effective_date`
|
||||
* so a meeting page dated 2026-04-28 stamps its facts as claimed-on
|
||||
* that date instead of "the import timestamp".
|
||||
*
|
||||
* Precedence chain:
|
||||
* 1. Explicit `validFrom:` in the fence row (today's behavior, preserved).
|
||||
* 2. `pageEffectiveDate` when set.
|
||||
* 3. `undefined` → engine.insertFact defaults to now() at insert time.
|
||||
*
|
||||
* Optional because `facts/fence-write.ts` calls this from a context
|
||||
* with no `Page` object available (Codex F6). Null and undefined are
|
||||
* treated identically: fall through to behavior (3).
|
||||
*/
|
||||
pageEffectiveDate?: Date | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.35.4 (D-ENG-4) — normalized metric vocabulary.
|
||||
*
|
||||
* Seed map for common founder/company metrics. Free-text labels normalize
|
||||
* to lowercase snake_case so trajectory queries don't fragment across
|
||||
* capitalization variants. Unknown labels still pass through (lowercased
|
||||
* + spaces → underscores) so the user can author arbitrary metrics
|
||||
* without a code change. Exported so tests can pin the map.
|
||||
*/
|
||||
export const METRIC_NORMALIZATION_MAP: ReadonlyMap<string, string> = new Map([
|
||||
// Revenue / financial
|
||||
['mrr', 'mrr'],
|
||||
['monthly recurring revenue', 'mrr'],
|
||||
['arr', 'arr'],
|
||||
['annual recurring revenue', 'arr'],
|
||||
['revenue', 'revenue'],
|
||||
['burn', 'burn_rate'],
|
||||
['burn rate', 'burn_rate'],
|
||||
['runway', 'runway'],
|
||||
['cash', 'cash'],
|
||||
['gross margin', 'gross_margin'],
|
||||
// Funding
|
||||
['fundraise', 'fundraise'],
|
||||
['raise', 'fundraise'],
|
||||
// People
|
||||
['headcount', 'headcount'],
|
||||
['team size', 'team_size'],
|
||||
['team', 'team_size'],
|
||||
// Users / engagement
|
||||
['users', 'users'],
|
||||
['mau', 'mau'],
|
||||
['monthly active users', 'mau'],
|
||||
['dau', 'dau'],
|
||||
['daily active users', 'dau'],
|
||||
['churn', 'churn_rate'],
|
||||
['churn rate', 'churn_rate'],
|
||||
// Unit economics
|
||||
['cac', 'cac'],
|
||||
['ltv', 'ltv'],
|
||||
]);
|
||||
|
||||
/**
|
||||
* Normalize a free-text metric label to lowercase snake_case. Known
|
||||
* labels (seed map above) map to canonical names; unknown labels are
|
||||
* lowercased + whitespace-collapsed → underscores. Returns undefined for
|
||||
* empty / whitespace-only input so the caller can treat it as "no
|
||||
* metric set" without an extra null check.
|
||||
*/
|
||||
export function normalizeMetricLabel(raw: string | undefined | null): string | undefined {
|
||||
if (raw === undefined || raw === null) return undefined;
|
||||
const trimmed = raw.trim().toLowerCase();
|
||||
if (!trimmed) return undefined;
|
||||
const seed = METRIC_NORMALIZATION_MAP.get(trimmed);
|
||||
if (seed) return seed;
|
||||
// Collapse runs of whitespace to single underscore, strip non-alphanumeric
|
||||
// edges. Allows users to write "Net Promoter Score" → `net_promoter_score`
|
||||
// without registering it.
|
||||
return trimmed.replace(/\s+/g, '_').replace(/[^a-z0-9_]/g, '');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -91,7 +168,7 @@ export interface ExtractFromFenceOpts {
|
||||
* @param slug The entity page slug (also becomes source_markdown_slug)
|
||||
* @param sourceId The source binding (resolved from sources.local_path
|
||||
* by the caller; multi-source brains thread this through)
|
||||
* @param opts Test-only overrides
|
||||
* @param opts Optional overrides (test nowOverride; v0.35.4 page-date fallback)
|
||||
*/
|
||||
export function extractFactsFromFenceText(
|
||||
facts: ParsedFact[],
|
||||
@@ -100,9 +177,12 @@ export function extractFactsFromFenceText(
|
||||
opts: ExtractFromFenceOpts = {},
|
||||
): FenceExtractedFact[] {
|
||||
const today = opts.nowOverride ?? todayUtcDate();
|
||||
const pageDateFallback = opts.pageEffectiveDate ?? undefined;
|
||||
|
||||
return facts.map(f => {
|
||||
const validFrom = parseValidDate(f.validFrom);
|
||||
// v0.35.4 (D-ENG-1) valid_from precedence: fence > pageEffectiveDate > engine default (now).
|
||||
const fenceDate = parseValidDate(f.validFrom);
|
||||
const validFrom = fenceDate ?? pageDateFallback ?? undefined;
|
||||
|
||||
// valid_until derivation. Three branches:
|
||||
// 1. Explicit validUntil in the fence → honor as-is.
|
||||
@@ -137,6 +217,13 @@ export function extractFactsFromFenceText(
|
||||
confidence: f.confidence,
|
||||
row_num: f.rowNum,
|
||||
source_markdown_slug: slug,
|
||||
// v0.35.4 (D-CDX-5) — typed-claim threading. Metric label normalized
|
||||
// here so the DB-side index hits use the canonical name; value /
|
||||
// unit / period stored verbatim.
|
||||
claim_metric: normalizeMetricLabel(f.claimMetric) ?? null,
|
||||
claim_value: f.claimValue ?? null,
|
||||
claim_unit: f.claimUnit ?? null,
|
||||
claim_period: f.claimPeriod ?? null,
|
||||
};
|
||||
return row;
|
||||
});
|
||||
|
||||
@@ -26,6 +26,7 @@ import type { ChatResult } from '../ai/gateway.ts';
|
||||
import { INJECTION_PATTERNS } from '../think/sanitize.ts';
|
||||
import { resolveModel } from '../model-config.ts';
|
||||
import type { BrainEngine, NewFact, FactKind } from '../engine.ts';
|
||||
import { normalizeMetricLabel } from './extract-from-fence.ts';
|
||||
|
||||
/**
|
||||
* v0.31 (D15): kill-switch for fact extraction.
|
||||
@@ -101,7 +102,9 @@ const EXTRACTOR_SYSTEM = [
|
||||
'Output strictly one JSON object on a single line:',
|
||||
'{"facts":[{"fact":"<terse claim>","kind":"event|preference|commitment|belief|fact",',
|
||||
'"entity":"<canonical slug or display name or null>","confidence":<0..1>,',
|
||||
'"notability":"high|medium|low"}]}.',
|
||||
'"notability":"high|medium|low",',
|
||||
'"metric":"<lowercase snake_case or null>","value":<number or null>,',
|
||||
'"unit":"<USD|people|pct|... or null>","period":"<monthly|annual|quarterly|null>"}]}.',
|
||||
'No prose, no code fences. Empty facts array is valid when nothing claim-worthy was said.',
|
||||
'',
|
||||
'Rules:',
|
||||
@@ -124,6 +127,19 @@ const EXTRACTOR_SYSTEM = [
|
||||
' Can wait for batch processing.',
|
||||
' * "low": Logistical noise, restaurant orders, routine scheduling, "we\'re at X place".',
|
||||
' Skip entirely — not worth storing.',
|
||||
'',
|
||||
'- Typed-claim fields (metric/value/unit/period) — emit ONLY when the claim',
|
||||
' carries a quantitative metric assertion. Examples:',
|
||||
' * "MRR: $50K (Jan 2026)" → metric=mrr, value=50000, unit=USD, period=monthly',
|
||||
' * "ARR: $2M" → metric=arr, value=2000000, unit=USD, period=annual',
|
||||
' * "Team size: 12" → metric=team_size, value=12, unit=people, period=null',
|
||||
' * "Closed Series A: $15M" → metric=fundraise, value=15000000, unit=USD, period=null',
|
||||
' * "User churn: 5%" → metric=churn_rate, value=0.05, unit=pct, period=null',
|
||||
' Use lowercase snake_case for metric. Common labels: mrr, arr, revenue,',
|
||||
' runway, burn_rate, cash, gross_margin, team_size, headcount, users, mau,',
|
||||
' dau, cac, ltv, churn_rate, fundraise. For non-metric claims (preferences,',
|
||||
' events, beliefs), set all four to null. Numeric values: emit the raw',
|
||||
' number after currency/scale normalization (50000 not "$50K"; 0.05 not "5%").',
|
||||
].join('\n');
|
||||
|
||||
const MAX_TURN_TEXT_CHARS = 8000;
|
||||
@@ -207,6 +223,15 @@ export async function extractFactsFromTurn(input: ExtractInput): Promise<Extract
|
||||
embedding = null;
|
||||
}
|
||||
|
||||
// v0.35.4 (D-CDX-2) — typed-claim threading. Normalize the metric label
|
||||
// here so all storage paths see canonical lowercase snake_case names.
|
||||
// Value is already a finite number from parseExtractorJson; unit and
|
||||
// period are stored verbatim.
|
||||
const claimMetric = normalizeMetricLabel(candidate.metric ?? undefined) ?? null;
|
||||
const claimValue = candidate.value ?? null;
|
||||
const claimUnit = candidate.unit ?? null;
|
||||
const claimPeriod = candidate.period ?? null;
|
||||
|
||||
facts.push({
|
||||
fact: factText,
|
||||
kind,
|
||||
@@ -216,6 +241,10 @@ export async function extractFactsFromTurn(input: ExtractInput): Promise<Extract
|
||||
confidence,
|
||||
notability,
|
||||
embedding,
|
||||
claim_metric: claimMetric,
|
||||
claim_value: claimValue,
|
||||
claim_unit: claimUnit,
|
||||
claim_period: claimPeriod,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -228,6 +257,12 @@ interface RawExtracted {
|
||||
entity?: string | null;
|
||||
confidence?: number;
|
||||
notability?: string;
|
||||
// v0.35.4 (D-CDX-2) — typed-claim fields. All optional; emit only for
|
||||
// metric-shaped claims. See EXTRACTOR_SYSTEM rules above.
|
||||
metric?: string | null;
|
||||
value?: number | null;
|
||||
unit?: string | null;
|
||||
period?: string | null;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -266,6 +301,14 @@ function tryArrayShape(s: string): RawExtracted[] | null {
|
||||
entity: typeof o.entity === 'string' ? o.entity : null,
|
||||
confidence: typeof o.confidence === 'number' ? o.confidence : 1.0,
|
||||
notability: typeof o.notability === 'string' ? o.notability : undefined,
|
||||
// v0.35.4 (D-CDX-2) — typed-claim fields. Strict shape: metric/unit/period
|
||||
// must be string-or-null; value must be a finite number-or-null. Anything
|
||||
// else falls through to undefined so the downstream pipeline treats it
|
||||
// as "no metric set" rather than corrupted data.
|
||||
metric: typeof o.metric === 'string' ? o.metric : null,
|
||||
value: (typeof o.value === 'number' && Number.isFinite(o.value)) ? o.value : null,
|
||||
unit: typeof o.unit === 'string' ? o.unit : null,
|
||||
period: typeof o.period === 'string' ? o.period : null,
|
||||
});
|
||||
}
|
||||
return out;
|
||||
|
||||
@@ -3210,6 +3210,37 @@ export const MIGRATIONS: Migration[] = [
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
version: 67,
|
||||
name: 'facts_typed_claim_columns',
|
||||
// v0.35.4 — typed-claim columns for trajectory queries.
|
||||
//
|
||||
// Adds four optional columns to `facts` so metric assertions like
|
||||
// "$50K MRR" can be stored as (claim_metric=mrr, claim_value=50000,
|
||||
// claim_unit=USD, claim_period=monthly) and queried chronologically
|
||||
// by `gbrain eval trajectory` + the `find_trajectory` MCP op.
|
||||
//
|
||||
// All columns nullable: existing fence rows persist identically.
|
||||
// The partial index covers only metric-bearing rows and stays
|
||||
// zero-byte until the v0.35.4 extraction path (`src/core/facts/extract.ts`)
|
||||
// starts emitting typed fields, so this migration is metadata-only
|
||||
// on both engines.
|
||||
//
|
||||
// See plan: ~/.claude/plans/system-instruction-you-are-working-curious-jellyfish.md
|
||||
// Locked decisions D1 (inline extension), D-CDX-7 (v66→v67 renumber).
|
||||
idempotent: true,
|
||||
sql: `
|
||||
ALTER TABLE facts
|
||||
ADD COLUMN IF NOT EXISTS claim_metric TEXT,
|
||||
ADD COLUMN IF NOT EXISTS claim_value DOUBLE PRECISION,
|
||||
ADD COLUMN IF NOT EXISTS claim_unit TEXT,
|
||||
ADD COLUMN IF NOT EXISTS claim_period TEXT;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS facts_typed_claim_idx
|
||||
ON facts (entity_slug, claim_metric, valid_from)
|
||||
WHERE claim_metric IS NOT NULL;
|
||||
`,
|
||||
},
|
||||
];
|
||||
|
||||
export const LATEST_VERSION = MIGRATIONS.length > 0
|
||||
|
||||
@@ -90,6 +90,21 @@ export const FIND_CONTRADICTIONS_DESCRIPTION =
|
||||
"Reads the cached run row — does NOT trigger a new probe; users run " +
|
||||
"`gbrain eval suspected-contradictions` for that.";
|
||||
|
||||
export const FIND_TRAJECTORY_DESCRIPTION =
|
||||
"v0.35.4 — return the chronological claim trajectory for an entity (typed " +
|
||||
"metric values over time, plus auto-detected regressions and narrative drift). " +
|
||||
"Use this when the user asks 'how has Acme's MRR trended', 'show me what " +
|
||||
"alice-example said about runway over time', 'is this founder consistent', " +
|
||||
"'find regressions for fund-a's portfolio', or wants a time-series view of an " +
|
||||
"entity's structured claims. Returns " +
|
||||
"`{points: [{fact_id, valid_from, metric, value, unit, period, text, source_session, source_markdown_slug}], " +
|
||||
"regressions: [{metric, from_value, from_date, to_value, to_date, delta_pct}], " +
|
||||
"drift_score: number|null, schema_version: 1}`. Drift score 0 = stable narrative, " +
|
||||
"1 = every consecutive claim is unrelated; null when fewer than 3 typed points " +
|
||||
"exist. Visibility-filtered for remote callers (world-only); source-scoped by " +
|
||||
"the caller's OAuth source binding. Pair with `gbrain founder scorecard <slug>` " +
|
||||
"for an aggregated rollup of the same data.";
|
||||
|
||||
// ──────────────────────────────────────────────────────────────────────────────
|
||||
// v0.33.3 Cathedral III foundation — code-intelligence ops (MCP-exposed).
|
||||
// Pre-v0.33.3 the callers/callees/def/refs commands were CLI-only — agents
|
||||
|
||||
@@ -31,6 +31,7 @@ import {
|
||||
QUERY_DESCRIPTION,
|
||||
SEARCH_DESCRIPTION,
|
||||
FIND_CONTRADICTIONS_DESCRIPTION,
|
||||
FIND_TRAJECTORY_DESCRIPTION,
|
||||
CODE_CALLERS_DESCRIPTION,
|
||||
CODE_CALLEES_DESCRIPTION,
|
||||
CODE_DEF_DESCRIPTION,
|
||||
@@ -2444,6 +2445,86 @@ const find_contradictions: Operation = {
|
||||
cliHints: { name: 'find-contradictions' },
|
||||
};
|
||||
|
||||
const find_trajectory: Operation = {
|
||||
name: 'find_trajectory',
|
||||
description: FIND_TRAJECTORY_DESCRIPTION,
|
||||
scope: 'read',
|
||||
// localOnly intentionally NOT set — federated OAuth clients should be
|
||||
// able to query trajectories for entities in their scope. Visibility
|
||||
// filtering (D-CDX-1) inside the engine restricts remote callers to
|
||||
// visibility='world' facts.
|
||||
params: {
|
||||
entity_slug: {
|
||||
type: 'string',
|
||||
description: 'Required. Entity slug to chart (e.g. "companies/acme-example", "people/alice-example").',
|
||||
},
|
||||
metric: {
|
||||
type: 'string',
|
||||
description: 'Optional. Filter to a single canonical metric (e.g. "mrr", "arr", "team_size"). When omitted, all metrics return.',
|
||||
},
|
||||
since: {
|
||||
type: 'string',
|
||||
description: 'Optional lower bound on valid_from (YYYY-MM-DD or ISO).',
|
||||
},
|
||||
until: {
|
||||
type: 'string',
|
||||
description: 'Optional upper bound on valid_from (YYYY-MM-DD or ISO).',
|
||||
},
|
||||
limit: {
|
||||
type: 'number',
|
||||
description: 'Max points returned. Default 100, max 500.',
|
||||
},
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
if (typeof p.entity_slug !== 'string' || !p.entity_slug.trim()) {
|
||||
throw new Error('find_trajectory requires entity_slug (string)');
|
||||
}
|
||||
const metric = typeof p.metric === 'string' ? p.metric : undefined;
|
||||
const since = typeof p.since === 'string' ? p.since : undefined;
|
||||
const until = typeof p.until === 'string' ? p.until : undefined;
|
||||
const limit = typeof p.limit === 'number' ? p.limit : undefined;
|
||||
const scope = sourceScopeOpts(ctx);
|
||||
|
||||
// D-CDX-1: thread ctx.remote into the engine so visibility filtering
|
||||
// happens at SQL level. Mirrors recall's posture for untrusted callers.
|
||||
const points = await ctx.engine.findTrajectory({
|
||||
entitySlug: p.entity_slug,
|
||||
...scope,
|
||||
remote: ctx.remote === true,
|
||||
metric,
|
||||
since,
|
||||
until,
|
||||
limit,
|
||||
});
|
||||
|
||||
const { computeTrajectoryStats, TRAJECTORY_SCHEMA_VERSION } = await import('./trajectory.ts');
|
||||
const { regressions, drift_score } = computeTrajectoryStats(points);
|
||||
|
||||
// Engine result includes raw embeddings (Float32Array); strip those
|
||||
// before sending over MCP — they're bulky binary noise that consumers
|
||||
// never need at this layer.
|
||||
const wirePoints = points.map(pt => ({
|
||||
fact_id: pt.fact_id,
|
||||
valid_from: pt.valid_from.toISOString().slice(0, 10),
|
||||
metric: pt.metric,
|
||||
value: pt.value,
|
||||
unit: pt.unit,
|
||||
period: pt.period,
|
||||
text: pt.text,
|
||||
source_session: pt.source_session,
|
||||
source_markdown_slug: pt.source_markdown_slug,
|
||||
}));
|
||||
|
||||
return {
|
||||
points: wirePoints,
|
||||
regressions,
|
||||
drift_score,
|
||||
schema_version: TRAJECTORY_SCHEMA_VERSION,
|
||||
};
|
||||
},
|
||||
cliHints: { name: 'find-trajectory' },
|
||||
};
|
||||
|
||||
const get_recent_transcripts: Operation = {
|
||||
name: 'get_recent_transcripts',
|
||||
description: GET_RECENT_TRANSCRIPTS_DESCRIPTION,
|
||||
@@ -3172,6 +3253,8 @@ export const operations: Operation[] = [
|
||||
find_contradictions,
|
||||
// v0.33: expertise + relationship-proximity routing
|
||||
find_experts,
|
||||
// v0.35.4: temporal trajectory (typed claims over time + regression detection)
|
||||
find_trajectory,
|
||||
// v0.33.3: Cathedral III code-intelligence (MCP-exposed; were CLI_ONLY pre-v0.33.3)
|
||||
code_callers, code_callees, code_def, code_refs,
|
||||
// v0.34 W3: recursive code_blast + code_flow
|
||||
|
||||
+179
-32
@@ -2297,22 +2297,41 @@ export class PGLiteEngine implements BrainEngine {
|
||||
const embedding = input.embedding ?? null;
|
||||
const embeddedAt = embedding ? new Date() : null;
|
||||
const embedStr = embedding ? toPgVectorLiteral(embedding) : null;
|
||||
// v0.35.4 (D-CDX-5) — typed-claim columns. All four nullable.
|
||||
const claimMetric = input.claim_metric ?? null;
|
||||
const claimValue = input.claim_value ?? null;
|
||||
const claimUnit = input.claim_unit ?? null;
|
||||
const claimPeriod = input.claim_period ?? null;
|
||||
|
||||
if (ctx.supersedeId !== undefined) {
|
||||
// Supersede flow: insert new + expire old in one txn so observers never
|
||||
// see both rows active simultaneously.
|
||||
const result = await this.db.transaction(async (tx) => {
|
||||
const ins = await tx.query<{ id: number }>(
|
||||
`INSERT INTO facts (
|
||||
source_id, entity_slug, fact, kind, visibility, notability, context,
|
||||
valid_from, valid_until, source, source_session, confidence,
|
||||
embedding, embedded_at
|
||||
) VALUES (
|
||||
$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12, ${embedStr === null ? 'NULL' : `$13::vector`}, ${embedStr === null ? 'NULL' : '$14'}
|
||||
) RETURNING id`,
|
||||
embedStr === null
|
||||
? [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence]
|
||||
: [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embedStr, embeddedAt],
|
||||
? `INSERT INTO facts (
|
||||
source_id, entity_slug, fact, kind, visibility, notability, context,
|
||||
valid_from, valid_until, source, source_session, confidence,
|
||||
embedding, embedded_at,
|
||||
claim_metric, claim_value, claim_unit, claim_period
|
||||
) VALUES (
|
||||
$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,
|
||||
NULL, NULL,
|
||||
$13, $14, $15, $16
|
||||
) RETURNING id`
|
||||
: `INSERT INTO facts (
|
||||
source_id, entity_slug, fact, kind, visibility, notability, context,
|
||||
valid_from, valid_until, source, source_session, confidence,
|
||||
embedding, embedded_at,
|
||||
claim_metric, claim_value, claim_unit, claim_period
|
||||
) VALUES (
|
||||
$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,
|
||||
$13::vector, $14,
|
||||
$15, $16, $17, $18
|
||||
) RETURNING id`,
|
||||
embedStr === null
|
||||
? [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, claimMetric, claimValue, claimUnit, claimPeriod]
|
||||
: [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embedStr, embeddedAt, claimMetric, claimValue, claimUnit, claimPeriod],
|
||||
);
|
||||
const newId = ins.rows[0].id;
|
||||
await tx.query(
|
||||
@@ -2326,16 +2345,30 @@ export class PGLiteEngine implements BrainEngine {
|
||||
}
|
||||
|
||||
const ins = await this.db.query<{ id: number }>(
|
||||
`INSERT INTO facts (
|
||||
source_id, entity_slug, fact, kind, visibility, notability, context,
|
||||
valid_from, valid_until, source, source_session, confidence,
|
||||
embedding, embedded_at
|
||||
) VALUES (
|
||||
$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12, ${embedStr === null ? 'NULL' : `$13::vector`}, ${embedStr === null ? 'NULL' : '$14'}
|
||||
) RETURNING id`,
|
||||
embedStr === null
|
||||
? [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence]
|
||||
: [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embedStr, embeddedAt],
|
||||
? `INSERT INTO facts (
|
||||
source_id, entity_slug, fact, kind, visibility, notability, context,
|
||||
valid_from, valid_until, source, source_session, confidence,
|
||||
embedding, embedded_at,
|
||||
claim_metric, claim_value, claim_unit, claim_period
|
||||
) VALUES (
|
||||
$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,
|
||||
NULL, NULL,
|
||||
$13, $14, $15, $16
|
||||
) RETURNING id`
|
||||
: `INSERT INTO facts (
|
||||
source_id, entity_slug, fact, kind, visibility, notability, context,
|
||||
valid_from, valid_until, source, source_session, confidence,
|
||||
embedding, embedded_at,
|
||||
claim_metric, claim_value, claim_unit, claim_period
|
||||
) VALUES (
|
||||
$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,
|
||||
$13::vector, $14,
|
||||
$15, $16, $17, $18
|
||||
) RETURNING id`,
|
||||
embedStr === null
|
||||
? [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, claimMetric, claimValue, claimUnit, claimPeriod]
|
||||
: [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embedStr, embeddedAt, claimMetric, claimValue, claimUnit, claimPeriod],
|
||||
);
|
||||
return { id: ins.rows[0].id, status: 'inserted' };
|
||||
}
|
||||
@@ -2376,23 +2409,45 @@ export class PGLiteEngine implements BrainEngine {
|
||||
const embedding = input.embedding ?? null;
|
||||
const embeddedAt = embedding ? new Date() : null;
|
||||
const embedStr = embedding ? toPgVectorLiteral(embedding) : null;
|
||||
// v0.35.4 (D-CDX-5) — typed-claim columns. All four nullable.
|
||||
const claimMetric = input.claim_metric ?? null;
|
||||
const claimValue = input.claim_value ?? null;
|
||||
const claimUnit = input.claim_unit ?? null;
|
||||
const claimPeriod = input.claim_period ?? null;
|
||||
|
||||
// Param-positional dispatch: embedStr presence shifts the trailing
|
||||
// slots by one. Order of named slots stays stable across both
|
||||
// branches: embedded_at, row_num, source_markdown_slug,
|
||||
// claim_metric, claim_value, claim_unit, claim_period.
|
||||
const ins = await tx.query<{ id: number }>(
|
||||
`INSERT INTO facts (
|
||||
source_id, entity_slug, fact, kind, visibility, notability, context,
|
||||
valid_from, valid_until, source, source_session, confidence,
|
||||
embedding, embedded_at,
|
||||
row_num, source_markdown_slug
|
||||
) VALUES (
|
||||
$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,
|
||||
${embedStr === null ? 'NULL' : `$13::vector`},
|
||||
${embedStr === null ? '$13' : '$14'},
|
||||
${embedStr === null ? '$14' : '$15'},
|
||||
${embedStr === null ? '$15' : '$16'}
|
||||
) RETURNING id`,
|
||||
embedStr === null
|
||||
? [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embeddedAt, input.row_num, input.source_markdown_slug]
|
||||
: [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embedStr, embeddedAt, input.row_num, input.source_markdown_slug],
|
||||
? `INSERT INTO facts (
|
||||
source_id, entity_slug, fact, kind, visibility, notability, context,
|
||||
valid_from, valid_until, source, source_session, confidence,
|
||||
embedding, embedded_at,
|
||||
row_num, source_markdown_slug,
|
||||
claim_metric, claim_value, claim_unit, claim_period
|
||||
) VALUES (
|
||||
$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,
|
||||
NULL, $13,
|
||||
$14, $15,
|
||||
$16, $17, $18, $19
|
||||
) RETURNING id`
|
||||
: `INSERT INTO facts (
|
||||
source_id, entity_slug, fact, kind, visibility, notability, context,
|
||||
valid_from, valid_until, source, source_session, confidence,
|
||||
embedding, embedded_at,
|
||||
row_num, source_markdown_slug,
|
||||
claim_metric, claim_value, claim_unit, claim_period
|
||||
) VALUES (
|
||||
$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,
|
||||
$13::vector, $14,
|
||||
$15, $16,
|
||||
$17, $18, $19, $20
|
||||
) RETURNING id`,
|
||||
embedStr === null
|
||||
? [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embeddedAt, input.row_num, input.source_markdown_slug, claimMetric, claimValue, claimUnit, claimPeriod]
|
||||
: [ctx.source_id, entitySlug, input.fact, kind, visibility, notability, context, validFrom, validUntil, input.source, sourceSession, confidence, embedStr, embeddedAt, input.row_num, input.source_markdown_slug, claimMetric, claimValue, claimUnit, claimPeriod],
|
||||
);
|
||||
out.push(ins.rows[0].id);
|
||||
}
|
||||
@@ -2517,6 +2572,98 @@ export class PGLiteEngine implements BrainEngine {
|
||||
return result.rows.map(rowToFact);
|
||||
}
|
||||
|
||||
async findTrajectory(opts: import('./engine.ts').TrajectoryOpts): Promise<import('./engine.ts').TrajectoryPoint[]> {
|
||||
const limit = clampSearchLimit(opts.limit, 100, 500);
|
||||
const sinceDate = opts.since ? new Date(opts.since) : null;
|
||||
const untilDate = opts.until ? new Date(opts.until) : null;
|
||||
const metric = opts.metric ?? null;
|
||||
const useArray = Array.isArray(opts.sourceIds) && opts.sourceIds.length > 0;
|
||||
const sourceIds = useArray ? opts.sourceIds! : null;
|
||||
const sourceId = opts.sourceId ?? 'default';
|
||||
const remoteFilter = opts.remote === true;
|
||||
|
||||
// Build SQL dynamically. PGLite uses $N positional params; we
|
||||
// assemble the WHERE clauses + params array in tandem to keep them
|
||||
// aligned. Final shape is single SELECT, ORDER BY (valid_from, id) ASC.
|
||||
const where: string[] = [
|
||||
useArray ? `source_id = ANY($1::text[])` : `source_id = $1`,
|
||||
`entity_slug = $2`,
|
||||
`expired_at IS NULL`,
|
||||
];
|
||||
const params: unknown[] = [useArray ? sourceIds : sourceId, opts.entitySlug];
|
||||
let p = 3;
|
||||
if (remoteFilter) {
|
||||
where.push(`visibility = 'world'`);
|
||||
}
|
||||
if (metric !== null) {
|
||||
where.push(`claim_metric = $${p}`);
|
||||
params.push(metric);
|
||||
p += 1;
|
||||
}
|
||||
if (sinceDate) {
|
||||
where.push(`valid_from >= $${p}`);
|
||||
params.push(sinceDate);
|
||||
p += 1;
|
||||
}
|
||||
if (untilDate) {
|
||||
where.push(`valid_from <= $${p}`);
|
||||
params.push(untilDate);
|
||||
p += 1;
|
||||
}
|
||||
params.push(limit);
|
||||
const limitPlaceholder = p;
|
||||
|
||||
const sqlText = `
|
||||
SELECT id, valid_from,
|
||||
claim_metric, claim_value, claim_unit, claim_period,
|
||||
fact, source_session, source_markdown_slug,
|
||||
embedding
|
||||
FROM facts
|
||||
WHERE ${where.join(' AND ')}
|
||||
ORDER BY valid_from ASC, id ASC
|
||||
LIMIT $${limitPlaceholder}
|
||||
`;
|
||||
const result = await this.db.query<{
|
||||
id: number;
|
||||
valid_from: Date | string;
|
||||
claim_metric: string | null;
|
||||
claim_value: number | null;
|
||||
claim_unit: string | null;
|
||||
claim_period: string | null;
|
||||
fact: string;
|
||||
source_session: string | null;
|
||||
source_markdown_slug: string | null;
|
||||
embedding: string | number[] | Float32Array | null;
|
||||
}>(sqlText, params);
|
||||
|
||||
return result.rows.map(r => {
|
||||
// Inline embedding parser — mirrors rowToFact() at line 3911.
|
||||
let embedding: Float32Array | null = null;
|
||||
if (r.embedding != null) {
|
||||
if (r.embedding instanceof Float32Array) embedding = r.embedding;
|
||||
else if (Array.isArray(r.embedding)) embedding = new Float32Array(r.embedding);
|
||||
else if (typeof r.embedding === 'string') {
|
||||
const trimmed = r.embedding.trim();
|
||||
const inner = trimmed.startsWith('[') ? trimmed.slice(1, -1) : trimmed;
|
||||
const parts = inner.split(',').map(s => parseFloat(s.trim())).filter(Number.isFinite);
|
||||
embedding = parts.length > 0 ? new Float32Array(parts) : null;
|
||||
}
|
||||
}
|
||||
return {
|
||||
fact_id: Number(r.id),
|
||||
valid_from: r.valid_from instanceof Date ? r.valid_from : new Date(r.valid_from),
|
||||
metric: r.claim_metric,
|
||||
value: r.claim_value === null ? null : Number(r.claim_value),
|
||||
unit: r.claim_unit,
|
||||
period: r.claim_period,
|
||||
text: r.fact,
|
||||
source_session: r.source_session,
|
||||
source_markdown_slug: r.source_markdown_slug,
|
||||
embedding,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
async consolidateFact(id: number, takeId: number): Promise<void> {
|
||||
await this.db.query(
|
||||
`UPDATE facts SET consolidated_at = now(), consolidated_into = $1 WHERE id = $2`,
|
||||
|
||||
@@ -2342,6 +2342,11 @@ export class PostgresEngine implements BrainEngine {
|
||||
const embedding = input.embedding ?? null;
|
||||
const embeddedAt = embedding ? new Date() : null;
|
||||
const embedLit = embedding ? toPgVectorLiteral(embedding) : null;
|
||||
// v0.35.4 (D-CDX-5) — typed-claim columns. All four nullable.
|
||||
const claimMetric = input.claim_metric ?? null;
|
||||
const claimValue = input.claim_value ?? null;
|
||||
const claimUnit = input.claim_unit ?? null;
|
||||
const claimPeriod = input.claim_period ?? null;
|
||||
|
||||
if (ctx.supersedeId !== undefined) {
|
||||
// Per-entity advisory lock + atomic insert + supersede in one txn.
|
||||
@@ -2354,11 +2359,13 @@ export class PostgresEngine implements BrainEngine {
|
||||
INSERT INTO facts (
|
||||
source_id, entity_slug, fact, kind, visibility, notability, context,
|
||||
valid_from, valid_until, source, source_session, confidence,
|
||||
embedding, embedded_at
|
||||
embedding, embedded_at,
|
||||
claim_metric, claim_value, claim_unit, claim_period
|
||||
) VALUES (
|
||||
${ctx.source_id}, ${entitySlug}, ${input.fact}, ${kind}, ${visibility}, ${notability}, ${context},
|
||||
${validFrom}, ${validUntil}, ${input.source}, ${sourceSession}, ${confidence},
|
||||
${embedLit === null ? null : tx.unsafe(`'${embedLit}'::vector`)}, ${embeddedAt}
|
||||
${embedLit === null ? null : tx.unsafe(`'${embedLit}'::vector`)}, ${embeddedAt},
|
||||
${claimMetric}, ${claimValue}, ${claimUnit}, ${claimPeriod}
|
||||
) RETURNING id
|
||||
`;
|
||||
const id = Number(ins[0].id);
|
||||
@@ -2378,11 +2385,13 @@ export class PostgresEngine implements BrainEngine {
|
||||
INSERT INTO facts (
|
||||
source_id, entity_slug, fact, kind, visibility, notability, context,
|
||||
valid_from, valid_until, source, source_session, confidence,
|
||||
embedding, embedded_at
|
||||
embedding, embedded_at,
|
||||
claim_metric, claim_value, claim_unit, claim_period
|
||||
) VALUES (
|
||||
${ctx.source_id}, ${entitySlug}, ${input.fact}, ${kind}, ${visibility}, ${notability}, ${context},
|
||||
${validFrom}, ${validUntil}, ${input.source}, ${sourceSession}, ${confidence},
|
||||
${embedLit === null ? null : tx.unsafe(`'${embedLit}'::vector`)}, ${embeddedAt}
|
||||
${embedLit === null ? null : tx.unsafe(`'${embedLit}'::vector`)}, ${embeddedAt},
|
||||
${claimMetric}, ${claimValue}, ${claimUnit}, ${claimPeriod}
|
||||
) RETURNING id
|
||||
`;
|
||||
return Number(ins[0].id);
|
||||
@@ -2429,18 +2438,25 @@ export class PostgresEngine implements BrainEngine {
|
||||
const embedding = input.embedding ?? null;
|
||||
const embeddedAt = embedding ? new Date() : null;
|
||||
const embedLit = embedding ? toPgVectorLiteral(embedding) : null;
|
||||
// v0.35.4 (D-CDX-5) — typed-claim columns. All four nullable.
|
||||
const claimMetric = input.claim_metric ?? null;
|
||||
const claimValue = input.claim_value ?? null;
|
||||
const claimUnit = input.claim_unit ?? null;
|
||||
const claimPeriod = input.claim_period ?? null;
|
||||
|
||||
const ins = await tx<Array<{ id: number }>>`
|
||||
INSERT INTO facts (
|
||||
source_id, entity_slug, fact, kind, visibility, notability, context,
|
||||
valid_from, valid_until, source, source_session, confidence,
|
||||
embedding, embedded_at,
|
||||
row_num, source_markdown_slug
|
||||
row_num, source_markdown_slug,
|
||||
claim_metric, claim_value, claim_unit, claim_period
|
||||
) VALUES (
|
||||
${ctx.source_id}, ${entitySlug}, ${input.fact}, ${kind}, ${visibility}, ${notability}, ${context},
|
||||
${validFrom}, ${validUntil}, ${input.source}, ${sourceSession}, ${confidence},
|
||||
${embedLit === null ? null : tx.unsafe(`'${embedLit}'::vector`)}, ${embeddedAt},
|
||||
${input.row_num}, ${input.source_markdown_slug}
|
||||
${input.row_num}, ${input.source_markdown_slug},
|
||||
${claimMetric}, ${claimValue}, ${claimUnit}, ${claimPeriod}
|
||||
) RETURNING id
|
||||
`;
|
||||
out.push(Number(ins[0].id));
|
||||
@@ -2599,6 +2615,62 @@ export class PostgresEngine implements BrainEngine {
|
||||
await sql`UPDATE facts SET consolidated_at = now(), consolidated_into = ${takeId} WHERE id = ${id}`;
|
||||
}
|
||||
|
||||
async findTrajectory(opts: import('./engine.ts').TrajectoryOpts): Promise<import('./engine.ts').TrajectoryPoint[]> {
|
||||
const sql = this.sql;
|
||||
const limit = clampSearchLimit(opts.limit, 100, 500);
|
||||
const sinceDate = opts.since ? new Date(opts.since) : null;
|
||||
const untilDate = opts.until ? new Date(opts.until) : null;
|
||||
const metric = opts.metric ?? null;
|
||||
const useArray = Array.isArray(opts.sourceIds) && opts.sourceIds.length > 0;
|
||||
const sourceIds = useArray ? opts.sourceIds! : null;
|
||||
const sourceId = opts.sourceId ?? 'default';
|
||||
const remoteFilter = opts.remote === true;
|
||||
|
||||
// Source-scope predicate: array path (federated) wins over scalar.
|
||||
// Engine.ts contract: returns chronological points; regressions +
|
||||
// drift_score are computed by the caller (src/core/trajectory.ts).
|
||||
const rows = await sql<Array<{
|
||||
id: number;
|
||||
valid_from: Date;
|
||||
claim_metric: string | null;
|
||||
claim_value: number | null;
|
||||
claim_unit: string | null;
|
||||
claim_period: string | null;
|
||||
fact: string;
|
||||
source_session: string | null;
|
||||
source_markdown_slug: string | null;
|
||||
embedding: string | null;
|
||||
}>>`
|
||||
SELECT id, valid_from,
|
||||
claim_metric, claim_value, claim_unit, claim_period,
|
||||
fact, source_session, source_markdown_slug,
|
||||
embedding::text AS embedding
|
||||
FROM facts
|
||||
WHERE ${useArray ? sql`source_id = ANY(${sourceIds}::text[])` : sql`source_id = ${sourceId}`}
|
||||
AND entity_slug = ${opts.entitySlug}
|
||||
AND expired_at IS NULL
|
||||
${remoteFilter ? sql`AND visibility = 'world'` : sql``}
|
||||
${metric !== null ? sql`AND claim_metric = ${metric}` : sql``}
|
||||
${sinceDate ? sql`AND valid_from >= ${sinceDate}` : sql``}
|
||||
${untilDate ? sql`AND valid_from <= ${untilDate}` : sql``}
|
||||
ORDER BY valid_from ASC, id ASC
|
||||
LIMIT ${limit}
|
||||
`;
|
||||
|
||||
return rows.map(r => ({
|
||||
fact_id: Number(r.id),
|
||||
valid_from: r.valid_from,
|
||||
metric: r.claim_metric,
|
||||
value: r.claim_value === null ? null : Number(r.claim_value),
|
||||
unit: r.claim_unit,
|
||||
period: r.claim_period,
|
||||
text: r.fact,
|
||||
source_session: r.source_session,
|
||||
source_markdown_slug: r.source_markdown_slug,
|
||||
embedding: tryParseEmbedding(r.embedding),
|
||||
}));
|
||||
}
|
||||
|
||||
async getFactsHealth(source_id: string): Promise<FactsHealth> {
|
||||
const sql = this.sql;
|
||||
const totals = await sql<Array<{
|
||||
|
||||
@@ -0,0 +1,170 @@
|
||||
/**
|
||||
* v0.35.4 — trajectory derived metrics.
|
||||
*
|
||||
* Pure functions over `TrajectoryPoint[]` (engine output). Both the
|
||||
* `find_trajectory` MCP op and the `gbrain eval trajectory` CLI consume
|
||||
* this module so the regression + drift_score definitions stay in one
|
||||
* place.
|
||||
*
|
||||
* Locked specs from the plan:
|
||||
*
|
||||
* - Regression detection (D-ENG-2): a regression fires for every
|
||||
* consecutive (metric, value) pair where the newer value is at least
|
||||
* 10% lower than the prior value. Threshold is configurable via the
|
||||
* env var `GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD` (default 0.10).
|
||||
* Only points with `claim_value !== null` participate.
|
||||
*
|
||||
* - Drift score (D-ENG-3): `1 - mean(cosine(emb[i], emb[i-1]))` over
|
||||
* points with non-null embeddings. Clamped to [0, 1]. Returns null
|
||||
* when fewer than 3 points have embeddings (graceful degradation
|
||||
* for pre-v0.35.4 facts that arrived without one).
|
||||
*/
|
||||
|
||||
import type { TrajectoryPoint } from './engine.ts';
|
||||
|
||||
/** Default regression threshold (10% drop). Locked decision D-ENG-2. */
|
||||
export const DEFAULT_REGRESSION_THRESHOLD = 0.10;
|
||||
|
||||
/** Schema version for the trajectory + scorecard JSON contract. Additive-only across releases. */
|
||||
export const TRAJECTORY_SCHEMA_VERSION = 1;
|
||||
|
||||
export interface TrajectoryRegression {
|
||||
metric: string;
|
||||
from_value: number;
|
||||
from_date: string; // YYYY-MM-DD
|
||||
to_value: number;
|
||||
to_date: string;
|
||||
delta_pct: number; // negative for a drop; range typically [-1, 0)
|
||||
}
|
||||
|
||||
export interface TrajectoryStats {
|
||||
regressions: TrajectoryRegression[];
|
||||
drift_score: number | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the regression threshold from `GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD`
|
||||
* with fallback to the locked default. Invalid input falls back silently —
|
||||
* the threshold is a soft tuning knob, not a correctness gate.
|
||||
*/
|
||||
export function resolveRegressionThreshold(): number {
|
||||
const raw = process.env.GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD;
|
||||
if (!raw) return DEFAULT_REGRESSION_THRESHOLD;
|
||||
const n = parseFloat(raw);
|
||||
if (!Number.isFinite(n) || n <= 0 || n >= 1) return DEFAULT_REGRESSION_THRESHOLD;
|
||||
return n;
|
||||
}
|
||||
|
||||
function toISODate(d: Date): string {
|
||||
return d.toISOString().slice(0, 10);
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute cosine similarity between two equal-length vectors. Returns 0
|
||||
* when either vector has length zero (defensive — never throws).
|
||||
*/
|
||||
function cosineSim(a: Float32Array, b: Float32Array): number {
|
||||
if (a.length === 0 || b.length === 0 || a.length !== b.length) return 0;
|
||||
let dot = 0;
|
||||
let na = 0;
|
||||
let nb = 0;
|
||||
for (let i = 0; i < a.length; i++) {
|
||||
dot += a[i] * b[i];
|
||||
na += a[i] * a[i];
|
||||
nb += b[i] * b[i];
|
||||
}
|
||||
if (na === 0 || nb === 0) return 0;
|
||||
return dot / (Math.sqrt(na) * Math.sqrt(nb));
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect chronological regressions in a sorted trajectory.
|
||||
*
|
||||
* Iterates per-metric (so trajectories that interleave mrr + arr + team_size
|
||||
* don't trip false regressions across metric boundaries). Within each metric,
|
||||
* walks consecutive value pairs; a pair fires when
|
||||
* `(newer - older) / older <= -threshold`.
|
||||
*
|
||||
* Pre-condition: caller passed points sorted by (valid_from ASC, fact_id ASC).
|
||||
* The engine's `findTrajectory` enforces this. No re-sort here.
|
||||
*/
|
||||
export function detectRegressions(
|
||||
points: TrajectoryPoint[],
|
||||
threshold: number = DEFAULT_REGRESSION_THRESHOLD,
|
||||
): TrajectoryRegression[] {
|
||||
const out: TrajectoryRegression[] = [];
|
||||
// Group by metric so each metric's regression detection is independent.
|
||||
const byMetric = new Map<string, TrajectoryPoint[]>();
|
||||
for (const p of points) {
|
||||
if (p.metric === null || p.value === null) continue;
|
||||
if (!Number.isFinite(p.value)) continue;
|
||||
if (!byMetric.has(p.metric)) byMetric.set(p.metric, []);
|
||||
byMetric.get(p.metric)!.push(p);
|
||||
}
|
||||
|
||||
for (const [metric, series] of byMetric) {
|
||||
for (let i = 1; i < series.length; i++) {
|
||||
const older = series[i - 1];
|
||||
const newer = series[i];
|
||||
const oldVal = older.value!;
|
||||
const newVal = newer.value!;
|
||||
// Guard against division-by-zero: a metric starting at exactly 0
|
||||
// can't compute a relative delta. Skip.
|
||||
if (oldVal === 0) continue;
|
||||
const delta = (newVal - oldVal) / oldVal;
|
||||
if (delta <= -threshold) {
|
||||
out.push({
|
||||
metric,
|
||||
from_value: oldVal,
|
||||
from_date: toISODate(older.valid_from),
|
||||
to_value: newVal,
|
||||
to_date: toISODate(newer.valid_from),
|
||||
delta_pct: delta,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute drift score over the trajectory's existing embeddings.
|
||||
*
|
||||
* `1 - mean(cosine(emb[i], emb[i-1]))` clamped to [0, 1]. Range
|
||||
* interpretation: 0 = narrative stable text-wise; 1 = every consecutive
|
||||
* claim is unrelated to the prior.
|
||||
*
|
||||
* Returns null when fewer than 3 points have non-null embeddings — the
|
||||
* statistic is meaningless on tiny samples.
|
||||
*/
|
||||
export function computeDriftScore(points: TrajectoryPoint[]): number | null {
|
||||
const withEmb = points.filter(p => p.embedding !== null && p.embedding.length > 0);
|
||||
if (withEmb.length < 3) return null;
|
||||
let sumCos = 0;
|
||||
let pairs = 0;
|
||||
for (let i = 1; i < withEmb.length; i++) {
|
||||
sumCos += cosineSim(withEmb[i - 1].embedding!, withEmb[i].embedding!);
|
||||
pairs += 1;
|
||||
}
|
||||
if (pairs === 0) return null;
|
||||
const meanCos = sumCos / pairs;
|
||||
const drift = 1 - meanCos;
|
||||
if (drift < 0) return 0;
|
||||
if (drift > 1) return 1;
|
||||
return drift;
|
||||
}
|
||||
|
||||
/**
|
||||
* Compose the two derived metrics into a single TrajectoryStats. The MCP
|
||||
* op + CLI both call this so the JSON shape stays consistent.
|
||||
*/
|
||||
export function computeTrajectoryStats(
|
||||
points: TrajectoryPoint[],
|
||||
opts: { threshold?: number } = {},
|
||||
): TrajectoryStats {
|
||||
const threshold = opts.threshold ?? resolveRegressionThreshold();
|
||||
return {
|
||||
regressions: detectRegressions(points, threshold),
|
||||
drift_score: computeDriftScore(points),
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,226 @@
|
||||
/**
|
||||
* v0.35.4 (D-CDX-4) — consolidate semantic upsert + chronological
|
||||
* valid_until writeback.
|
||||
*
|
||||
* Pins:
|
||||
* - R4a: a cluster of 3 chronologically-ordered facts produces
|
||||
* 2 facts with valid_until set (older) and 1 with NULL (newest).
|
||||
* - R4b/R7: running consolidate twice on the same input produces zero
|
||||
* NEW takes (semantic upsert by (page_id, claim, since_date)).
|
||||
* This is the Codex F4 fix — without it, the second cycle's
|
||||
* extract_facts would clear consolidated_at and the second
|
||||
* consolidate would append duplicate takes via MAX(row_num)+1.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { runPhaseConsolidate } from '../src/core/cycle/phases/consolidate.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await engine.executeRaw(`DELETE FROM facts`);
|
||||
await engine.executeRaw(`DELETE FROM takes`);
|
||||
await engine.executeRaw(`DELETE FROM pages WHERE slug LIKE 'cdx4-%'`);
|
||||
});
|
||||
|
||||
function unitVec(): string {
|
||||
const a = new Float32Array(1536);
|
||||
a[0] = 1.0;
|
||||
return '[' + Array.from(a).join(',') + ']';
|
||||
}
|
||||
|
||||
async function seedPage(slug: string): Promise<number> {
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO pages (slug, type, title) VALUES ($1, 'company', 'Test') ON CONFLICT DO NOTHING`,
|
||||
[slug],
|
||||
);
|
||||
const r = await engine.executeRaw<{ id: number }>(
|
||||
`SELECT id FROM pages WHERE slug = $1 AND source_id = 'default'`,
|
||||
[slug],
|
||||
);
|
||||
return r[0].id;
|
||||
}
|
||||
|
||||
async function insertFact(args: {
|
||||
entity_slug: string;
|
||||
text: string;
|
||||
valid_from: Date;
|
||||
confidence?: number;
|
||||
}): Promise<number> {
|
||||
const r = await engine.executeRaw<{ id: number }>(
|
||||
`INSERT INTO facts (source_id, entity_slug, fact, kind, source, valid_from, confidence, embedding, embedded_at)
|
||||
VALUES ('default', $1, $2, 'fact', 'test', $3::timestamptz, $4, $5::vector, $3::timestamptz)
|
||||
RETURNING id`,
|
||||
[args.entity_slug, args.text, args.valid_from.toISOString(), args.confidence ?? 0.9, unitVec()],
|
||||
);
|
||||
return r[0].id;
|
||||
}
|
||||
|
||||
describe('R4a — chronological valid_until writeback', () => {
|
||||
test('cluster of 3 chronologically-ordered facts: 2 older get valid_until set, newest stays NULL', async () => {
|
||||
await seedPage('cdx4-acme-mrr');
|
||||
const olderDay = new Date('2026-01-15T00:00:00Z');
|
||||
const midDay = new Date('2026-04-12T00:00:00Z');
|
||||
const newest = new Date('2026-07-08T00:00:00Z');
|
||||
|
||||
// All three close enough in vector space to cluster together (identical
|
||||
// embeddings via unitVec()). Past the 24h "oldest age" gate.
|
||||
const idOlder = await insertFact({
|
||||
entity_slug: 'cdx4-acme-mrr',
|
||||
text: 'MRR claim',
|
||||
valid_from: olderDay,
|
||||
});
|
||||
const idMid = await insertFact({
|
||||
entity_slug: 'cdx4-acme-mrr',
|
||||
text: 'MRR claim',
|
||||
valid_from: midDay,
|
||||
});
|
||||
const idNewest = await insertFact({
|
||||
entity_slug: 'cdx4-acme-mrr',
|
||||
text: 'MRR claim',
|
||||
valid_from: newest,
|
||||
});
|
||||
|
||||
const r = await runPhaseConsolidate(engine, {});
|
||||
expect(r.details.facts_consolidated).toBe(3);
|
||||
expect(r.details.takes_written).toBe(1);
|
||||
|
||||
const rows = await engine.executeRaw<{ id: number; valid_until: Date | null }>(
|
||||
`SELECT id, valid_until FROM facts WHERE entity_slug = 'cdx4-acme-mrr' ORDER BY valid_from ASC`,
|
||||
);
|
||||
expect(rows.length).toBe(3);
|
||||
// Older fact's valid_until = mid.valid_from.
|
||||
expect(rows[0].id).toBe(idOlder);
|
||||
expect(rows[0].valid_until).not.toBeNull();
|
||||
expect(new Date(rows[0].valid_until!).toISOString().slice(0, 10)).toBe('2026-04-12');
|
||||
// Mid fact's valid_until = newest.valid_from.
|
||||
expect(rows[1].id).toBe(idMid);
|
||||
expect(rows[1].valid_until).not.toBeNull();
|
||||
expect(new Date(rows[1].valid_until!).toISOString().slice(0, 10)).toBe('2026-07-08');
|
||||
// Newest fact's valid_until stays NULL.
|
||||
expect(rows[2].id).toBe(idNewest);
|
||||
expect(rows[2].valid_until).toBeNull();
|
||||
});
|
||||
|
||||
test('same-day cluster (3 facts, identical valid_from): id tiebreaker establishes chronological order', async () => {
|
||||
await seedPage('cdx4-acme-sameday');
|
||||
const sameDay = new Date(Date.now() - 30 * 60 * 60 * 1000);
|
||||
const idA = await insertFact({ entity_slug: 'cdx4-acme-sameday', text: 'same day', valid_from: sameDay });
|
||||
const idB = await insertFact({ entity_slug: 'cdx4-acme-sameday', text: 'same day', valid_from: sameDay });
|
||||
const idC = await insertFact({ entity_slug: 'cdx4-acme-sameday', text: 'same day', valid_from: sameDay });
|
||||
|
||||
await runPhaseConsolidate(engine, {});
|
||||
|
||||
// All three valid_from values are equal; the (id ASC) tiebreaker
|
||||
// makes the lowest-id row the "oldest" chronologically. Pin that
|
||||
// contract since the trajectory CLI depends on this ordering.
|
||||
const rows = await engine.executeRaw<{ id: number; valid_until: Date | null }>(
|
||||
`SELECT id, valid_until FROM facts WHERE entity_slug = 'cdx4-acme-sameday' ORDER BY id ASC`,
|
||||
);
|
||||
expect(rows.length).toBe(3);
|
||||
expect(rows[0].id).toBe(idA);
|
||||
expect(rows[1].id).toBe(idB);
|
||||
expect(rows[2].id).toBe(idC);
|
||||
// First two are "older" by tiebreaker → both get valid_until set
|
||||
// (= sameDay, since the next-newer fact has the same valid_from).
|
||||
expect(rows[0].valid_until).not.toBeNull();
|
||||
expect(rows[1].valid_until).not.toBeNull();
|
||||
// Newest by tiebreaker stays NULL.
|
||||
expect(rows[2].valid_until).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('R4b / R7 — cycle idempotency: re-run consolidate produces zero new takes (Codex F4 fix)', () => {
|
||||
test('semantic upsert: second consolidate on identical state produces zero NEW takes', async () => {
|
||||
await seedPage('cdx4-idempo-1');
|
||||
const oldDate = new Date(Date.now() - 30 * 60 * 60 * 1000);
|
||||
for (let i = 0; i < 4; i++) {
|
||||
await insertFact({
|
||||
entity_slug: 'cdx4-idempo-1',
|
||||
text: 'stable claim',
|
||||
valid_from: new Date(oldDate.getTime() + i * 60 * 60 * 1000),
|
||||
});
|
||||
}
|
||||
|
||||
// First run: 1 take, 4 facts consolidated.
|
||||
const r1 = await runPhaseConsolidate(engine, {});
|
||||
expect(r1.details.takes_written).toBe(1);
|
||||
const countAfter1 = await engine.executeRaw<{ n: string }>(
|
||||
`SELECT COUNT(*)::text AS n FROM takes WHERE page_id = (SELECT id FROM pages WHERE slug = 'cdx4-idempo-1')`,
|
||||
);
|
||||
expect(parseInt(countAfter1[0].n, 10)).toBe(1);
|
||||
|
||||
// Simulate the Codex F4 scenario: clear consolidated_at on every fact
|
||||
// (extract_facts cycle phase wipes facts via delete-then-insert, which
|
||||
// is functionally identical to NULL-ing consolidated_at). DO NOT touch
|
||||
// valid_until — the prior consolidate wrote it; the semantic upsert
|
||||
// should still find the take.
|
||||
await engine.executeRaw(
|
||||
`UPDATE facts SET consolidated_at = NULL, consolidated_into = NULL
|
||||
WHERE entity_slug = 'cdx4-idempo-1'`,
|
||||
);
|
||||
|
||||
// Second run: must NOT append another take.
|
||||
const r2 = await runPhaseConsolidate(engine, {});
|
||||
expect(r2.details.facts_consolidated).toBe(4);
|
||||
// takes_written reports the NEW takes inserted this run; on the upsert
|
||||
// hit path it's 0 (no new INSERT) but facts still get marked consolidated.
|
||||
expect(r2.details.takes_written).toBe(0);
|
||||
|
||||
const countAfter2 = await engine.executeRaw<{ n: string }>(
|
||||
`SELECT COUNT(*)::text AS n FROM takes WHERE page_id = (SELECT id FROM pages WHERE slug = 'cdx4-idempo-1')`,
|
||||
);
|
||||
expect(parseInt(countAfter2[0].n, 10)).toBe(1); // STILL 1 — no duplicate
|
||||
|
||||
// Facts were re-consolidated into the existing take.
|
||||
const facts = await engine.executeRaw<{ consolidated_into: number }>(
|
||||
`SELECT consolidated_into FROM facts WHERE entity_slug = 'cdx4-idempo-1' AND consolidated_into IS NOT NULL`,
|
||||
);
|
||||
expect(facts.length).toBe(4);
|
||||
});
|
||||
|
||||
test('valid_until idempotency: second run leaves valid_until unchanged (no diff)', async () => {
|
||||
await seedPage('cdx4-idempo-2');
|
||||
const t1 = new Date('2026-01-15T00:00:00Z');
|
||||
const t2 = new Date('2026-04-12T00:00:00Z');
|
||||
const t3 = new Date('2026-07-08T00:00:00Z');
|
||||
await insertFact({ entity_slug: 'cdx4-idempo-2', text: 'iterable', valid_from: t1 });
|
||||
await insertFact({ entity_slug: 'cdx4-idempo-2', text: 'iterable', valid_from: t2 });
|
||||
await insertFact({ entity_slug: 'cdx4-idempo-2', text: 'iterable', valid_from: t3 });
|
||||
|
||||
await runPhaseConsolidate(engine, {});
|
||||
const before = await engine.executeRaw<{ id: number; valid_until: Date | null }>(
|
||||
`SELECT id, valid_until FROM facts WHERE entity_slug = 'cdx4-idempo-2' ORDER BY valid_from ASC`,
|
||||
);
|
||||
|
||||
// Reset consolidated_at to simulate extract_facts re-run.
|
||||
await engine.executeRaw(
|
||||
`UPDATE facts SET consolidated_at = NULL, consolidated_into = NULL
|
||||
WHERE entity_slug = 'cdx4-idempo-2'`,
|
||||
);
|
||||
|
||||
await runPhaseConsolidate(engine, {});
|
||||
const after = await engine.executeRaw<{ id: number; valid_until: Date | null }>(
|
||||
`SELECT id, valid_until FROM facts WHERE entity_slug = 'cdx4-idempo-2' ORDER BY valid_from ASC`,
|
||||
);
|
||||
// Same valid_until values; the IS DISTINCT FROM guard avoided rewrites.
|
||||
expect(after.length).toBe(3);
|
||||
for (let i = 0; i < before.length; i++) {
|
||||
expect(after[i].id).toBe(before[i].id);
|
||||
const a = after[i].valid_until ? new Date(after[i].valid_until!).toISOString() : null;
|
||||
const b = before[i].valid_until ? new Date(before[i].valid_until!).toISOString() : null;
|
||||
expect(a).toBe(b);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,320 @@
|
||||
/**
|
||||
* v0.35.4 — BrainEngine.findTrajectory (T4) + trajectory.ts derived
|
||||
* metrics tests.
|
||||
*
|
||||
* Pins:
|
||||
* - Chronological ordering by (valid_from ASC, fact_id ASC) — R3.
|
||||
* - Source scoping (scalar + federated array, D-CDX-6).
|
||||
* - Visibility filter for remote callers (D-CDX-1) — R6.
|
||||
* - Metric filter narrows results to a single canonical name.
|
||||
* - since/until window honored.
|
||||
* - Regression detection per locked threshold (D-ENG-2).
|
||||
* - Drift score returns null when <3 embedded points (G3).
|
||||
* - Empty entity returns {points: [], regressions: [], drift_score: null} (G1).
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import {
|
||||
detectRegressions,
|
||||
computeDriftScore,
|
||||
computeTrajectoryStats,
|
||||
DEFAULT_REGRESSION_THRESHOLD,
|
||||
} from '../src/core/trajectory.ts';
|
||||
import type { TrajectoryPoint } from '../src/core/engine.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await engine.executeRaw(`DELETE FROM facts WHERE entity_slug LIKE 'traj-%'`);
|
||||
await engine.executeRaw(`DELETE FROM sources WHERE id LIKE 'traj-%'`);
|
||||
});
|
||||
|
||||
function vecForMetric(metric: string, offset: number): string {
|
||||
// Deterministic per-metric/offset embedding: each metric gets a
|
||||
// unit-vector in a different "direction" of the embedding space, with
|
||||
// a small perturbation per offset so consecutive same-metric facts
|
||||
// are very-similar-but-not-identical (drift score lands between 0 and
|
||||
// some small value).
|
||||
const a = new Float32Array(1536);
|
||||
const idx = (metric.charCodeAt(0) + offset) % 1536;
|
||||
a[idx] = 1.0;
|
||||
a[(idx + 1) % 1536] = 0.05 * offset; // tiny drift between consecutive
|
||||
return '[' + Array.from(a).join(',') + ']';
|
||||
}
|
||||
|
||||
async function insertTyped(args: {
|
||||
source_id?: string;
|
||||
entity_slug: string;
|
||||
metric: string;
|
||||
value: number;
|
||||
unit?: string;
|
||||
period?: string;
|
||||
valid_from: Date;
|
||||
visibility?: 'private' | 'world';
|
||||
offset?: number;
|
||||
text?: string;
|
||||
}): Promise<number> {
|
||||
const sid = args.source_id ?? 'default';
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO sources (id, name) VALUES ($1, $1) ON CONFLICT DO NOTHING`,
|
||||
[sid],
|
||||
);
|
||||
const r = await engine.executeRaw<{ id: number }>(
|
||||
`INSERT INTO facts (source_id, entity_slug, fact, kind, source, valid_from,
|
||||
claim_metric, claim_value, claim_unit, claim_period,
|
||||
visibility, embedding, embedded_at)
|
||||
VALUES ($1, $2, $3, 'fact', 'test', $4::timestamptz,
|
||||
$5, $6, $7, $8,
|
||||
$9, $10::vector, $4::timestamptz)
|
||||
RETURNING id`,
|
||||
[
|
||||
sid, args.entity_slug, args.text ?? `${args.metric} ${args.value}`,
|
||||
args.valid_from.toISOString(),
|
||||
args.metric, args.value, args.unit ?? null, args.period ?? null,
|
||||
args.visibility ?? 'private',
|
||||
vecForMetric(args.metric, args.offset ?? 0),
|
||||
],
|
||||
);
|
||||
return r[0].id;
|
||||
}
|
||||
|
||||
describe('findTrajectory — chronological ordering (R3)', () => {
|
||||
test('returns points in (valid_from ASC, id ASC) order regardless of insert order', async () => {
|
||||
// Insert out of order. Engine must re-order.
|
||||
const idJul = await insertTyped({ entity_slug: 'traj-order', metric: 'mrr', value: 150000, valid_from: new Date('2026-07-08') });
|
||||
const idJan = await insertTyped({ entity_slug: 'traj-order', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') });
|
||||
const idApr = await insertTyped({ entity_slug: 'traj-order', metric: 'mrr', value: 200000, valid_from: new Date('2026-04-12') });
|
||||
|
||||
const points = await engine.findTrajectory({ entitySlug: 'traj-order' });
|
||||
expect(points.map(p => p.fact_id)).toEqual([idJan, idApr, idJul]);
|
||||
expect(points[0].valid_from.toISOString().slice(0, 10)).toBe('2026-01-15');
|
||||
expect(points[2].valid_from.toISOString().slice(0, 10)).toBe('2026-07-08');
|
||||
});
|
||||
});
|
||||
|
||||
describe('findTrajectory — source scoping (D-CDX-6)', () => {
|
||||
test('scalar sourceId returns only that source', async () => {
|
||||
await insertTyped({ source_id: 'traj-src-A', entity_slug: 'traj-srcscope', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') });
|
||||
await insertTyped({ source_id: 'traj-src-B', entity_slug: 'traj-srcscope', metric: 'mrr', value: 99999, valid_from: new Date('2026-01-15') });
|
||||
|
||||
const pointsA = await engine.findTrajectory({ entitySlug: 'traj-srcscope', sourceId: 'traj-src-A' });
|
||||
expect(pointsA.length).toBe(1);
|
||||
expect(pointsA[0].value).toBe(50000);
|
||||
|
||||
const pointsB = await engine.findTrajectory({ entitySlug: 'traj-srcscope', sourceId: 'traj-src-B' });
|
||||
expect(pointsB.length).toBe(1);
|
||||
expect(pointsB[0].value).toBe(99999);
|
||||
});
|
||||
|
||||
test('federated sourceIds returns union across the array', async () => {
|
||||
await insertTyped({ source_id: 'traj-src-A', entity_slug: 'traj-fed', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') });
|
||||
await insertTyped({ source_id: 'traj-src-B', entity_slug: 'traj-fed', metric: 'mrr', value: 99999, valid_from: new Date('2026-04-12') });
|
||||
await insertTyped({ source_id: 'traj-src-C', entity_slug: 'traj-fed', metric: 'mrr', value: 11111, valid_from: new Date('2026-07-08') });
|
||||
|
||||
const points = await engine.findTrajectory({
|
||||
entitySlug: 'traj-fed',
|
||||
sourceIds: ['traj-src-A', 'traj-src-B'],
|
||||
});
|
||||
// Two of three sources visible, in chronological order.
|
||||
expect(points.length).toBe(2);
|
||||
expect(points.map(p => p.value)).toEqual([50000, 99999]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('findTrajectory — visibility filter (D-CDX-1 / R6)', () => {
|
||||
test('remote=true returns ONLY world-visibility points', async () => {
|
||||
await insertTyped({ entity_slug: 'traj-vis', metric: 'mrr', value: 50000, visibility: 'private', valid_from: new Date('2026-01-15') });
|
||||
await insertTyped({ entity_slug: 'traj-vis', metric: 'mrr', value: 99999, visibility: 'world', valid_from: new Date('2026-04-12') });
|
||||
|
||||
const trusted = await engine.findTrajectory({ entitySlug: 'traj-vis', remote: false });
|
||||
expect(trusted.length).toBe(2); // local CLI sees both
|
||||
|
||||
const remote = await engine.findTrajectory({ entitySlug: 'traj-vis', remote: true });
|
||||
expect(remote.length).toBe(1); // OAuth client sees world only
|
||||
expect(remote[0].value).toBe(99999);
|
||||
});
|
||||
|
||||
test('remote default (undefined) is treated as trusted — sees both', async () => {
|
||||
await insertTyped({ entity_slug: 'traj-vis-default', metric: 'mrr', value: 50000, visibility: 'private', valid_from: new Date('2026-01-15') });
|
||||
await insertTyped({ entity_slug: 'traj-vis-default', metric: 'mrr', value: 99999, visibility: 'world', valid_from: new Date('2026-04-12') });
|
||||
|
||||
// No `remote` field — engine default must be trusted.
|
||||
const all = await engine.findTrajectory({ entitySlug: 'traj-vis-default' });
|
||||
expect(all.length).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('findTrajectory — metric + since + until filters', () => {
|
||||
test('metric filter narrows to one canonical name', async () => {
|
||||
await insertTyped({ entity_slug: 'traj-m', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') });
|
||||
await insertTyped({ entity_slug: 'traj-m', metric: 'arr', value: 600000, valid_from: new Date('2026-01-15') });
|
||||
|
||||
const mrrOnly = await engine.findTrajectory({ entitySlug: 'traj-m', metric: 'mrr' });
|
||||
expect(mrrOnly.length).toBe(1);
|
||||
expect(mrrOnly[0].metric).toBe('mrr');
|
||||
});
|
||||
|
||||
test('since/until window honored', async () => {
|
||||
await insertTyped({ entity_slug: 'traj-w', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') });
|
||||
await insertTyped({ entity_slug: 'traj-w', metric: 'mrr', value: 99999, valid_from: new Date('2026-04-12') });
|
||||
await insertTyped({ entity_slug: 'traj-w', metric: 'mrr', value: 11111, valid_from: new Date('2026-07-08') });
|
||||
|
||||
const inWindow = await engine.findTrajectory({
|
||||
entitySlug: 'traj-w',
|
||||
since: '2026-02-01',
|
||||
until: '2026-05-01',
|
||||
});
|
||||
expect(inWindow.length).toBe(1);
|
||||
expect(inWindow[0].value).toBe(99999);
|
||||
});
|
||||
|
||||
test('unknown entity returns empty array', async () => {
|
||||
const empty = await engine.findTrajectory({ entitySlug: 'traj-does-not-exist' });
|
||||
expect(empty).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
// trajectory.ts pure-function tests
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
|
||||
function makePoint(args: {
|
||||
id: number;
|
||||
metric: string;
|
||||
value: number;
|
||||
date: string;
|
||||
emb?: Float32Array | null;
|
||||
}): TrajectoryPoint {
|
||||
return {
|
||||
fact_id: args.id,
|
||||
valid_from: new Date(args.date),
|
||||
metric: args.metric,
|
||||
value: args.value,
|
||||
unit: 'USD',
|
||||
period: 'monthly',
|
||||
text: `${args.metric} = ${args.value}`,
|
||||
source_session: null,
|
||||
source_markdown_slug: null,
|
||||
embedding: args.emb ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
describe('detectRegressions (D-ENG-2)', () => {
|
||||
test('emits a regression when newer value drops by >= threshold', () => {
|
||||
const points: TrajectoryPoint[] = [
|
||||
makePoint({ id: 1, metric: 'mrr', value: 200000, date: '2026-04-12' }),
|
||||
makePoint({ id: 2, metric: 'mrr', value: 150000, date: '2026-07-08' }), // -25%
|
||||
];
|
||||
const regs = detectRegressions(points, DEFAULT_REGRESSION_THRESHOLD);
|
||||
expect(regs.length).toBe(1);
|
||||
expect(regs[0].metric).toBe('mrr');
|
||||
expect(regs[0].delta_pct).toBeCloseTo(-0.25, 4);
|
||||
expect(regs[0].from_date).toBe('2026-04-12');
|
||||
expect(regs[0].to_date).toBe('2026-07-08');
|
||||
});
|
||||
|
||||
test('skips when drop is below threshold (5% with default 10%)', () => {
|
||||
const points: TrajectoryPoint[] = [
|
||||
makePoint({ id: 1, metric: 'mrr', value: 100000, date: '2026-01-15' }),
|
||||
makePoint({ id: 2, metric: 'mrr', value: 95000, date: '2026-04-12' }), // -5%
|
||||
];
|
||||
expect(detectRegressions(points).length).toBe(0);
|
||||
});
|
||||
|
||||
test('multiple metrics tracked independently', () => {
|
||||
const points: TrajectoryPoint[] = [
|
||||
makePoint({ id: 1, metric: 'mrr', value: 200000, date: '2026-04-12' }),
|
||||
makePoint({ id: 2, metric: 'arr', value: 600000, date: '2026-04-12' }),
|
||||
makePoint({ id: 3, metric: 'mrr', value: 150000, date: '2026-07-08' }), // -25% mrr
|
||||
makePoint({ id: 4, metric: 'arr', value: 700000, date: '2026-07-08' }), // +16% arr → no regression
|
||||
];
|
||||
const regs = detectRegressions(points);
|
||||
expect(regs.length).toBe(1);
|
||||
expect(regs[0].metric).toBe('mrr');
|
||||
});
|
||||
|
||||
test('skips points with null value', () => {
|
||||
const points: TrajectoryPoint[] = [
|
||||
makePoint({ id: 1, metric: 'mrr', value: 200000, date: '2026-04-12' }),
|
||||
{ ...makePoint({ id: 2, metric: 'mrr', value: 0, date: '2026-07-08' }), value: null },
|
||||
];
|
||||
expect(detectRegressions(points).length).toBe(0);
|
||||
});
|
||||
|
||||
test('skips when older value is 0 (division-by-zero guard)', () => {
|
||||
const points: TrajectoryPoint[] = [
|
||||
makePoint({ id: 1, metric: 'mrr', value: 0, date: '2026-04-12' }),
|
||||
makePoint({ id: 2, metric: 'mrr', value: 1000, date: '2026-07-08' }),
|
||||
];
|
||||
expect(detectRegressions(points).length).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('computeDriftScore (D-ENG-3 / G3)', () => {
|
||||
function unitVec(dim: number, offset: number): Float32Array {
|
||||
const a = new Float32Array(8);
|
||||
a[offset % 8] = 1.0;
|
||||
return a;
|
||||
}
|
||||
|
||||
test('returns null with fewer than 3 embedded points', () => {
|
||||
const points: TrajectoryPoint[] = [
|
||||
makePoint({ id: 1, metric: 'mrr', value: 1, date: '2026-01-15', emb: unitVec(8, 0) }),
|
||||
makePoint({ id: 2, metric: 'mrr', value: 2, date: '2026-04-12', emb: unitVec(8, 1) }),
|
||||
];
|
||||
expect(computeDriftScore(points)).toBeNull();
|
||||
});
|
||||
|
||||
test('returns null when no points have embeddings (G3 graceful fallback)', () => {
|
||||
const points: TrajectoryPoint[] = [
|
||||
makePoint({ id: 1, metric: 'mrr', value: 1, date: '2026-01-15' }),
|
||||
makePoint({ id: 2, metric: 'mrr', value: 2, date: '2026-04-12' }),
|
||||
makePoint({ id: 3, metric: 'mrr', value: 3, date: '2026-07-08' }),
|
||||
];
|
||||
expect(computeDriftScore(points)).toBeNull();
|
||||
});
|
||||
|
||||
test('identical consecutive embeddings → drift 0 (cohesive narrative)', () => {
|
||||
const v = unitVec(8, 0);
|
||||
const points: TrajectoryPoint[] = [
|
||||
makePoint({ id: 1, metric: 'mrr', value: 1, date: '2026-01-15', emb: v }),
|
||||
makePoint({ id: 2, metric: 'mrr', value: 2, date: '2026-04-12', emb: v }),
|
||||
makePoint({ id: 3, metric: 'mrr', value: 3, date: '2026-07-08', emb: v }),
|
||||
];
|
||||
expect(computeDriftScore(points)).toBe(0);
|
||||
});
|
||||
|
||||
test('orthogonal consecutive embeddings → drift 1 (every claim unrelated)', () => {
|
||||
const points: TrajectoryPoint[] = [
|
||||
makePoint({ id: 1, metric: 'mrr', value: 1, date: '2026-01-15', emb: unitVec(8, 0) }),
|
||||
makePoint({ id: 2, metric: 'mrr', value: 2, date: '2026-04-12', emb: unitVec(8, 1) }),
|
||||
makePoint({ id: 3, metric: 'mrr', value: 3, date: '2026-07-08', emb: unitVec(8, 2) }),
|
||||
];
|
||||
expect(computeDriftScore(points)).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('computeTrajectoryStats — composed shape', () => {
|
||||
test('returns both regressions + drift_score in one call', () => {
|
||||
const v = new Float32Array(4);
|
||||
v[0] = 1;
|
||||
const points: TrajectoryPoint[] = [
|
||||
makePoint({ id: 1, metric: 'mrr', value: 200000, date: '2026-04-12', emb: v }),
|
||||
makePoint({ id: 2, metric: 'mrr', value: 150000, date: '2026-07-08', emb: v }),
|
||||
];
|
||||
const stats = computeTrajectoryStats(points);
|
||||
expect(stats.regressions.length).toBe(1);
|
||||
expect(stats.drift_score).toBeNull(); // <3 embedded
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,158 @@
|
||||
/**
|
||||
* v0.35.4 (R1 + R8 — D-CDX-4 + D-CDX-7) — IRON-RULE: the contradiction
|
||||
* probe NEVER writes `valid_until` on the facts table.
|
||||
*
|
||||
* The temporal trajectory wave (v0.35.4) gives `consolidate` the
|
||||
* authority to write `valid_until` on chronologically-superseded facts.
|
||||
* That authority is exclusive: the contradiction probe surfaces
|
||||
* `temporal_supersession` verdicts via paste-ready commands, but it
|
||||
* must NEVER auto-mutate. This preserves the
|
||||
* `src/core/eval-contradictions/auto-supersession.ts:4` invariant.
|
||||
*
|
||||
* Two layered guards:
|
||||
* R1 — grep guard over the entire `src/core/eval-contradictions/`
|
||||
* subtree and the `src/commands/eval-suspected-contradictions*.ts`
|
||||
* files: no code path may UPDATE facts.valid_until.
|
||||
* R8 — broader guard over all of `src/`: the only file that writes
|
||||
* valid_until is `src/core/cycle/phases/consolidate.ts`. Any new
|
||||
* write site fails this guard; the human adding it must explicitly
|
||||
* amend the allow-list AND document the deliberate design change.
|
||||
*/
|
||||
|
||||
import { test, expect, describe } from 'bun:test';
|
||||
import { readdirSync, readFileSync, statSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
|
||||
// Allow-listed files that legitimately write `valid_until`. Adding a new
|
||||
// file here means you've thought carefully about the
|
||||
// auto-supersession.ts:4 invariant and decided the new write site
|
||||
// preserves it.
|
||||
//
|
||||
// - consolidate.ts (v0.35.4 — chronological writeback)
|
||||
// - facts/forget.ts (v0.32.2 — user-initiated `gbrain forget`; user is
|
||||
// the supersession authority, not the probe)
|
||||
const VALID_UNTIL_WRITE_ALLOWLIST: ReadonlySet<string> = new Set([
|
||||
'src/core/cycle/phases/consolidate.ts',
|
||||
'src/core/facts/forget.ts',
|
||||
]);
|
||||
|
||||
function walkTs(dir: string, acc: string[] = []): string[] {
|
||||
for (const name of readdirSync(dir)) {
|
||||
const full = join(dir, name);
|
||||
const st = statSync(full);
|
||||
if (st.isDirectory()) {
|
||||
// Skip generated / vendored / test directories.
|
||||
if (name === 'node_modules' || name === '.git' || name === 'dist') continue;
|
||||
walkTs(full, acc);
|
||||
} else if (name.endsWith('.ts') && !name.endsWith('.test.ts')) {
|
||||
acc.push(full);
|
||||
}
|
||||
}
|
||||
return acc;
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect lines that look like they are UPDATEing facts.valid_until.
|
||||
* Permissive on SQL formatting (the same UPDATE can be split across lines,
|
||||
* use template strings, or use parameterized SQL via postgres.js's
|
||||
* tagged-template syntax). We look for the pair of substrings near
|
||||
* each other in the same file: `UPDATE facts` and `valid_until`.
|
||||
*
|
||||
* Tolerated: a reference to `valid_until` in a SELECT projection list
|
||||
* (which is fine — reading the column is not writing it) is filtered out
|
||||
* by also requiring a write verb (SET) nearby OR an INSERT INTO facts
|
||||
* with valid_until in the column list (INSERT path is OK from
|
||||
* src/core/postgres-engine.ts + src/core/pglite-engine.ts because those
|
||||
* are the engine layer, intentionally writing on caller request via
|
||||
* insertFact/insertFacts; the issue is whether the contradiction probe
|
||||
* path triggers those writes).
|
||||
*/
|
||||
function findValidUntilWrites(source: string): string[] {
|
||||
// Quick-fail: no mention of valid_until at all.
|
||||
if (!source.includes('valid_until')) return [];
|
||||
const lines = source.split('\n');
|
||||
const hits: string[] = [];
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const line = lines[i];
|
||||
// Detect actual SQL writes. The narrow pattern `UPDATE facts SET ...
|
||||
// valid_until` is unambiguous — UPDATE is a SQL verb, not text
|
||||
// anyone writes in a description string. Tolerates same-line and
|
||||
// multi-line UPDATEs (look at the next 4 lines after `UPDATE facts`
|
||||
// for `valid_until`).
|
||||
if (/\bUPDATE\s+facts\b/i.test(line)) {
|
||||
const window = lines.slice(i, i + 5).join('\n');
|
||||
if (/\bSET\b[\s\S]*\bvalid_until\b/i.test(window)) {
|
||||
hits.push(`${i + 1}: ${line.trim()}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
return hits;
|
||||
}
|
||||
|
||||
describe('R1 — contradiction probe never writes valid_until', () => {
|
||||
test('no file under src/core/eval-contradictions/ writes facts.valid_until', () => {
|
||||
const dir = 'src/core/eval-contradictions';
|
||||
const files = walkTs(dir);
|
||||
expect(files.length).toBeGreaterThan(0);
|
||||
for (const f of files) {
|
||||
const src = readFileSync(f, 'utf-8');
|
||||
const hits = findValidUntilWrites(src);
|
||||
expect(hits, `${f} contains valid_until write: ${hits.join(' | ')}`).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
test('no src/commands/eval-suspected-contradictions* file writes facts.valid_until', () => {
|
||||
const dir = 'src/commands';
|
||||
const files = readdirSync(dir)
|
||||
.filter(n => n.startsWith('eval-suspected-contradictions') && n.endsWith('.ts'))
|
||||
.map(n => join(dir, n));
|
||||
expect(files.length).toBeGreaterThanOrEqual(1);
|
||||
for (const f of files) {
|
||||
const src = readFileSync(f, 'utf-8');
|
||||
const hits = findValidUntilWrites(src);
|
||||
expect(hits, `${f} contains valid_until write: ${hits.join(' | ')}`).toEqual([]);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('R8 — only the consolidate phase + engine insert layer may write valid_until', () => {
|
||||
test('every src/ TypeScript file that writes valid_until is on the allow-list', () => {
|
||||
const files = walkTs('src');
|
||||
const offenders: Array<{ file: string; hits: string[] }> = [];
|
||||
|
||||
for (const f of files) {
|
||||
// Normalize path separator for cross-platform matching of the
|
||||
// allow-list keys.
|
||||
const relForCheck = f.replace(/\\/g, '/');
|
||||
if (VALID_UNTIL_WRITE_ALLOWLIST.has(relForCheck)) continue;
|
||||
|
||||
// Engine implementation files (postgres-engine.ts, pglite-engine.ts)
|
||||
// legitimately write valid_until inside insertFact/insertFacts —
|
||||
// those are caller-driven INSERTs. Pattern is `INSERT INTO facts (
|
||||
// ... valid_until ...) VALUES`. Detect and exempt that pattern.
|
||||
// The R8 guard is about UPDATE; INSERT carrying valid_until as a
|
||||
// column value is not the failure mode auto-supersession.ts:4 cares
|
||||
// about.
|
||||
const src = readFileSync(f, 'utf-8');
|
||||
const hits = findValidUntilWrites(src);
|
||||
if (hits.length > 0) offenders.push({ file: relForCheck, hits });
|
||||
}
|
||||
|
||||
expect(
|
||||
offenders,
|
||||
`Unexpected valid_until UPDATE sites:\n` +
|
||||
offenders.map(o => ` ${o.file}:\n ${o.hits.join('\n ')}`).join('\n') +
|
||||
`\n\nIf you added a deliberate write, append the path to ` +
|
||||
`VALID_UNTIL_WRITE_ALLOWLIST in this test AND review the ` +
|
||||
`\`auto-supersession.ts:4\` invariant first.`,
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
test('VALID_UNTIL_WRITE_ALLOWLIST is non-empty (consolidate is on it)', () => {
|
||||
// Self-check: if the test file ever ships with an empty allow-list,
|
||||
// the R8 guard collapses to "no code writes valid_until anywhere" and
|
||||
// becomes a tautology. Keep the consolidate phase on the list as the
|
||||
// explicit positive control.
|
||||
expect(VALID_UNTIL_WRITE_ALLOWLIST.has('src/core/cycle/phases/consolidate.ts')).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,153 @@
|
||||
/**
|
||||
* v0.35.4 — `gbrain eval trajectory` CLI (T6) tests.
|
||||
*
|
||||
* Pins:
|
||||
* - argv parser: positional entity-slug required; --metric / --since /
|
||||
* --until / --limit / --json honored; unknown flags rejected.
|
||||
* - --json output has the stable schema_version: 1 envelope (R5).
|
||||
* - Human format includes the regression marker for points that match.
|
||||
* - Empty result graceful shape (G1).
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { runEvalTrajectory } from '../src/commands/eval-trajectory.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await engine.executeRaw(`DELETE FROM facts WHERE entity_slug LIKE 'cli-traj-%'`);
|
||||
});
|
||||
|
||||
function unitVec(idx = 0): string {
|
||||
const a = new Float32Array(1536);
|
||||
a[idx % 1536] = 1.0;
|
||||
return '[' + Array.from(a).join(',') + ']';
|
||||
}
|
||||
|
||||
async function insertTyped(args: {
|
||||
entity_slug: string;
|
||||
metric: string;
|
||||
value: number;
|
||||
valid_from: Date;
|
||||
unit?: string;
|
||||
}): Promise<void> {
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO facts (source_id, entity_slug, fact, kind, source, valid_from,
|
||||
claim_metric, claim_value, claim_unit, claim_period,
|
||||
visibility, embedding, embedded_at)
|
||||
VALUES ('default', $1, $2, 'fact', 'test', $3::timestamptz,
|
||||
$4, $5, $6, 'monthly',
|
||||
'private', $7::vector, $3::timestamptz)`,
|
||||
[args.entity_slug, `${args.metric} ${args.value}`, args.valid_from.toISOString(),
|
||||
args.metric, args.value, args.unit ?? 'USD', unitVec()],
|
||||
);
|
||||
}
|
||||
|
||||
/** Capture console.log output to assert on. */
|
||||
async function captureRun(args: string[]): Promise<{ out: string; err: string }> {
|
||||
const origLog = console.log;
|
||||
const origErr = console.error;
|
||||
let out = '';
|
||||
let err = '';
|
||||
console.log = (...a: unknown[]) => { out += a.map(String).join(' ') + '\n'; };
|
||||
console.error = (...a: unknown[]) => { err += a.map(String).join(' ') + '\n'; };
|
||||
try {
|
||||
await runEvalTrajectory(engine, args);
|
||||
} finally {
|
||||
console.log = origLog;
|
||||
console.error = origErr;
|
||||
}
|
||||
return { out, err };
|
||||
}
|
||||
|
||||
describe('eval-trajectory CLI — arg parsing', () => {
|
||||
test('--help prints usage and returns without DB call', async () => {
|
||||
const { out } = await captureRun(['--help']);
|
||||
expect(out).toContain('Usage: gbrain eval trajectory');
|
||||
expect(out).toContain('--metric');
|
||||
});
|
||||
|
||||
test('missing positional arg surfaces an error + non-zero exit', async () => {
|
||||
// process.exit throws inside Bun test runner; capture via try/catch.
|
||||
let exitCode: number | undefined;
|
||||
const origExit = process.exit;
|
||||
(process as any).exit = (code?: number) => {
|
||||
exitCode = code;
|
||||
throw new Error('__exit_intercept__');
|
||||
};
|
||||
try {
|
||||
await captureRun([]);
|
||||
} catch (e: any) {
|
||||
if (!String(e).includes('__exit_intercept__')) throw e;
|
||||
} finally {
|
||||
process.exit = origExit;
|
||||
}
|
||||
expect(exitCode).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('eval-trajectory CLI — JSON envelope (R5)', () => {
|
||||
test('--json output has schema_version: 1 and points + regressions + drift_score keys', async () => {
|
||||
await insertTyped({ entity_slug: 'cli-traj-shape', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') });
|
||||
const { out } = await captureRun(['cli-traj-shape', '--json']);
|
||||
const parsed = JSON.parse(out);
|
||||
expect(parsed.schema_version).toBe(1);
|
||||
expect(parsed).toHaveProperty('points');
|
||||
expect(parsed).toHaveProperty('regressions');
|
||||
expect(parsed).toHaveProperty('drift_score');
|
||||
// Engine's raw embedding is NOT in the CLI JSON output.
|
||||
expect(parsed.points[0]).not.toHaveProperty('embedding');
|
||||
expect(parsed.points[0].valid_from).toMatch(/^\d{4}-\d{2}-\d{2}$/);
|
||||
});
|
||||
|
||||
test('--json empty-entity output is the same shape (G1)', async () => {
|
||||
const { out } = await captureRun(['cli-traj-nonexistent', '--json']);
|
||||
const parsed = JSON.parse(out);
|
||||
expect(parsed.points).toEqual([]);
|
||||
expect(parsed.regressions).toEqual([]);
|
||||
expect(parsed.drift_score).toBeNull();
|
||||
expect(parsed.schema_version).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('eval-trajectory CLI — regression annotation in human output', () => {
|
||||
test('regression line is marked with [REGRESSION ↓XX.X%] in human format', async () => {
|
||||
await insertTyped({ entity_slug: 'cli-traj-reg', metric: 'mrr', value: 200000, valid_from: new Date('2026-04-12') });
|
||||
await insertTyped({ entity_slug: 'cli-traj-reg', metric: 'mrr', value: 150000, valid_from: new Date('2026-07-08') });
|
||||
|
||||
const { out } = await captureRun(['cli-traj-reg']);
|
||||
expect(out).toContain('Entity: cli-traj-reg');
|
||||
expect(out).toContain('mrr');
|
||||
expect(out).toContain('REGRESSION');
|
||||
expect(out).toContain('25.0%');
|
||||
});
|
||||
|
||||
test('empty entity produces the friendly no-claims message', async () => {
|
||||
const { out } = await captureRun(['cli-traj-nothing']);
|
||||
expect(out).toContain('Entity: cli-traj-nothing');
|
||||
expect(out).toContain('(no typed claims');
|
||||
});
|
||||
});
|
||||
|
||||
describe('eval-trajectory CLI — metric filter narrows results', () => {
|
||||
test('--metric arr returns only ARR points', async () => {
|
||||
await insertTyped({ entity_slug: 'cli-traj-flt', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') });
|
||||
await insertTyped({ entity_slug: 'cli-traj-flt', metric: 'arr', value: 600000, valid_from: new Date('2026-01-15') });
|
||||
|
||||
const { out } = await captureRun(['cli-traj-flt', '--metric', 'arr', '--json']);
|
||||
const parsed = JSON.parse(out);
|
||||
expect(parsed.points.length).toBe(1);
|
||||
expect(parsed.points[0].metric).toBe('arr');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,321 @@
|
||||
// v0.35.4 — typed-claim fence parser+renderer round-trip + normalization.
|
||||
//
|
||||
// Pins:
|
||||
// 1. Backward compat — a fence authored without typed fields still
|
||||
// parses, renders as 10-cell shape, and round-trips byte-identical.
|
||||
// 2. Typed fields parse from the 14-cell widened fence.
|
||||
// 3. Renderer widens to 14 cells iff ANY row has a non-undefined typed
|
||||
// field; otherwise stays at 10-cell (no diff noise on existing fences).
|
||||
// 4. Round-trip preservation: parse → render → parse produces the same
|
||||
// ParsedFact array, including typed fields.
|
||||
// 5. Numeric value cell tolerates thousand separators (`50,000`).
|
||||
|
||||
import { test, expect, describe } from 'bun:test';
|
||||
import {
|
||||
parseFactsFence,
|
||||
renderFactsTable,
|
||||
upsertFactRow,
|
||||
type ParsedFact,
|
||||
} from '../src/core/facts-fence.ts';
|
||||
import {
|
||||
extractFactsFromFenceText,
|
||||
normalizeMetricLabel,
|
||||
METRIC_NORMALIZATION_MAP,
|
||||
} from '../src/core/facts/extract-from-fence.ts';
|
||||
|
||||
function wrap(inner: string): string {
|
||||
return `## Facts\n\n<!--- gbrain:facts:begin -->\n${inner}\n<!--- gbrain:facts:end -->\n`;
|
||||
}
|
||||
|
||||
describe('v0.35.4 — facts fence typed-claim parser', () => {
|
||||
test('legacy 10-cell fence parses as before; all typed fields undefined', () => {
|
||||
const body = wrap(
|
||||
`| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context |
|
||||
|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|
|
||||
| 1 | Founded Acme in 2017 | fact | 1.0 | world | high | 2017-01-01 | | linkedin | |`,
|
||||
);
|
||||
const { facts, warnings } = parseFactsFence(body);
|
||||
expect(warnings).toEqual([]);
|
||||
expect(facts.length).toBe(1);
|
||||
expect(facts[0].claim).toBe('Founded Acme in 2017');
|
||||
expect(facts[0].claimMetric).toBeUndefined();
|
||||
expect(facts[0].claimValue).toBeUndefined();
|
||||
expect(facts[0].claimUnit).toBeUndefined();
|
||||
expect(facts[0].claimPeriod).toBeUndefined();
|
||||
});
|
||||
|
||||
test('14-cell typed fence parses all four typed-claim fields', () => {
|
||||
const body = wrap(
|
||||
`| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | claim_metric | claim_value | claim_unit | claim_period |
|
||||
|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|--------------|-------------|------------|--------------|
|
||||
| 1 | MRR hit $50K | fact | 1.0 | private | high | 2026-01-15 | | OH transcript | | mrr | 50000 | USD | monthly |`,
|
||||
);
|
||||
const { facts, warnings } = parseFactsFence(body);
|
||||
expect(warnings).toEqual([]);
|
||||
expect(facts.length).toBe(1);
|
||||
expect(facts[0].claimMetric).toBe('mrr');
|
||||
expect(facts[0].claimValue).toBe(50000);
|
||||
expect(facts[0].claimUnit).toBe('USD');
|
||||
expect(facts[0].claimPeriod).toBe('monthly');
|
||||
});
|
||||
|
||||
test('numeric value cell tolerates comma thousand separators', () => {
|
||||
const body = wrap(
|
||||
`| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | claim_metric | claim_value | claim_unit | claim_period |
|
||||
|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|--------------|-------------|------------|--------------|
|
||||
| 1 | ARR | fact | 1.0 | private | high | 2026-04-12 | | bo call | | arr | 2,000,000 | USD | annual |`,
|
||||
);
|
||||
const { facts } = parseFactsFence(body);
|
||||
expect(facts[0].claimValue).toBe(2000000);
|
||||
});
|
||||
|
||||
test('empty typed cells parse as undefined (not "")', () => {
|
||||
const body = wrap(
|
||||
`| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | claim_metric | claim_value | claim_unit | claim_period |
|
||||
|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|--------------|-------------|------------|--------------|
|
||||
| 1 | bare claim | fact | 1.0 | private | high | 2026-01-01 | | | | | | | |`,
|
||||
);
|
||||
const { facts } = parseFactsFence(body);
|
||||
expect(facts[0].claimMetric).toBeUndefined();
|
||||
expect(facts[0].claimValue).toBeUndefined();
|
||||
expect(facts[0].claimUnit).toBeUndefined();
|
||||
expect(facts[0].claimPeriod).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('v0.35.4 — facts fence typed-claim renderer', () => {
|
||||
test('renders 10-cell shape when no row has typed fields (backward compat)', () => {
|
||||
const facts: ParsedFact[] = [
|
||||
{
|
||||
rowNum: 1,
|
||||
claim: 'Founded Acme in 2017',
|
||||
kind: 'fact',
|
||||
confidence: 1.0,
|
||||
visibility: 'world',
|
||||
notability: 'high',
|
||||
validFrom: '2017-01-01',
|
||||
source: 'linkedin',
|
||||
active: true,
|
||||
},
|
||||
];
|
||||
const out = renderFactsTable(facts);
|
||||
// 10-cell header
|
||||
expect(out).toContain('| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context |');
|
||||
// NOT the 14-cell variant
|
||||
expect(out).not.toContain('claim_metric');
|
||||
});
|
||||
|
||||
test('widens to 14 cells when ANY row has typed fields', () => {
|
||||
const facts: ParsedFact[] = [
|
||||
{
|
||||
rowNum: 1,
|
||||
claim: 'plain claim',
|
||||
kind: 'fact',
|
||||
confidence: 1.0,
|
||||
visibility: 'private',
|
||||
notability: 'medium',
|
||||
active: true,
|
||||
},
|
||||
{
|
||||
rowNum: 2,
|
||||
claim: 'MRR hit $50K',
|
||||
kind: 'fact',
|
||||
confidence: 1.0,
|
||||
visibility: 'private',
|
||||
notability: 'high',
|
||||
active: true,
|
||||
claimMetric: 'mrr',
|
||||
claimValue: 50000,
|
||||
claimUnit: 'USD',
|
||||
claimPeriod: 'monthly',
|
||||
},
|
||||
];
|
||||
const out = renderFactsTable(facts);
|
||||
expect(out).toContain('claim_metric');
|
||||
expect(out).toContain('claim_value');
|
||||
expect(out).toContain('claim_unit');
|
||||
expect(out).toContain('claim_period');
|
||||
// Row 1 has empty typed cells; row 2 has the values.
|
||||
expect(out).toContain('| mrr | 50000 | USD | monthly |');
|
||||
});
|
||||
|
||||
test('round-trip preservation: parse → render → parse is structurally idempotent for typed facts', () => {
|
||||
const body = wrap(
|
||||
`| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | claim_metric | claim_value | claim_unit | claim_period |
|
||||
|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|--------------|-------------|------------|--------------|
|
||||
| 1 | MRR $50K | fact | 1.0 | private | high | 2026-01-15 | | OH | | mrr | 50000 | USD | monthly |
|
||||
| 2 | Team grew to 12 | fact | 1.0 | private | medium | 2026-02-01 | | meeting | | team_size | 12 | people | |
|
||||
| 3 | Plain non-typed claim | fact | 0.85 | private | low | 2026-03-01 | | inferred | | | | | |`,
|
||||
);
|
||||
const first = parseFactsFence(body);
|
||||
expect(first.warnings).toEqual([]);
|
||||
expect(first.facts.length).toBe(3);
|
||||
const rendered = renderFactsTable(first.facts);
|
||||
const second = parseFactsFence(rendered);
|
||||
expect(second.warnings).toEqual([]);
|
||||
expect(second.facts).toEqual(first.facts);
|
||||
});
|
||||
|
||||
test('upsertFactRow threads typed fields when a new row carries them', () => {
|
||||
// Start from a fence with NO typed fields → 10-cell shape.
|
||||
const body = wrap(
|
||||
`| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context |
|
||||
|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|
|
||||
| 1 | Founded Acme in 2017 | fact | 1.0 | world | high | 2017-01-01 | | linkedin | |`,
|
||||
);
|
||||
const { body: newBody, rowNum } = upsertFactRow(body, {
|
||||
claim: 'MRR hit $50K',
|
||||
kind: 'fact',
|
||||
confidence: 1.0,
|
||||
visibility: 'private',
|
||||
notability: 'high',
|
||||
validFrom: '2026-01-15',
|
||||
source: 'OH transcript',
|
||||
claimMetric: 'mrr',
|
||||
claimValue: 50000,
|
||||
claimUnit: 'USD',
|
||||
claimPeriod: 'monthly',
|
||||
});
|
||||
expect(rowNum).toBe(2);
|
||||
// Adding a typed row widens the table.
|
||||
expect(newBody).toContain('claim_metric');
|
||||
const parsed = parseFactsFence(newBody);
|
||||
expect(parsed.warnings).toEqual([]);
|
||||
expect(parsed.facts.length).toBe(2);
|
||||
expect(parsed.facts[1].claimMetric).toBe('mrr');
|
||||
expect(parsed.facts[1].claimValue).toBe(50000);
|
||||
});
|
||||
});
|
||||
|
||||
describe('v0.35.4 — metric normalization (D-ENG-4)', () => {
|
||||
test('known seed-map labels normalize to canonical snake_case', () => {
|
||||
expect(normalizeMetricLabel('MRR')).toBe('mrr');
|
||||
expect(normalizeMetricLabel('Monthly Recurring Revenue')).toBe('mrr');
|
||||
expect(normalizeMetricLabel('ARR')).toBe('arr');
|
||||
expect(normalizeMetricLabel('annual recurring revenue')).toBe('arr');
|
||||
expect(normalizeMetricLabel('Team Size')).toBe('team_size');
|
||||
expect(normalizeMetricLabel('Burn Rate')).toBe('burn_rate');
|
||||
expect(normalizeMetricLabel('Churn')).toBe('churn_rate');
|
||||
});
|
||||
|
||||
test('unknown labels lowercase + spaces → underscores; non-alphanumeric stripped', () => {
|
||||
expect(normalizeMetricLabel('Net Promoter Score')).toBe('net_promoter_score');
|
||||
expect(normalizeMetricLabel(' CAC ')).toBe('cac');
|
||||
expect(normalizeMetricLabel('Time-to-Hire')).toBe('timetohire');
|
||||
});
|
||||
|
||||
test('empty / null / undefined → undefined (the "no metric set" signal)', () => {
|
||||
expect(normalizeMetricLabel(undefined)).toBeUndefined();
|
||||
expect(normalizeMetricLabel(null)).toBeUndefined();
|
||||
expect(normalizeMetricLabel('')).toBeUndefined();
|
||||
expect(normalizeMetricLabel(' ')).toBeUndefined();
|
||||
});
|
||||
|
||||
test('METRIC_NORMALIZATION_MAP covers the 15-metric seed list named in the plan', () => {
|
||||
// Pin the seed map so the docs in CLAUDE.md / CHANGELOG match.
|
||||
const required = [
|
||||
'mrr', 'arr', 'runway', 'headcount', 'team_size',
|
||||
'cac', 'ltv', 'gross_margin', 'burn_rate', 'cash',
|
||||
'users', 'mau', 'dau', 'churn_rate', 'revenue',
|
||||
];
|
||||
const canonicalValues = new Set(METRIC_NORMALIZATION_MAP.values());
|
||||
for (const r of required) {
|
||||
expect(canonicalValues.has(r)).toBe(true);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('v0.35.4 (D-ENG-1) — extractFactsFromFenceText valid_from precedence', () => {
|
||||
const fixedToday = new Date('2026-05-17T00:00:00.000Z');
|
||||
|
||||
test('Path 1: explicit validFrom in fence row wins', () => {
|
||||
const facts: ParsedFact[] = [{
|
||||
rowNum: 1,
|
||||
claim: 'fence-dated',
|
||||
kind: 'fact',
|
||||
confidence: 1.0,
|
||||
visibility: 'private',
|
||||
notability: 'medium',
|
||||
validFrom: '2026-01-15',
|
||||
active: true,
|
||||
}];
|
||||
const out = extractFactsFromFenceText(facts, 'people/alice-example', 'default', {
|
||||
nowOverride: fixedToday,
|
||||
pageEffectiveDate: new Date('2026-04-28'),
|
||||
});
|
||||
expect(out[0].valid_from?.toISOString().slice(0, 10)).toBe('2026-01-15');
|
||||
});
|
||||
|
||||
test('Path 2: missing fence validFrom + pageEffectiveDate set → uses page date', () => {
|
||||
const facts: ParsedFact[] = [{
|
||||
rowNum: 1,
|
||||
claim: 'no fence date',
|
||||
kind: 'fact',
|
||||
confidence: 1.0,
|
||||
visibility: 'private',
|
||||
notability: 'medium',
|
||||
active: true,
|
||||
}];
|
||||
const out = extractFactsFromFenceText(facts, 'people/alice-example', 'default', {
|
||||
nowOverride: fixedToday,
|
||||
pageEffectiveDate: new Date('2026-04-28'),
|
||||
});
|
||||
expect(out[0].valid_from?.toISOString().slice(0, 10)).toBe('2026-04-28');
|
||||
});
|
||||
|
||||
test('Path 3: missing fence validFrom AND undefined pageEffectiveDate → undefined (engine defaults to now)', () => {
|
||||
const facts: ParsedFact[] = [{
|
||||
rowNum: 1,
|
||||
claim: 'no dates at all',
|
||||
kind: 'fact',
|
||||
confidence: 1.0,
|
||||
visibility: 'private',
|
||||
notability: 'medium',
|
||||
active: true,
|
||||
}];
|
||||
const out = extractFactsFromFenceText(facts, 'people/alice-example', 'default', {
|
||||
nowOverride: fixedToday,
|
||||
pageEffectiveDate: null,
|
||||
});
|
||||
// valid_from is left undefined; the engine layer applies now() at insert.
|
||||
expect(out[0].valid_from).toBeUndefined();
|
||||
});
|
||||
|
||||
test('typed-claim fields thread through with metric normalization applied', () => {
|
||||
const facts: ParsedFact[] = [{
|
||||
rowNum: 1,
|
||||
claim: 'MRR $50K',
|
||||
kind: 'fact',
|
||||
confidence: 1.0,
|
||||
visibility: 'private',
|
||||
notability: 'high',
|
||||
active: true,
|
||||
claimMetric: 'Monthly Recurring Revenue', // unnormalized
|
||||
claimValue: 50000,
|
||||
claimUnit: 'USD',
|
||||
claimPeriod: 'monthly',
|
||||
}];
|
||||
const out = extractFactsFromFenceText(facts, 'companies/acme-example', 'default');
|
||||
expect(out[0].claim_metric).toBe('mrr'); // normalized
|
||||
expect(out[0].claim_value).toBe(50000);
|
||||
expect(out[0].claim_unit).toBe('USD');
|
||||
expect(out[0].claim_period).toBe('monthly');
|
||||
});
|
||||
|
||||
test('rows with no typed-claim fields land with null claim_* columns (backward compat)', () => {
|
||||
const facts: ParsedFact[] = [{
|
||||
rowNum: 1,
|
||||
claim: 'bare claim',
|
||||
kind: 'fact',
|
||||
confidence: 1.0,
|
||||
visibility: 'private',
|
||||
notability: 'low',
|
||||
active: true,
|
||||
}];
|
||||
const out = extractFactsFromFenceText(facts, 'people/bob-example', 'default');
|
||||
expect(out[0].claim_metric).toBeNull();
|
||||
expect(out[0].claim_value).toBeNull();
|
||||
expect(out[0].claim_unit).toBeNull();
|
||||
expect(out[0].claim_period).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,242 @@
|
||||
/**
|
||||
* v0.35.4 — `gbrain founder scorecard` CLI (T7) tests.
|
||||
*
|
||||
* Pins:
|
||||
* - Pure compute fn: each of the four rollup fields produces correct math.
|
||||
* - JSON envelope has schema_version: 1 + every required field (R5).
|
||||
* - G2: empty entity (no facts, no takes) returns a valid empty rollup
|
||||
* with no NaN / nulls in numeric slots.
|
||||
* - Red flags fire for regressions + high drift + missed predictions.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { computeFounderScorecard } from '../src/commands/founder-scorecard.ts';
|
||||
import type { TrajectoryPoint, Take } from '../src/core/engine.ts';
|
||||
|
||||
function pt(args: {
|
||||
id: number;
|
||||
metric: string;
|
||||
value: number;
|
||||
date: string;
|
||||
emb?: Float32Array | null;
|
||||
}): TrajectoryPoint {
|
||||
return {
|
||||
fact_id: args.id,
|
||||
valid_from: new Date(args.date),
|
||||
metric: args.metric,
|
||||
value: args.value,
|
||||
unit: 'USD',
|
||||
period: 'monthly',
|
||||
text: `${args.metric} = ${args.value}`,
|
||||
source_session: null,
|
||||
source_markdown_slug: null,
|
||||
embedding: args.emb ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
function take(args: {
|
||||
id: number;
|
||||
claim: string;
|
||||
resolved_outcome: boolean | null;
|
||||
}): Take {
|
||||
return {
|
||||
id: args.id,
|
||||
page_id: 1,
|
||||
page_slug: 'companies/acme-example',
|
||||
row_num: args.id,
|
||||
claim: args.claim,
|
||||
kind: 'fact',
|
||||
holder: 'self',
|
||||
weight: 0.9,
|
||||
since_date: '2026-01-15',
|
||||
until_date: null,
|
||||
source: 'test',
|
||||
active: true,
|
||||
resolved_at: args.resolved_outcome === null ? null : '2026-06-01',
|
||||
resolved_outcome: args.resolved_outcome,
|
||||
resolved_value: null,
|
||||
resolved_unit: null,
|
||||
resolved_source: null,
|
||||
resolved_outcome_label: null,
|
||||
resolved_by: null,
|
||||
superseded_by: null,
|
||||
embedded_at: null,
|
||||
created_at: '2026-01-15',
|
||||
updated_at: '2026-01-15',
|
||||
} as unknown as Take;
|
||||
}
|
||||
|
||||
describe('computeFounderScorecard — JSON envelope (R5)', () => {
|
||||
test('empty inputs → valid empty rollup, schema_version: 1, no NaN (G2)', () => {
|
||||
const sc = computeFounderScorecard({
|
||||
entitySlug: 'companies/empty',
|
||||
windowSince: '2025-05-17',
|
||||
windowUntil: '2026-05-17',
|
||||
points: [],
|
||||
takes: [],
|
||||
});
|
||||
expect(sc.schema_version).toBe(1);
|
||||
expect(sc.entity_slug).toBe('companies/empty');
|
||||
expect(sc.window.since).toBe('2025-05-17');
|
||||
expect(sc.window.until).toBe('2026-05-17');
|
||||
expect(sc.claim_accuracy.predicted).toBe(0);
|
||||
expect(sc.claim_accuracy.accurate).toBe(0);
|
||||
expect(sc.claim_accuracy.pct).toBeNull();
|
||||
expect(sc.consistency.score).toBeNull();
|
||||
expect(sc.consistency.metric_changes).toBe(0);
|
||||
expect(sc.consistency.typed_facts).toBe(0);
|
||||
expect(sc.growth_trajectory).toEqual([]);
|
||||
expect(sc.red_flags).toEqual([]);
|
||||
// No NaN slipped into numeric slots.
|
||||
expect(Number.isNaN(sc.claim_accuracy.predicted)).toBe(false);
|
||||
expect(Number.isNaN(sc.consistency.metric_changes)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('computeFounderScorecard — claim_accuracy', () => {
|
||||
test('3 takes, 1 accurate, 1 missed, 1 unresolved → 1/2 = 50% over RESOLVED only', () => {
|
||||
const sc = computeFounderScorecard({
|
||||
entitySlug: 'companies/acme-example',
|
||||
windowSince: null, windowUntil: null,
|
||||
points: [],
|
||||
takes: [
|
||||
take({ id: 1, claim: 'will hit $1M ARR', resolved_outcome: true }),
|
||||
take({ id: 2, claim: 'will close X', resolved_outcome: false }),
|
||||
take({ id: 3, claim: 'might do Y', resolved_outcome: null }),
|
||||
],
|
||||
});
|
||||
expect(sc.claim_accuracy.predicted).toBe(2);
|
||||
expect(sc.claim_accuracy.accurate).toBe(1);
|
||||
expect(sc.claim_accuracy.pct).toBeCloseTo(0.5, 3);
|
||||
});
|
||||
});
|
||||
|
||||
describe('computeFounderScorecard — consistency + growth_trajectory', () => {
|
||||
test('3-point stable trajectory → 0 changes, score 1.0, growth direction matches latest delta', () => {
|
||||
const sc = computeFounderScorecard({
|
||||
entitySlug: 'companies/stable',
|
||||
windowSince: null, windowUntil: null,
|
||||
points: [
|
||||
pt({ id: 1, metric: 'mrr', value: 100, date: '2026-01-15' }),
|
||||
pt({ id: 2, metric: 'mrr', value: 101, date: '2026-04-12' }),
|
||||
pt({ id: 3, metric: 'mrr', value: 102, date: '2026-07-08' }),
|
||||
],
|
||||
takes: [],
|
||||
});
|
||||
// 1% deltas are below the 5% change threshold.
|
||||
expect(sc.consistency.metric_changes).toBe(0);
|
||||
expect(sc.consistency.typed_facts).toBe(3);
|
||||
expect(sc.consistency.score).toBeCloseTo(1.0, 3);
|
||||
expect(sc.growth_trajectory.length).toBe(1);
|
||||
expect(sc.growth_trajectory[0].metric).toBe('mrr');
|
||||
// 101 → 102 = 0.99% delta < 1% threshold → 'flat'.
|
||||
expect(sc.growth_trajectory[0].direction).toBe('flat');
|
||||
});
|
||||
|
||||
test('trajectory with one big drop → 1 change, score 0.667, direction down', () => {
|
||||
const sc = computeFounderScorecard({
|
||||
entitySlug: 'companies/declining',
|
||||
windowSince: null, windowUntil: null,
|
||||
points: [
|
||||
pt({ id: 1, metric: 'mrr', value: 200000, date: '2026-04-12' }),
|
||||
pt({ id: 2, metric: 'mrr', value: 150000, date: '2026-07-08' }),
|
||||
pt({ id: 3, metric: 'mrr', value: 150500, date: '2026-09-01' }),
|
||||
],
|
||||
takes: [],
|
||||
});
|
||||
expect(sc.consistency.metric_changes).toBe(1);
|
||||
expect(sc.consistency.typed_facts).toBe(3);
|
||||
expect(sc.consistency.score).toBeCloseTo(1 - 1 / 3, 3);
|
||||
expect(sc.growth_trajectory[0].direction).toBe('flat'); // last delta is tiny
|
||||
});
|
||||
|
||||
test('multiple metrics: each gets its own growth entry, alphabetically ordered', () => {
|
||||
const sc = computeFounderScorecard({
|
||||
entitySlug: 'companies/multi',
|
||||
windowSince: null, windowUntil: null,
|
||||
points: [
|
||||
pt({ id: 1, metric: 'mrr', value: 100, date: '2026-01-15' }),
|
||||
pt({ id: 2, metric: 'arr', value: 1200, date: '2026-01-15' }),
|
||||
pt({ id: 3, metric: 'mrr', value: 130, date: '2026-04-12' }),
|
||||
pt({ id: 4, metric: 'arr', value: 1500, date: '2026-04-12' }),
|
||||
],
|
||||
takes: [],
|
||||
});
|
||||
expect(sc.growth_trajectory.map(g => g.metric)).toEqual(['arr', 'mrr']);
|
||||
expect(sc.growth_trajectory[0].direction).toBe('up'); // arr +25%
|
||||
expect(sc.growth_trajectory[1].direction).toBe('up'); // mrr +30%
|
||||
});
|
||||
});
|
||||
|
||||
describe('computeFounderScorecard — red_flags', () => {
|
||||
test('regression fires a red flag', () => {
|
||||
const sc = computeFounderScorecard({
|
||||
entitySlug: 'companies/regression',
|
||||
windowSince: null, windowUntil: null,
|
||||
points: [
|
||||
pt({ id: 1, metric: 'mrr', value: 200000, date: '2026-04-12' }),
|
||||
pt({ id: 2, metric: 'mrr', value: 150000, date: '2026-07-08' }),
|
||||
],
|
||||
takes: [],
|
||||
});
|
||||
const reg = sc.red_flags.find(f => f.kind === 'regression');
|
||||
expect(reg).toBeDefined();
|
||||
expect(reg!.metric).toBe('mrr');
|
||||
expect(reg!.text).toContain('25.0%');
|
||||
});
|
||||
|
||||
test('missed predictions surface as red flags', () => {
|
||||
const sc = computeFounderScorecard({
|
||||
entitySlug: 'companies/missed',
|
||||
windowSince: null, windowUntil: null,
|
||||
points: [],
|
||||
takes: [
|
||||
take({ id: 1, claim: 'predicted X by June, did not hit it', resolved_outcome: false }),
|
||||
],
|
||||
});
|
||||
const missed = sc.red_flags.find(f => f.kind === 'missed_prediction');
|
||||
expect(missed).toBeDefined();
|
||||
expect(missed!.text).toContain('did not hit it');
|
||||
});
|
||||
|
||||
test('high drift score (>=0.5) fires a narrative_drift flag', () => {
|
||||
function v(i: number): Float32Array {
|
||||
const a = new Float32Array(8);
|
||||
a[i % 8] = 1.0;
|
||||
return a;
|
||||
}
|
||||
const sc = computeFounderScorecard({
|
||||
entitySlug: 'companies/drift',
|
||||
windowSince: null, windowUntil: null,
|
||||
points: [
|
||||
pt({ id: 1, metric: 'mrr', value: 100, date: '2026-01-15', emb: v(0) }),
|
||||
pt({ id: 2, metric: 'mrr', value: 101, date: '2026-04-12', emb: v(3) }),
|
||||
pt({ id: 3, metric: 'mrr', value: 102, date: '2026-07-08', emb: v(6) }),
|
||||
],
|
||||
takes: [],
|
||||
});
|
||||
const drift = sc.red_flags.find(f => f.kind === 'narrative_drift');
|
||||
expect(drift).toBeDefined();
|
||||
});
|
||||
|
||||
test('clean trajectory + accurate takes + low drift = zero red flags', () => {
|
||||
function v(): Float32Array {
|
||||
const a = new Float32Array(8);
|
||||
a[0] = 1.0;
|
||||
return a;
|
||||
}
|
||||
const sc = computeFounderScorecard({
|
||||
entitySlug: 'companies/clean',
|
||||
windowSince: null, windowUntil: null,
|
||||
points: [
|
||||
pt({ id: 1, metric: 'mrr', value: 100, date: '2026-01-15', emb: v() }),
|
||||
pt({ id: 2, metric: 'mrr', value: 110, date: '2026-04-12', emb: v() }),
|
||||
pt({ id: 3, metric: 'mrr', value: 120, date: '2026-07-08', emb: v() }),
|
||||
],
|
||||
takes: [
|
||||
take({ id: 1, claim: 'accurate prediction', resolved_outcome: true }),
|
||||
],
|
||||
});
|
||||
expect(sc.red_flags).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -784,6 +784,108 @@ describe('migrate runner v66 — partial index materialized on PGLite', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
// v0.35.4 — migration v67 (facts_typed_claim_columns)
|
||||
// Adds four optional typed-claim columns to `facts` + a partial index
|
||||
// keyed on (entity_slug, claim_metric, valid_from) WHERE claim_metric IS NOT NULL.
|
||||
// All fields nullable; the migration is metadata-only on both engines
|
||||
// because no DEFAULT is set and the partial index covers zero rows until
|
||||
// extraction emits typed fields.
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
|
||||
describe('migrate v67 — facts_typed_claim_columns', () => {
|
||||
const v67 = MIGRATIONS.find(m => m.version === 67);
|
||||
|
||||
test('v67 exists and uses an inline sql field (no handler needed)', () => {
|
||||
expect(v67).toBeDefined();
|
||||
expect(v67!.name).toBe('facts_typed_claim_columns');
|
||||
expect(v67!.idempotent).toBe(true);
|
||||
expect(typeof v67!.sql).toBe('string');
|
||||
expect((v67!.sql as string).length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
test('v67 sql adds all four typed-claim columns', () => {
|
||||
const sql = v67!.sql as string;
|
||||
expect(sql).toContain('ADD COLUMN IF NOT EXISTS claim_metric');
|
||||
expect(sql).toContain('ADD COLUMN IF NOT EXISTS claim_value');
|
||||
expect(sql).toContain('ADD COLUMN IF NOT EXISTS claim_unit');
|
||||
expect(sql).toContain('ADD COLUMN IF NOT EXISTS claim_period');
|
||||
// DOUBLE PRECISION is the numeric type for claim_value (per plan D-CDX).
|
||||
expect(sql).toContain('DOUBLE PRECISION');
|
||||
});
|
||||
|
||||
test('v67 creates partial index on (entity_slug, claim_metric, valid_from)', () => {
|
||||
const sql = v67!.sql as string;
|
||||
expect(sql).toContain('CREATE INDEX IF NOT EXISTS facts_typed_claim_idx');
|
||||
expect(sql).toContain('ON facts (entity_slug, claim_metric, valid_from)');
|
||||
expect(sql).toContain('WHERE claim_metric IS NOT NULL');
|
||||
});
|
||||
});
|
||||
|
||||
describe('migrate runner v67 — typed-claim columns materialized on PGLite', () => {
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
test('v67 added claim_metric, claim_value, claim_unit, claim_period columns to facts', async () => {
|
||||
const rows = await (engine as any).db.query(
|
||||
`SELECT column_name, data_type FROM information_schema.columns
|
||||
WHERE table_name = 'facts'
|
||||
AND column_name IN ('claim_metric', 'claim_value', 'claim_unit', 'claim_period')
|
||||
ORDER BY column_name`,
|
||||
);
|
||||
const names = rows.rows.map((r: any) => r.column_name).sort();
|
||||
expect(names).toEqual(['claim_metric', 'claim_period', 'claim_unit', 'claim_value']);
|
||||
const byName: Record<string, string> = Object.fromEntries(
|
||||
rows.rows.map((r: any) => [r.column_name, r.data_type]),
|
||||
);
|
||||
// claim_value is DOUBLE PRECISION; the others are TEXT.
|
||||
expect(byName['claim_value']).toBe('double precision');
|
||||
expect(byName['claim_metric']).toBe('text');
|
||||
expect(byName['claim_unit']).toBe('text');
|
||||
expect(byName['claim_period']).toBe('text');
|
||||
});
|
||||
|
||||
test('v67 created facts_typed_claim_idx partial index on PGLite', async () => {
|
||||
const rows = await (engine as any).db.query(
|
||||
`SELECT indexname, indexdef FROM pg_indexes WHERE indexname = 'facts_typed_claim_idx'`,
|
||||
);
|
||||
expect(rows.rows.length).toBe(1);
|
||||
// The partial-predicate appears in the materialized index definition.
|
||||
expect(rows.rows[0].indexdef).toContain('claim_metric');
|
||||
});
|
||||
|
||||
test('v67 columns are nullable — existing facts persist with NULL typed fields (backward compat)', async () => {
|
||||
// Insert a fact via raw SQL with no typed-claim values; assert the
|
||||
// four columns remain NULL. The cycle path (extract_facts) hits this
|
||||
// backward-compat surface every time it processes a fence without
|
||||
// metric assertions.
|
||||
const db = (engine as any).db;
|
||||
await db.exec(`INSERT INTO sources (id, name) VALUES ('v67-test', 'v67-test') ON CONFLICT DO NOTHING`);
|
||||
await db.exec(
|
||||
`INSERT INTO facts (source_id, entity_slug, fact, source, valid_from)
|
||||
VALUES ('v67-test', 'v67/example', 'plain non-typed claim', 'test', now())`,
|
||||
);
|
||||
const row = await db.query(
|
||||
`SELECT claim_metric, claim_value, claim_unit, claim_period
|
||||
FROM facts WHERE source_id = 'v67-test' AND entity_slug = 'v67/example'`,
|
||||
);
|
||||
expect(row.rows.length).toBe(1);
|
||||
expect(row.rows[0].claim_metric).toBeNull();
|
||||
expect(row.rows[0].claim_value).toBeNull();
|
||||
expect(row.rows[0].claim_unit).toBeNull();
|
||||
expect(row.rows[0].claim_period).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('migrate: v8 (links_dedup) regression — must be fast on 1K duplicate rows', () => {
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
|
||||
@@ -0,0 +1,185 @@
|
||||
/**
|
||||
* v0.35.4 — find_trajectory MCP op (T5) tests.
|
||||
*
|
||||
* Pins:
|
||||
* - Param validation: entity_slug required, non-empty.
|
||||
* - Visibility filter on remote=true callers (R6 / D-CDX-1).
|
||||
* - Source scoping via sourceScopeOpts (federated vs scalar).
|
||||
* - Stable JSON envelope: points + regressions + drift_score + schema_version=1 (R5).
|
||||
* - Engine result's raw Float32Array embedding is NOT serialized to wire.
|
||||
* - Empty-result graceful shape (G1).
|
||||
* - The op is registered + read-scope + not localOnly.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { operationsByName } from '../src/core/operations.ts';
|
||||
import type { OperationContext } from '../src/core/operations.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await engine.executeRaw(`DELETE FROM facts WHERE entity_slug LIKE 'optraj-%'`);
|
||||
await engine.executeRaw(`DELETE FROM sources WHERE id LIKE 'optraj-%'`);
|
||||
});
|
||||
|
||||
function unitVec(idx: number): string {
|
||||
const a = new Float32Array(1536);
|
||||
a[idx % 1536] = 1.0;
|
||||
return '[' + Array.from(a).join(',') + ']';
|
||||
}
|
||||
|
||||
async function insertTyped(args: {
|
||||
source_id?: string;
|
||||
entity_slug: string;
|
||||
metric: string;
|
||||
value: number;
|
||||
valid_from: Date;
|
||||
visibility?: 'private' | 'world';
|
||||
vecIdx?: number;
|
||||
}): Promise<void> {
|
||||
const sid = args.source_id ?? 'default';
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO sources (id, name) VALUES ($1, $1) ON CONFLICT DO NOTHING`,
|
||||
[sid],
|
||||
);
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO facts (source_id, entity_slug, fact, kind, source, valid_from,
|
||||
claim_metric, claim_value, claim_unit, claim_period,
|
||||
visibility, embedding, embedded_at)
|
||||
VALUES ($1, $2, $3, 'fact', 'test', $4::timestamptz,
|
||||
$5, $6, 'USD', 'monthly',
|
||||
$7, $8::vector, $4::timestamptz)`,
|
||||
[
|
||||
sid, args.entity_slug, `${args.metric} = ${args.value}`,
|
||||
args.valid_from.toISOString(), args.metric, args.value,
|
||||
args.visibility ?? 'private', unitVec(args.vecIdx ?? 0),
|
||||
],
|
||||
);
|
||||
}
|
||||
|
||||
function mkCtx(overrides: Partial<OperationContext> = {}): OperationContext {
|
||||
return {
|
||||
engine,
|
||||
config: {} as any,
|
||||
logger: { info: () => {}, warn: () => {}, error: () => {} } as any,
|
||||
dryRun: false,
|
||||
remote: false,
|
||||
...overrides,
|
||||
} as OperationContext;
|
||||
}
|
||||
|
||||
describe('find_trajectory MCP op — registration + shape', () => {
|
||||
test('registered with read scope, NOT localOnly', () => {
|
||||
const op = operationsByName['find_trajectory'];
|
||||
expect(op).toBeDefined();
|
||||
expect(op.scope).toBe('read');
|
||||
expect(op.localOnly).toBeUndefined();
|
||||
// Description references the v0.35.4 contract.
|
||||
expect(op.description).toContain('schema_version');
|
||||
});
|
||||
|
||||
test('throws on missing entity_slug', async () => {
|
||||
const op = operationsByName['find_trajectory'];
|
||||
await expect(op.handler(mkCtx(), {})).rejects.toThrow(/entity_slug/);
|
||||
await expect(op.handler(mkCtx(), { entity_slug: '' })).rejects.toThrow(/entity_slug/);
|
||||
await expect(op.handler(mkCtx(), { entity_slug: ' ' })).rejects.toThrow(/entity_slug/);
|
||||
});
|
||||
|
||||
test('returns stable JSON shape with schema_version: 1', async () => {
|
||||
await insertTyped({ entity_slug: 'optraj-shape', metric: 'mrr', value: 50000, valid_from: new Date('2026-01-15') });
|
||||
const op = operationsByName['find_trajectory'];
|
||||
const result = await op.handler(mkCtx(), { entity_slug: 'optraj-shape' }) as any;
|
||||
expect(result).toHaveProperty('points');
|
||||
expect(result).toHaveProperty('regressions');
|
||||
expect(result).toHaveProperty('drift_score');
|
||||
expect(result.schema_version).toBe(1);
|
||||
// Embedding NOT serialized to the wire.
|
||||
expect(result.points[0]).not.toHaveProperty('embedding');
|
||||
// valid_from is YYYY-MM-DD string.
|
||||
expect(result.points[0].valid_from).toMatch(/^\d{4}-\d{2}-\d{2}$/);
|
||||
});
|
||||
|
||||
test('unknown entity returns graceful empty shape (G1)', async () => {
|
||||
const op = operationsByName['find_trajectory'];
|
||||
const result = await op.handler(mkCtx(), { entity_slug: 'optraj-does-not-exist' }) as any;
|
||||
expect(result.points).toEqual([]);
|
||||
expect(result.regressions).toEqual([]);
|
||||
expect(result.drift_score).toBeNull();
|
||||
expect(result.schema_version).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('find_trajectory MCP op — visibility filter (R6 / D-CDX-1)', () => {
|
||||
test('remote=true sees only world-visibility points', async () => {
|
||||
await insertTyped({ entity_slug: 'optraj-vis', metric: 'mrr', value: 50000, visibility: 'private', valid_from: new Date('2026-01-15') });
|
||||
await insertTyped({ entity_slug: 'optraj-vis', metric: 'mrr', value: 99999, visibility: 'world', valid_from: new Date('2026-04-12') });
|
||||
|
||||
const op = operationsByName['find_trajectory'];
|
||||
const local = await op.handler(mkCtx({ remote: false }), { entity_slug: 'optraj-vis' }) as any;
|
||||
expect(local.points.length).toBe(2);
|
||||
|
||||
const remote = await op.handler(mkCtx({ remote: true }), { entity_slug: 'optraj-vis' }) as any;
|
||||
expect(remote.points.length).toBe(1);
|
||||
expect(remote.points[0].value).toBe(99999);
|
||||
});
|
||||
});
|
||||
|
||||
describe('find_trajectory MCP op — source scoping (D-CDX-6)', () => {
|
||||
test('federated sourceIds from auth.allowedSources narrows scope', async () => {
|
||||
await insertTyped({ source_id: 'optraj-A', entity_slug: 'optraj-fed', metric: 'mrr', value: 1, valid_from: new Date('2026-01-15') });
|
||||
await insertTyped({ source_id: 'optraj-B', entity_slug: 'optraj-fed', metric: 'mrr', value: 2, valid_from: new Date('2026-04-12') });
|
||||
await insertTyped({ source_id: 'optraj-C', entity_slug: 'optraj-fed', metric: 'mrr', value: 3, valid_from: new Date('2026-07-08') });
|
||||
|
||||
const op = operationsByName['find_trajectory'];
|
||||
const ctx = mkCtx({
|
||||
auth: { allowedSources: ['optraj-A', 'optraj-B'] } as any,
|
||||
});
|
||||
const result = await op.handler(ctx, { entity_slug: 'optraj-fed' }) as any;
|
||||
expect(result.points.length).toBe(2);
|
||||
expect(result.points.map((p: any) => p.value)).toEqual([1, 2]);
|
||||
});
|
||||
|
||||
test('scalar ctx.sourceId narrows to that single source', async () => {
|
||||
await insertTyped({ source_id: 'optraj-X', entity_slug: 'optraj-scalar', metric: 'mrr', value: 100, valid_from: new Date('2026-01-15') });
|
||||
await insertTyped({ source_id: 'optraj-Y', entity_slug: 'optraj-scalar', metric: 'mrr', value: 200, valid_from: new Date('2026-01-15') });
|
||||
|
||||
const op = operationsByName['find_trajectory'];
|
||||
const ctx = mkCtx({ sourceId: 'optraj-X' });
|
||||
const result = await op.handler(ctx, { entity_slug: 'optraj-scalar' }) as any;
|
||||
expect(result.points.length).toBe(1);
|
||||
expect(result.points[0].value).toBe(100);
|
||||
});
|
||||
});
|
||||
|
||||
describe('find_trajectory MCP op — regression + drift surface', () => {
|
||||
test('regressions populate when newer value drops >= 10% (D-ENG-2 default)', async () => {
|
||||
await insertTyped({ entity_slug: 'optraj-reg', metric: 'mrr', value: 200000, valid_from: new Date('2026-04-12'), vecIdx: 0 });
|
||||
await insertTyped({ entity_slug: 'optraj-reg', metric: 'mrr', value: 150000, valid_from: new Date('2026-07-08'), vecIdx: 0 });
|
||||
|
||||
const op = operationsByName['find_trajectory'];
|
||||
const result = await op.handler(mkCtx(), { entity_slug: 'optraj-reg' }) as any;
|
||||
expect(result.regressions.length).toBe(1);
|
||||
expect(result.regressions[0].metric).toBe('mrr');
|
||||
expect(result.regressions[0].delta_pct).toBeCloseTo(-0.25, 3);
|
||||
});
|
||||
|
||||
test('drift_score returns null with <3 embedded points (G3)', async () => {
|
||||
await insertTyped({ entity_slug: 'optraj-drift', metric: 'mrr', value: 1, valid_from: new Date('2026-01-15') });
|
||||
await insertTyped({ entity_slug: 'optraj-drift', metric: 'mrr', value: 2, valid_from: new Date('2026-04-12') });
|
||||
|
||||
const op = operationsByName['find_trajectory'];
|
||||
const result = await op.handler(mkCtx(), { entity_slug: 'optraj-drift' }) as any;
|
||||
expect(result.drift_score).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -640,6 +640,14 @@ const COLUMN_EXEMPTIONS = new Set<string>([
|
||||
'pages.source_path',
|
||||
'content_chunks.edges_backfilled_at',
|
||||
'query_cache.knobs_hash',
|
||||
// v0.35.6 (migration v67) — typed-claim columns + facts_typed_claim_idx
|
||||
// partial index are co-defined in the same migration, so the schema-blob
|
||||
// forward-reference path isn't tripped. Bootstrap is only required when an
|
||||
// index in PGLITE_SCHEMA_SQL references a column added by a later migration.
|
||||
'facts.claim_metric',
|
||||
'facts.claim_value',
|
||||
'facts.claim_unit',
|
||||
'facts.claim_period',
|
||||
]);
|
||||
|
||||
test('every ALTER TABLE ADD COLUMN in MIGRATIONS is covered by applyForwardReferenceBootstrap (column-only class)', async () => {
|
||||
|
||||
Reference in New Issue
Block a user