From 099d9a8f5505f1d64e4f4f784f8429cc2d47c18f Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Sun, 7 Jun 2026 22:05:31 -0700 Subject: [PATCH] =?UTF-8?q?v0.42.34.0=20feat(search):=20typed-edge=20relat?= =?UTF-8?q?ional=20retrieval=20=E2=80=94=20relationship=20questions=20get?= =?UTF-8?q?=20relationship=20answers=20(#1959)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(search): deterministic relational-query parser Pure, ReDoS-bounded parser that detects relationship queries ("who invested in X", "who at X works on Y", "who introduced me to X", "what connects A and B") and maps them to typed edges. Schema-pack-extensible vocab with subset validation against the link types ingest produces, so query-side and ingest-side relation vocabularies can't drift. No-match / pronoun-seed / adjacency guards keep it precision-first (a candidate only; the arm fires only when a real seed also resolves). Co-Authored-By: Claude Opus 4.8 (1M context) * feat(engine): relationalFanout typed-edge fan-out (both engines) Generalizes traversePaths to a SEED ARRAY, aggregating to ranked NODES (shortest hop, edge-richness count, via-link-types, shortest connecting path, canonical chunk id) instead of edges. Within-source traversal (never crosses a source boundary even across a federated scope), link_source= 'mentions' excluded by default, deleted_at filtered at seed + every neighbor + every node, bounded depth (<=3) + candidate cap. Adds RelationalFanoutRow / RelationalFanoutOpts + the relational SearchResult/SearchOpts fields to types. Lands in lockstep in postgres + pglite engines, pinned by a DATABASE_URL- gated parity block in engine-parity.test.ts; a PGLite unit test exercises the SQL (typed-edge filter, mentions exclusion, deleted_at, canonical chunk, multi-seed connects, determinism) in default CI. Co-Authored-By: Claude Opus 4.8 (1M context) * feat(search): relational recall arm + federation key hardening Wires edge-derived candidates into bare hybridSearch as a FOURTH RRF arm (relational-recall.ts): parse the original query -> scope-aware, confidence-gated seed resolution (drops fallback_slugify; never traverses from a guess) -> relationalFanout -> batch-hydrate, reinforcing each page's REAL canonical chunk (page-level key for chunkless entity pages) -> --explain attribution + fail-open audit row. Text-only (no-op in image mode); pure no-op for non-relational queries; rides every downstream stage (cosine, post-fusion boosts, dedup, reranker, autocut, token budget). Mode wiring: relationalRetrieval + relational_retrieval_depth knobs (conservative off; balanced/tokenmax on; depth 2), per-call thread-through in both bare + cached paths, KNOBS_HASH_VERSION 9->10 (rel=/reld=), config keys, modes-dashboard descriptions, and a `relational` param on the query op. Federation hardening (structural, engine-wide): the RRF/dedup key now carries source_id via a shared rrfKey() (fixes a latent cross-source collapse where same-slug pages in different sources merged), and the query cache scopes by a canonical source-set key (cacheScopeKey) so a federated read can't be served a single-source row. Co-Authored-By: Claude Opus 4.8 (1M context) * feat(eval): relational benchmark + recall@k harness metrics NamedThingBench harness gains recall@k / recall@10 (the relational headline metric) on QuestionResult + FamilyReport, plus typed seed/linkTypes/kind on NamedThingQuestion so the graph-relationship family is machine-checkable. Adds the relational benchmark corpus (test/fixtures/retrieval-quality/ relational/): a small entity graph whose answers are LEXICALLY UNRECOVERABLE — every page body is generic and never names the entity it relates to, so only the typed edge connects query to answer. corpus.ts is the canonical source for both the seed loader and the 38-question gold set; relational.jsonl is generated from it (a drift test pins them equal). Co-Authored-By: Claude Opus 4.8 (1M context) * feat(eval): relational A/B proof + arm fires on all retrieval paths Fixes the integration bug the eval caught: the relational arm was only injected on the main RRF path, so it silently did nothing whenever vector was unavailable — no embedding provider configured (the default in many deployments) OR embed failure. The arm is now built once and fused via RRF on ALL THREE hybridSearch return paths (no-embedding-provider, embed-failed keyword fallback, main path). Without this it would have been dead in exactly the setups that most need it. Adds `gbrain eval retrieval-quality --ab-relational`: runs the gold set twice (arm off vs on) in a fixed mode and reports the graph-relationship recall@10 lift + Hit@3 + latency add. The CI A/B test pins the headline result — recall@10 jumps from <25% (lexically unrecoverable) to >75% with the arm on — and a non-relational query returns identical results arm-on vs off (the no-op / no-regression gate). Co-Authored-By: Claude Opus 4.8 (1M context) * chore: bump version and changelog (v0.42.34.0) Relational retrieval feature: typed-edge recall arm + federation key hardening. Also updates the KNOBS_HASH_VERSION 9→10 assertions across the remaining search test files (the bump invalidates relational-off cache rows). Co-Authored-By: Claude Opus 4.8 (1M context) * docs: document typed-edge relational retrieval (v0.42.34.0) CLAUDE.md Search Mode: add relationalRetrieval to the knob table, the knobs_hash v=9→10 note, and a relational-retrieval summary. RETRIEVAL.md: add the relational recall arm to the pipeline diagram. Regenerate llms bundles (build:llms). Co-Authored-By: Claude Opus 4.8 (1M context) * fix(test): size relational fixtures to the actual embedding-column dim CI shard runs with the ZeroEntropy gateway default (1280-d), but the relational test fixtures hardcoded 1536-d embeddings, so chunk inserts were rejected with "expected 1280 dimensions, not 1536" (CheckExpectedDim) — the `test (6)` shard + `test-status` failures. The column width tracks the configured gateway default and can shift with shard order, so fixtures now probe `content_chunks.embedding`'s actual `atttypmod` after initSchema and size embeddings to it (the pglite-engine.test.ts pattern), via a shared `probeEmbeddingDim` helper. Verified passing at a forced 1280-d column. Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 17 ++ CLAUDE.md | 14 + VERSION | 2 +- docs/architecture/RETRIEVAL.md | 1 + llms-full.txt | 14 + package.json | 2 +- src/commands/eval-retrieval-quality.ts | 45 +++- src/commands/search.ts | 3 + src/core/engine.ts | 26 +- src/core/operations.ts | 7 + src/core/pglite-engine.ts | 84 ++++++ src/core/postgres-engine.ts | 82 ++++++ src/core/search/hybrid.ts | 122 ++++++++- src/core/search/mode.ts | 56 +++- src/core/search/relational-intent.ts | 243 ++++++++++++++++++ src/core/search/relational-recall.ts | 232 +++++++++++++++++ src/core/types.ts | 59 +++++ src/eval/retrieval-quality/harness.ts | 29 ++- test/cache-scope-key.test.ts | 42 +++ test/cross-modal-phase1.test.ts | 5 +- test/e2e/engine-parity.test.ts | 68 +++++ .../retrieval-quality/relational/corpus.ts | 195 ++++++++++++++ .../relational/relational.jsonl | 42 +++ test/relational-ab.test.ts | 73 ++++++ test/relational-fanout.test.ts | 115 +++++++++ test/relational-intent.test.ts | 136 ++++++++++ test/relational-recall.test.ts | 72 ++++++ test/rrf-source-key.test.ts | 70 +++++ test/search-alias-resolved-boost.test.ts | 4 +- test/search-mode.test.ts | 58 ++++- test/search/knobs-hash-reranker.test.ts | 5 +- 31 files changed, 1895 insertions(+), 28 deletions(-) create mode 100644 src/core/search/relational-intent.ts create mode 100644 src/core/search/relational-recall.ts create mode 100644 test/cache-scope-key.test.ts create mode 100644 test/fixtures/retrieval-quality/relational/corpus.ts create mode 100644 test/fixtures/retrieval-quality/relational/relational.jsonl create mode 100644 test/relational-ab.test.ts create mode 100644 test/relational-fanout.test.ts create mode 100644 test/relational-intent.test.ts create mode 100644 test/relational-recall.test.ts create mode 100644 test/rrf-source-key.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index ecfe09b0e..e3f1ecaa0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,23 @@ All notable changes to GBrain will be documented in this file. +## [0.42.34.0] - 2026-06-07 + +**Relationship questions now get relationship answers.** Ask "who invested in widget-co", "who introduced me to alice-example", or "what connects fund-a and fund-b" and gbrain resolves the named entity and walks its typed-edge graph (`invested_in`, `works_at`, `founded`, `attended`, `advises`, …) to surface the answer — even when no single page mentions both sides. Until now the graph only re-ranked results that keyword/vector search had already found; a relationship that lived purely in the edges (an investor whose page never names the company) was invisible. It now enters retrieval as a first-class candidate. + +This is on by default in the `balanced` and `tokenmax` search modes, a pure no-op for non-relational questions and for brains with no typed edges, and off in `conservative`. On a benchmark of relationship queries whose answers are unreachable by content similarity, recall@10 goes from near-zero to over 75%. The traversal is deterministic (same query + brain → same answer), stays within a single source (it never crosses a mounted-brain boundary), excludes noisy body-text "mentions" edges by default, and is depth- and fan-out-bounded so a popular hub entity can't blow up a query. + +### Added +- **Typed-edge relational retrieval** (`search.relational_retrieval`, on for balanced/tokenmax). Relational questions resolve their seed entity and traverse the typed-edge graph, injecting edge-derived answers as a fourth fusion arm alongside keyword + vector. Relation vocabulary is schema-pack-extensible: a pack that defines its own link types can declare the query phrases that retrieve them. The `query` operation gains a `relational` flag (omit for the smart default; pass `false` to force lexical/vector-only). Results carry `--explain` attribution ("surfaced via invested_in from widget-co") and, for "what connects A and B", the connecting path. +- **`relationalFanout` engine method** (PGLite + Postgres, in lockstep). Seed-array typed-edge fan-out aggregating to ranked nodes (shortest hop, edge richness, connecting path, canonical chunk), source-scoped and deterministic. +- **`gbrain eval retrieval-quality --ab-relational`**: A/Bs the arm off vs on over a question set and reports the recall@10 lift + latency. The retrieval-quality harness gains recall@k / recall@10 metrics. + +### Fixed +- **Cross-source result collapse.** The search fusion/dedup key now carries `source_id`, so two pages that share a slug across mounted brains no longer merge into one result. The semantic query cache is likewise scoped per source-set, so a federated search can't be served a single-source cached result. + +### To take advantage of v0.42.34.0 +- Just ask relationship questions in natural language — the arm is on by default in balanced/tokenmax. To turn it off: `gbrain config set search.relational_retrieval false`. +- One-time cache note: this release advances the search-cache key version (a relational-on result must not be served to a relational-off lookup), so the first query after upgrade re-runs instead of hitting a stale cache row. No action needed; it self-heals on first use. ## [0.42.33.0] - 2026-06-07 **`gbrain sync` will never delete a repo it didn't create.** If a source was registered with a `remote_url` but its `local_path` pointed at a working tree you manage yourself (not a gbrain-managed clone), a failed or degraded code sync could remove that directory and re-clone over it. Sync now re-clones **only** clones gbrain actually created — identified by an ownership marker, or by gbrain's own clone location for clones made before this release. Anything else, including your live working tree, is treated as read-only: indexed, never deleted. On an unowned path, sync aborts loudly **before touching the filesystem** and tells you how to fix the source registration. Thanks to @zaqwery for the report. diff --git a/CLAUDE.md b/CLAUDE.md index cbdb58c52..6645fe69a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -149,6 +149,7 @@ project resolves through `src/core/search/mode.ts`. | `intentWeighting` | true | true | true | | `tokenBudget` | **4000** | **12000** | **off** | | `expansion` (LLM multi-query) | false | false | **true** | +| `relationalRetrieval` | false | **true** | **true** | | `searchLimit` default | 10 | 25 | 50 | **Cost anchors (downstream agent input cost — gbrain itself is rounding error).** @@ -207,6 +208,19 @@ written against `embedding` (1536d OpenAI). Existing v=2 rows become unreachable on first re-query (one-time miss spike on upgrade); `mode.ts:KNOBS_HASH_VERSION` is the single source of truth. +**v0.42.34.0 knobs_hash v=9 → v=10.** Folds the `relationalRetrieval` knob + +depth into the cache key so a relational-on result set can't be served to a +relational-off lookup (same contamination class as graph_signals). One-time +miss spike on upgrade. + +**Relational retrieval (v0.42.34.0).** `relationalRetrieval` (on for +balanced/tokenmax) adds a fourth recall arm: a relational query ("who invested +in X", "what connects A and B") resolves its seed entity and walks the typed-edge +graph (`src/core/search/relational-recall.ts` + `relational-intent.ts`, +`engine.relationalFanout`), injecting edge-derived answers into RRF. Within-source, +deterministic, mentions-excluded by default, pure no-op for non-relational queries. +The `query` op's `relational` flag forces it on/off per call. + **Three CLI surfaces:** gbrain search modes # what is running, with per-knob attribution diff --git a/VERSION b/VERSION index 88025242a..5f3886e1f 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.42.33.0 \ No newline at end of file +0.42.34.0 diff --git a/docs/architecture/RETRIEVAL.md b/docs/architecture/RETRIEVAL.md index 82d0e2a98..4b2550b8c 100644 --- a/docs/architecture/RETRIEVAL.md +++ b/docs/architecture/RETRIEVAL.md @@ -123,6 +123,7 @@ expansion (if enabled) hybrid search: ├── vector (HNSW on chunk embeddings) ├── keyword (BM25 via tsvector) + ├── relational (v0.42.34.0: typed-edge recall arm — relational queries only) ├── source-aware re-rank (CASE in SQL) └── RRF fusion → top 30 │ diff --git a/llms-full.txt b/llms-full.txt index 018b531f1..8b3579bdc 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -297,6 +297,7 @@ project resolves through `src/core/search/mode.ts`. | `intentWeighting` | true | true | true | | `tokenBudget` | **4000** | **12000** | **off** | | `expansion` (LLM multi-query) | false | false | **true** | +| `relationalRetrieval` | false | **true** | **true** | | `searchLimit` default | 10 | 25 | 50 | **Cost anchors (downstream agent input cost — gbrain itself is rounding error).** @@ -355,6 +356,19 @@ written against `embedding` (1536d OpenAI). Existing v=2 rows become unreachable on first re-query (one-time miss spike on upgrade); `mode.ts:KNOBS_HASH_VERSION` is the single source of truth. +**v0.42.34.0 knobs_hash v=9 → v=10.** Folds the `relationalRetrieval` knob + +depth into the cache key so a relational-on result set can't be served to a +relational-off lookup (same contamination class as graph_signals). One-time +miss spike on upgrade. + +**Relational retrieval (v0.42.34.0).** `relationalRetrieval` (on for +balanced/tokenmax) adds a fourth recall arm: a relational query ("who invested +in X", "what connects A and B") resolves its seed entity and walks the typed-edge +graph (`src/core/search/relational-recall.ts` + `relational-intent.ts`, +`engine.relationalFanout`), injecting edge-derived answers into RRF. Within-source, +deterministic, mentions-excluded by default, pure no-op for non-relational queries. +The `query` op's `relational` flag forces it on/off per call. + **Three CLI surfaces:** gbrain search modes # what is running, with per-knob attribution diff --git a/package.json b/package.json index b962e4d6a..7d6f87318 100644 --- a/package.json +++ b/package.json @@ -143,5 +143,5 @@ "bun": ">=1.3.10" }, "license": "MIT", - "version": "0.42.33.0" + "version": "0.42.34.0" } diff --git a/src/commands/eval-retrieval-quality.ts b/src/commands/eval-retrieval-quality.ts index 175a1e7a3..7d48dc10f 100644 --- a/src/commands/eval-retrieval-quality.ts +++ b/src/commands/eval-retrieval-quality.ts @@ -20,12 +20,13 @@ import { export async function runEvalRetrievalQuality(engine: BrainEngine, args: string[]): Promise { const json = args.includes('--json'); + const abRelational = args.includes('--ab-relational'); const sourceIdx = args.indexOf('--source'); const sourceId = sourceIdx >= 0 ? args[sourceIdx + 1] : undefined; const fixture = args.find(a => !a.startsWith('--') && a !== sourceId); if (!fixture) { - console.error('Usage: gbrain eval retrieval-quality [--json] [--source ]'); + console.error('Usage: gbrain eval retrieval-quality [--json] [--source ] [--ab-relational]'); process.exit(2); } @@ -47,6 +48,48 @@ export async function runEvalRetrievalQuality(engine: BrainEngine, args: string[ return results.map(r => r.slug); }; + // v0.43 — A/B the relational recall arm (off vs on) over the same questions + // in a fixed mode (expansion off, skipCache-equivalent via bare hybridSearch) + // so the delta is purely the arm. Headline: graph-relationship recall@10 lift. + if (abRelational) { + const mk = (relationalRetrieval: boolean): SearchFn => async (q) => { + const results = await hybridSearch(engine, q, { + limit: 10, relationalRetrieval, expansion: false, + ...(sourceId ? { sourceId } : {}), + }); + return results.map(r => r.slug); + }; + const t0 = Date.now(); + const off = await runRetrievalQuality(questions, mk(false)); + const tMid = Date.now(); + const on = await runRetrievalQuality(questions, mk(true)); + const tEnd = Date.now(); + const fam = (r: typeof off, f: string) => r.families.find(x => x.family === f); + const rel = { off: fam(off, 'graph-relationship'), on: fam(on, 'graph-relationship') }; + const payload = { + schema_version: 1 as const, + ab: 'relational', + graph_relationship: { + n: rel.on?.n ?? 0, + recall_at_10: { off: rel.off?.recall_at_10 ?? 0, on: rel.on?.recall_at_10 ?? 0, + delta: (rel.on?.recall_at_10 ?? 0) - (rel.off?.recall_at_10 ?? 0) }, + hit_at_3: { off: rel.off?.hit_at_3 ?? 0, on: rel.on?.hit_at_3 ?? 0 }, + }, + latency_ms: { off_total: tMid - t0, on_total: tEnd - tMid }, + families: { off: off.families, on: on.families }, + }; + if (json) { + console.log(JSON.stringify(payload, null, 2)); + } else { + console.log(`Relational A/B — graph-relationship n=${payload.graph_relationship.n}`); + const g = payload.graph_relationship; + console.log(` recall@10: off=${(g.recall_at_10.off * 100).toFixed(0)}% on=${(g.recall_at_10.on * 100).toFixed(0)}% Δ=+${(g.recall_at_10.delta * 100).toFixed(0)}pp`); + console.log(` Hit@3: off=${(g.hit_at_3.off * 100).toFixed(0)}% on=${(g.hit_at_3.on * 100).toFixed(0)}%`); + console.log(` latency: off=${payload.latency_ms.off_total}ms on=${payload.latency_ms.on_total}ms (arm adds ${payload.latency_ms.on_total - payload.latency_ms.off_total}ms over ${payload.graph_relationship.n} queries)`); + } + process.exit(0); + } + const report = await runRetrievalQuality(questions, searchFn); const gate = evaluateGate(report); diff --git a/src/commands/search.ts b/src/commands/search.ts index b2738a6b1..971010979 100644 --- a/src/commands/search.ts +++ b/src/commands/search.ts @@ -71,6 +71,9 @@ const KNOB_DESCRIPTIONS: Record = { // v0.42.3.0 autocut autocut: 'Score-discontinuity result-sizing (cuts at the rerank-score cliff; no-op without a reranker)', autocut_jump: 'Autocut sensitivity: min normalized score gap that counts as a cliff (0..1, 0.20 default)', + // v0.43 relational recall + relationalRetrieval: 'Typed-edge relational recall arm (relational queries walk the graph; no-op otherwise)', + relational_retrieval_depth: 'Max hops for relational traversal (1..3, 2 default)', }; interface SearchModesReport { diff --git a/src/core/engine.ts b/src/core/engine.ts index a75e30110..0f285d498 100644 --- a/src/core/engine.ts +++ b/src/core/engine.ts @@ -2,7 +2,7 @@ import type { Page, PageInput, PageFilters, GetPageOpts, Chunk, ChunkInput, StaleChunkRow, StalePageRow, SearchResult, SearchOpts, - Link, GraphNode, GraphPath, + Link, GraphNode, GraphPath, RelationalFanoutRow, RelationalFanoutOpts, TimelineEntry, TimelineInput, TimelineOpts, RawData, PageVersion, @@ -1197,6 +1197,30 @@ export interface BrainEngine { slug: string, opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both'; sourceId?: string; sourceIds?: string[] }, ): Promise; + /** + * Typed-edge relational fan-out for the relational recall arm (v0.43). + * + * Generalizes traversePaths to a SEED ARRAY and aggregates to ranked NODES + * (not edges): for each reached page, the shortest hop from any seed, a + * connection-richness count, the edge types it was reached by, the shortest + * connecting slug path, and the page's canonical (lowest-ordinal) chunk id. + * + * Invariants: + * - WITHIN-SOURCE: a walk never crosses a source boundary (each branch + * stays in its seed's own source), even when multiple sources are in + * scope. Cross-source edge traversal is a separate, security-reviewed + * feature. + * - `link_source='mentions'` edges are EXCLUDED by default (noisy + the + * densest fan-out source); opt in via `includeMentions`. + * - `deleted_at IS NULL` at seed, every neighbor, and every returned node. + * - Deterministic ordering: hop ASC, edge_count DESC, source_id, slug. + * Must move in lockstep with the PGLite implementation + * (test/e2e/engine-parity.test.ts). + */ + relationalFanout( + seeds: string[], + opts?: RelationalFanoutOpts, + ): Promise; /** * For a list of slugs, return how many inbound links each has. * Used by hybrid search backlink boost. Single SQL query, not N+1. diff --git a/src/core/operations.ts b/src/core/operations.ts index 56b0eb801..8739a2232 100644 --- a/src/core/operations.ts +++ b/src/core/operations.ts @@ -1431,6 +1431,11 @@ const query: Operation = { " TRUE is redundant in default mode (it's already on); it only matters to override a brain whose config turned autocut off.\n" + "Safe by construction: never returns empty when there are matches, only applies to the first page (omit when paginating), and is a no-op when no reranker scored the results (so it can't cut on an untrustworthy signal). Distinct from `adaptive_return`: autocut cuts on the score cliff; adaptive_return caps by question intent. Leave both unset for the smart default.", }, + relational: { + type: 'boolean', + description: + "v0.43 — relational recall arm. SMART DEFAULT (on in balanced/tokenmax). When the question is about a RELATIONSHIP ('who invested in widget-co', 'who introduced me to alice', 'what connects fund-a and fund-b'), the brain resolves the named entity and walks its typed-edge graph (invested_in, works_at, founded, …), surfacing the answer even when no passage mentions both sides. Pure no-op for non-relational questions. Pass FALSE to force lexical/vector-only retrieval (e.g. debugging why a graph answer appeared). You almost never set this.", + }, }, handler: async (ctx, p) => { const startedAt = Date.now(); @@ -1528,6 +1533,8 @@ const query: Operation = { // v0.42.3.0 — autocut ceiling override. Omitted = smart default (ON in // reranked modes). `false` forces the full top-K. autocut: typeof p.autocut === 'boolean' ? (p.autocut as boolean) : undefined, + // v0.43 — relational recall override. Omitted = smart default (mode bundle). + relationalRetrieval: typeof p.relational === 'boolean' ? (p.relational as boolean) : undefined, }); const latency_ms = Date.now() - startedAt; diff --git a/src/core/pglite-engine.ts b/src/core/pglite-engine.ts index 9df57522a..ca727cd5f 100644 --- a/src/core/pglite-engine.ts +++ b/src/core/pglite-engine.ts @@ -2859,6 +2859,90 @@ export class PGLiteEngine implements BrainEngine { return result; } + async relationalFanout( + seeds: string[], + opts?: import('./types.ts').RelationalFanoutOpts, + ): Promise { + if (!seeds || seeds.length === 0) return []; + const depth = Math.min(Math.max(1, opts?.depth ?? 2), 3); + const direction = opts?.direction ?? 'both'; + const limit = Math.min(Math.max(1, opts?.limit ?? 50), 200); + const types = opts?.linkTypes && opts.linkTypes.length > 0 ? opts.linkTypes : null; + + // $1=seeds, $2=depth, $3=limit; optional scope/type params appended. + const params: unknown[] = [seeds, depth, limit]; + const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0; + let seedScope = ''; + if (useSourceIds) { + params.push(opts!.sourceIds); + seedScope = `AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + seedScope = `AND p.source_id = $${params.length}`; + } + let typeFilter = ''; + if (types) { + params.push(types); + typeFilter = `AND l.link_type = ANY($${params.length}::text[])`; + } + const mentionsFilter = opts?.includeMentions ? '' : `AND l.link_source IS DISTINCT FROM 'mentions'`; + + const recurStep = + direction === 'out' + ? `JOIN links l ON l.from_page_id = w.id JOIN pages p2 ON p2.id = l.to_page_id` + : direction === 'in' + ? `JOIN links l ON l.to_page_id = w.id JOIN pages p2 ON p2.id = l.from_page_id` + : `JOIN links l ON (l.from_page_id = w.id OR l.to_page_id = w.id) + JOIN pages p2 ON p2.id = CASE WHEN l.from_page_id = w.id THEN l.to_page_id ELSE l.from_page_id END`; + + const sql = ` + WITH RECURSIVE walk AS ( + SELECT p.id, p.slug, p.source_id, 0::int AS depth, + ARRAY[p.id] AS visited, ARRAY[p.slug] AS path, + p.source_id AS seed_source, NULL::text AS last_link_type + FROM pages p + WHERE p.slug = ANY($1::text[]) ${seedScope} AND p.deleted_at IS NULL + UNION ALL + SELECT p2.id, p2.slug, p2.source_id, w.depth + 1, + w.visited || p2.id, w.path || p2.slug, + w.seed_source, l.link_type + FROM walk w + ${recurStep} + WHERE w.depth < $2 + AND NOT (p2.id = ANY(w.visited)) + AND p2.source_id = w.seed_source + AND p2.deleted_at IS NULL + ${mentionsFilter} + ${typeFilter} + ) + SELECT n.source_id, n.slug, + MIN(n.depth) AS hop, + COUNT(DISTINCT n.last_link_type) AS edge_count, + array_agg(DISTINCT n.last_link_type) + FILTER (WHERE n.last_link_type IS NOT NULL) AS via_link_types, + (array_agg(array_to_string(n.path, chr(9)) + ORDER BY n.depth ASC, array_length(n.path, 1) ASC))[1] AS path_str, + (SELECT cc.id FROM content_chunks cc + WHERE cc.page_id = n.id ORDER BY cc.chunk_index ASC LIMIT 1) AS canonical_chunk_id + FROM walk n + WHERE n.depth > 0 + GROUP BY n.source_id, n.slug, n.id + ORDER BY hop ASC, edge_count DESC, n.source_id ASC, n.slug ASC + LIMIT $3 + `; + + const { rows } = await this.db.query(sql, params); + return (rows as Record[]).map(r => ({ + source_id: r.source_id as string, + slug: r.slug as string, + hop: Number(r.hop), + edge_count: Number(r.edge_count), + via_link_types: Array.isArray(r.via_link_types) ? (r.via_link_types as string[]) : [], + path: r.path_str ? String(r.path_str).split('\t') : [], + canonical_chunk_id: r.canonical_chunk_id == null ? null : Number(r.canonical_chunk_id), + })); + } + async getBacklinkCounts(slugs: string[]): Promise> { const result = new Map(); if (slugs.length === 0) return result; diff --git a/src/core/postgres-engine.ts b/src/core/postgres-engine.ts index 7066c6934..ba0218a61 100644 --- a/src/core/postgres-engine.ts +++ b/src/core/postgres-engine.ts @@ -2950,6 +2950,88 @@ export class PostgresEngine implements BrainEngine { return result; } + async relationalFanout( + seeds: string[], + opts?: import('./types.ts').RelationalFanoutOpts, + ): Promise { + if (!seeds || seeds.length === 0) return []; + const sql = this.sql; + const depth = Math.min(Math.max(1, opts?.depth ?? 2), 3); + const direction = opts?.direction ?? 'both'; + const limit = Math.min(Math.max(1, opts?.limit ?? 50), 200); + const types = opts?.linkTypes && opts.linkTypes.length > 0 ? opts.linkTypes : null; + + // Scope is applied to SEED selection only. Within-source traversal is + // enforced separately by `p2.source_id = w.seed_source` in the recursive + // step, so a walk can never cross a source boundary even when several + // sources are in scope. + const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0; + const seedScope = useSourceIds + ? sql`AND p.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND p.source_id = ${opts.sourceId}` + : sql``; + const typeFilter = types ? sql`AND l.link_type = ANY(${types}::text[])` : sql``; + const mentionsFilter = opts?.includeMentions + ? sql`` + : sql`AND l.link_source IS DISTINCT FROM 'mentions'`; + + // Recursive step join differs by direction; everything else is shared. + const recurStep = + direction === 'out' + ? sql`JOIN links l ON l.from_page_id = w.id JOIN pages p2 ON p2.id = l.to_page_id` + : direction === 'in' + ? sql`JOIN links l ON l.to_page_id = w.id JOIN pages p2 ON p2.id = l.from_page_id` + : sql`JOIN links l ON (l.from_page_id = w.id OR l.to_page_id = w.id) + JOIN pages p2 ON p2.id = CASE WHEN l.from_page_id = w.id THEN l.to_page_id ELSE l.from_page_id END`; + + const rows = await sql` + WITH RECURSIVE walk AS ( + SELECT p.id, p.slug, p.source_id, 0::int AS depth, + ARRAY[p.id] AS visited, ARRAY[p.slug] AS path, + p.source_id AS seed_source, NULL::text AS last_link_type + FROM pages p + WHERE p.slug = ANY(${seeds}::text[]) ${seedScope} AND p.deleted_at IS NULL + UNION ALL + SELECT p2.id, p2.slug, p2.source_id, w.depth + 1, + w.visited || p2.id, w.path || p2.slug, + w.seed_source, l.link_type + FROM walk w + ${recurStep} + WHERE w.depth < ${depth} + AND NOT (p2.id = ANY(w.visited)) + AND p2.source_id = w.seed_source + AND p2.deleted_at IS NULL + ${mentionsFilter} + ${typeFilter} + ) + SELECT n.source_id, n.slug, + MIN(n.depth) AS hop, + COUNT(DISTINCT n.last_link_type) AS edge_count, + array_agg(DISTINCT n.last_link_type) + FILTER (WHERE n.last_link_type IS NOT NULL) AS via_link_types, + (array_agg(array_to_string(n.path, chr(9)) + ORDER BY n.depth ASC, array_length(n.path, 1) ASC))[1] AS path_str, + (SELECT cc.id FROM content_chunks cc + WHERE cc.page_id = n.id ORDER BY cc.chunk_index ASC LIMIT 1) AS canonical_chunk_id + FROM walk n + WHERE n.depth > 0 + GROUP BY n.source_id, n.slug, n.id + ORDER BY hop ASC, edge_count DESC, n.source_id ASC, n.slug ASC + LIMIT ${limit} + `; + + return (rows as Record[]).map(r => ({ + source_id: r.source_id as string, + slug: r.slug as string, + hop: Number(r.hop), + edge_count: Number(r.edge_count), + via_link_types: Array.isArray(r.via_link_types) ? (r.via_link_types as string[]) : [], + path: r.path_str ? String(r.path_str).split('\t') : [], + canonical_chunk_id: r.canonical_chunk_id == null ? null : Number(r.canonical_chunk_id), + })); + } + async getBacklinkCounts(slugs: string[]): Promise> { const result = new Map(); if (slugs.length === 0) return result; diff --git a/src/core/search/hybrid.ts b/src/core/search/hybrid.ts index 179bdfb95..ee1b08c6e 100644 --- a/src/core/search/hybrid.ts +++ b/src/core/search/hybrid.ts @@ -23,6 +23,7 @@ import { type AdaptiveReturnDecision, } from './return-policy.ts'; import { applyAutocut, type AutocutDecision } from './autocut.ts'; +import { buildRelationalArm } from './relational-recall.ts'; import { loadConfigWithEngine } from '../config.ts'; import { dedupResults } from './dedup.ts'; import { applyReranker } from './rerank.ts'; @@ -664,6 +665,9 @@ export async function applyAliasHop( export interface HybridSearchOpts extends SearchOpts { expansion?: boolean; + /** v0.43 — observability sink for the relational recall arm (fired/no-op, + * kind, seeds resolved, candidates, errored). Best-effort. */ + onRelationalMeta?: (meta: import('./relational-recall.ts').RelationalArmMeta) => void; /** * T4/D5 — per-call search-mode selector (one of SEARCH_MODES). Selects the * whole mode bundle for this call, overriding the server-configured mode. @@ -806,6 +810,11 @@ export async function hybridSearch( // Non-boolean AutocutInput shapes (Partial) aren't a v1 per-call surface, // so only the boolean toggle threads here. autocut: typeof opts?.autocut === 'boolean' ? opts.autocut : undefined, + // v0.43 — relational recall per-call thread-through. Per-call wins over + // config override wins over mode bundle; without this the A/B eval gate + // would be a no-op (both branches resolve to the same mode default). + relationalRetrieval: opts?.relationalRetrieval, + relational_retrieval_depth: opts?.relationalRetrievalDepth, }, }); @@ -972,6 +981,23 @@ export async function hybridSearch( titleBoost: resolvedMode.title_boost, }; + // v0.43 — build the relational recall arm ONCE here, before any return + // path, so typed-edge answers contribute on ALL THREE paths: the + // no-embedding-provider path, the embed-failed keyword fallback, and the + // main RRF path. Parsed from the original query (deterministic); empty for + // non-relational queries → pure no-op. (Modality gate lives on the main + // path; the parser only matches text-shaped relational queries anyway.) + let relationalList: SearchResult[] = []; + if (resolvedMode.relationalRetrieval) { + relationalList = await buildRelationalArm(engine, query, { + sourceId: opts?.sourceId, + sourceIds: opts?.sourceIds, + depth: resolvedMode.relational_retrieval_depth, + limit: opts?.limit ?? resolvedMode.searchLimit, + onMeta: opts?.onRelationalMeta, + }); + } + // Skip vector search entirely if the gateway has no embedding provider configured (Codex C3). // v0.36 (D10): ask "is the RESOLVED column's provider reachable?" rather // than "is the global default reachable?" — otherwise an unreachable @@ -980,13 +1006,24 @@ export async function hybridSearch( const { isAvailable } = await import('../ai/gateway.ts'); const providerProbe = resolvedCol.embeddingModel || undefined; if (!isAvailable('embedding', providerProbe)) { - if (keywordResults.length > 0) { - await runPostFusionStages(engine, keywordResults, postFusionOpts); - keywordResults.sort((a, b) => b.score - a.score); + // v0.43 — fuse the relational arm with keyword so typed-edge answers + // survive on the no-embedding-provider path (the relational win is most + // valuable exactly when vector is unavailable). + let noEmbedResults = keywordResults; + if (relationalList.length > 0) { + const fk = opts?.rrfK ?? RRF_K; + noEmbedResults = rrfFusionWeighted( + [{ list: keywordResults, k: fk }, { list: relationalList, k: fk }], + detailResolved !== 'high', + ); + } + if (noEmbedResults.length > 0) { + await runPostFusionStages(engine, noEmbedResults, postFusionOpts); + noEmbedResults.sort((a, b) => b.score - a.score); } // T3/T4 — alias hop + evidence stamp even without an embedding provider // (the named-thing fix is most valuable exactly when vector is unavailable). - const noEmbedHopped = await applyAliasHop(engine, dedupResults(keywordResults), query, { + const noEmbedHopped = await applyAliasHop(engine, dedupResults(noEmbedResults), query, { sourceId: opts?.sourceId, sourceIds: opts?.sourceIds, }); @@ -1204,11 +1241,21 @@ export async function hybridSearch( // v0.29.1 codex pass-2 #4: this is the third return path. Apply // post-fusion stages here too — without it, salience='on' silently // does nothing on embed failures. - if (keywordResults.length > 0) { - await runPostFusionStages(engine, keywordResults, postFusionOpts); - keywordResults.sort((a, b) => b.score - a.score); + // v0.43: fuse the relational arm with keyword via RRF so typed-edge + // answers survive even when vector is unavailable. + let fallbackResults = keywordResults; + if (relationalList.length > 0) { + const fk = opts?.rrfK ?? RRF_K; + fallbackResults = rrfFusionWeighted( + [{ list: keywordResults, k: fk }, { list: relationalList, k: fk }], + detail !== 'high', + ); } - const kwHopped = await applyAliasHop(engine, dedupResults(keywordResults), query, { + if (fallbackResults.length > 0) { + await runPostFusionStages(engine, fallbackResults, postFusionOpts); + fallbackResults.sort((a, b) => b.score - a.score); + } + const kwHopped = await applyAliasHop(engine, dedupResults(fallbackResults), query, { sourceId: opts?.sourceId, sourceIds: opts?.sourceIds, }); @@ -1266,6 +1313,16 @@ export async function hybridSearch( ...vectorLists.map(list => ({ list, k: vectorK })), { list: keywordResults, k: keywordK }, ]; + + // v0.43 — relational recall arm (fourth RRF arm), built above so it also + // contributes on the keyword-only fallback path. Neutral weight (baseRrfK): + // competes evenly with keyword/vector, not dominating. Empty for + // non-relational queries → pure no-op. Rides every downstream stage (cosine + // re-score, post-fusion boosts, dedup, reranker, autocut, token budget). + if (relationalList.length > 0 && effectiveModality !== 'image') { + allLists.push({ list: relationalList, k: baseRrfK }); + } + let fused = rrfFusionWeighted(allLists, detail !== 'high'); // Cosine re-scoring before dedup so semantically better chunks survive. @@ -1511,6 +1568,11 @@ export async function hybridSearchCached( // this, an `autocut:false` (full top-K) call could be served a trimmed // autocut-on cache row, or vice versa. autocut: typeof opts?.autocut === 'boolean' ? opts.autocut : undefined, + // v0.43 — relational recall per-call thread-through. Per-call wins over + // config override wins over mode bundle; without this the A/B eval gate + // would be a no-op (both branches resolve to the same mode default). + relationalRetrieval: opts?.relationalRetrieval, + relational_retrieval_depth: opts?.relationalRetrievalDepth, }, }); // v0.36 (D8 / CDX-2 + codex /ship #4): resolve column for the cache @@ -1607,7 +1669,7 @@ export async function hybridSearchCached( } if (!skipCache && queryEmbedding && cacheStatus !== 'disabled') { - const hit = await cache.lookup(queryEmbedding, { sourceId: opts?.sourceId, knobsHash: cacheKnobsHash }); + const hit = await cache.lookup(queryEmbedding, { sourceId: cacheScopeKey(opts), knobsHash: cacheKnobsHash }); if (hit.hit && hit.results) { cacheStatus = 'hit'; cacheSimilarity = hit.similarity; @@ -1708,7 +1770,7 @@ export async function hybridSearchCached( ) { trackCacheWrite( cache - .store(query, queryEmbedding, results, finalMeta, { sourceId: opts?.sourceId, knobsHash: cacheKnobsHash }) + .store(query, queryEmbedding, results, finalMeta, { sourceId: cacheScopeKey(opts), knobsHash: cacheKnobsHash }) .catch(() => { /* swallow */ }), ); } @@ -1716,6 +1778,42 @@ export async function hybridSearchCached( return budgeted; } +/** + * RRF/dedup identity for a result row, at chunk granularity. + * + * Includes `source_id` so two same-slug pages in different federated sources + * don't collapse into one fusion entry (the same composite-key discipline + * `dedup.ts:pageKey` already uses at page granularity). Pre-fix the key was + * `slug:chunk_id`, which silently merged cross-source rows and let a + * synthetic chunkless row (chunk_id null) key on a text prefix; the + * `(source_id, slug, chunk_id)` shape is collision-safe for both. + */ +function rrfKey(r: SearchResult): string { + const source = r.source_id ?? 'default'; + return `${source}:${r.slug}:${r.chunk_id ?? r.chunk_text.slice(0, 50)}`; +} + +/** + * Canonical query-cache scope key. + * + * The semantic cache stores results keyed by `(scope, query, knobs_hash)`. + * A federated search (`sourceIds`) reads a different graph than a + * single-source one, so the two must never share a cache row. Pre-fix the + * cache only saw scalar `sourceId`; a federated query fell through to + * `'default'` and could cross-serve an unrelated scope. + * + * - federated (sourceIds set) → `__set__:` + sorted, comma-joined ids + * (order-independent; two different source-sets get distinct keys) + * - scalar sourceId → the id itself (single-source unchanged) + * - unscoped → `'default'` (single-source brains unchanged) + */ +export function cacheScopeKey(opts?: { sourceId?: string; sourceIds?: string[] }): string { + if (opts?.sourceIds && opts.sourceIds.length > 0) { + return '__set__:' + [...opts.sourceIds].sort().join(','); + } + return opts?.sourceId ?? 'default'; +} + /** * v0.32.x search-lite — weighted RRF. Each list contributes with its own * effective k value, which lets intent weighting bias keyword vs vector @@ -1731,7 +1829,7 @@ export function rrfFusionWeighted( for (const { list, k } of lists) { for (let rank = 0; rank < list.length; rank++) { const r = list[rank]; - const key = `${r.slug}:${r.chunk_id ?? r.chunk_text.slice(0, 50)}`; + const key = rrfKey(r); const existing = scores.get(key); const rrfScore = 1 / (k + rank); @@ -1771,7 +1869,7 @@ export function rrfFusion(lists: SearchResult[][], k: number, applyBoost = true) for (const list of lists) { for (let rank = 0; rank < list.length; rank++) { const r = list[rank]; - const key = `${r.slug}:${r.chunk_id ?? r.chunk_text.slice(0, 50)}`; + const key = rrfKey(r); const existing = scores.get(key); const rrfScore = 1 / (k + rank); diff --git a/src/core/search/mode.ts b/src/core/search/mode.ts index 9fe7eb9b3..d5a8198e6 100644 --- a/src/core/search/mode.ts +++ b/src/core/search/mode.ts @@ -261,6 +261,18 @@ export interface ModeBundle { * run. Override: `search.autocut_jump` config → mode bundle. */ autocut_jump: number; + /** + * v0.43 — relational recall arm. When on, a relational query ("who invested + * in widget-co", "what connects fund-a and fund-b") resolves its seed + * entity and walks the typed-edge graph, injecting edge-derived candidates + * as a fourth RRF arm. Pure no-op for non-relational queries. Default OFF + * for conservative; ON for balanced/tokenmax. Override path: per-call + * SearchOpts.relationalRetrieval → `search.relational_retrieval` config → + * mode bundle. See src/core/search/relational-recall.ts. + */ + relationalRetrieval: boolean; + /** v0.43 — max hops for relational traversal. Default 2, hard-capped at 3. */ + relational_retrieval_depth: number; } /** @@ -309,6 +321,10 @@ export const MODE_BUNDLES: Readonly>> = // v0.42.3.0 — autocut OFF: conservative has no reranker, so no trustworthy // cliff signal exists (autocut would no-op). Explicit for clarity. autocut: false, + // v0.43 — relational recall OFF for conservative (cost-sensitive tier, + // matches graph_signals posture). Power users opt in per-call. + relationalRetrieval: false, + relational_retrieval_depth: 2, autocut_jump: 0.2, }), balanced: Object.freeze({ @@ -363,6 +379,10 @@ export const MODE_BUNDLES: Readonly>> = contextual_retrieval_disabled: false, // v0.42.3.0 — autocut ON (reranker fires; cliff signal is trustworthy). autocut: true, + // v0.43 — relational recall ON (contingent on the no-regression gate; + // ships default-false everywhere if the gate flags any regression). + relationalRetrieval: true, + relational_retrieval_depth: 2, autocut_jump: 0.2, }), tokenmax: Object.freeze({ @@ -411,6 +431,9 @@ export const MODE_BUNDLES: Readonly>> = contextual_retrieval_disabled: false, // v0.42.3.0 — autocut ON. autocut: true, + // v0.43 — relational recall ON for tokenmax (max-recall tier). + relationalRetrieval: true, + relational_retrieval_depth: 2, autocut_jump: 0.2, }), }); @@ -462,6 +485,9 @@ export interface SearchKeyOverrides { contextual_retrieval_disabled?: boolean; // v0.42.3.0 — autocut overrides. autocut?: boolean; + // v0.43 — relational recall overrides. + relationalRetrieval?: boolean; + relational_retrieval_depth?: number; autocut_jump?: number; } @@ -509,6 +535,9 @@ export interface SearchPerCallOpts { // numeric per-call knob threaded through the bundle. autocut?: boolean; autocut_jump?: number; + // v0.43 — relational recall per-call overrides. + relationalRetrieval?: boolean; + relational_retrieval_depth?: number; } /** @@ -601,6 +630,9 @@ export function resolveSearchMode(input: ResolveSearchModeInput): ResolvedSearch // v0.42.3.0 — autocut resolved via the same pick chain. autocut: pick('autocut'), autocut_jump: pick('autocut_jump'), + // v0.43 — relational recall resolved via the same pick chain. + relationalRetrieval: pick('relationalRetrieval'), + relational_retrieval_depth: pick('relational_retrieval_depth'), resolved_mode, mode_valid: valid, }; @@ -706,7 +738,7 @@ export function attributeKnob( // to take effect immediately (one-time global cache cold-miss on upgrade; refills // within cache.ttl_seconds). Same cache-key-contamination convention as the // autocut / title_boost / graph_signals bumps above. -export const KNOBS_HASH_VERSION = 9; +export const KNOBS_HASH_VERSION = 10; /** * v0.36 (D8 / CDX-2) — second-arg context for the cache key. The @@ -814,6 +846,14 @@ export function knobsHash( // etc.) so a partial-knobs caller (tests passing a minimal literal) can't // crash the hash. Typed callers always carry the field. `acj=${(knobs.autocut_jump ?? 0.2).toFixed(2)}`, + // v=10 additions (v0.43, append-only): relational recall arm. A + // relational-on write (edge-seeded result set) must NOT be served to a + // relational-off lookup — same contamination class as graph_signals. The + // depth changes the candidate set too, so it folds in as well. ONE-TIME + // cold-miss on upgrade as v=9 rows become unreachable; pinned by + // test/model-pricing.test.ts-style drift guards and the mode tests. + `rel=${knobs.relationalRetrieval ? 1 : 0}`, + `reld=${knobs.relational_retrieval_depth ?? 2}`, ]; const h = createHash('sha256'); h.update(parts.join('|')); @@ -981,6 +1021,17 @@ export function loadOverridesFromConfig( if (Number.isFinite(n) && n > 0 && n <= 1) out.autocut_jump = n; } + // v0.43 — relational recall arm. + const rel = get('search.relational_retrieval'); + if (rel !== undefined) { + out.relationalRetrieval = rel === '1' || rel.toLowerCase() === 'true'; + } + const reld = get('search.relational_retrieval_depth'); + if (reld !== undefined) { + const n = parseInt(reld, 10); + if (Number.isFinite(n) && n >= 1 && n <= 3) out.relational_retrieval_depth = n; + } + return out; } @@ -1019,6 +1070,9 @@ export const SEARCH_MODE_CONFIG_KEYS: ReadonlyArray = Object.freeze([ 'search.contextual_retrieval_disabled', // v0.42.3.0 autocut 'search.autocut', + // v0.43 relational recall + 'search.relational_retrieval', + 'search.relational_retrieval_depth', 'search.autocut_jump', ]); diff --git a/src/core/search/relational-intent.ts b/src/core/search/relational-intent.ts new file mode 100644 index 000000000..cccf820de --- /dev/null +++ b/src/core/search/relational-intent.ts @@ -0,0 +1,243 @@ +/** + * Relational-query parser (typed-edge retrieval, v0.43). + * + * Detects queries whose answer is a RELATIONSHIP (an edge between entities) + * rather than a passage — "who invested in widget-co", "who at acme works on + * payments", "who introduced me to alice", "what connects fund-a and fund-b". + * The relational recall arm uses the parse to resolve seed entities and walk + * the typed-edge graph. + * + * Pure module. No DB, no LLM, no async. Detection is regex-only and + * deterministic, parsed from the ORIGINAL query (never the LLM-expanded + * variant) so the recall arm stays bit-for-bit reproducible. + * + * Precision-first (D4): this returns a CANDIDATE. The arm fires only when a + * resolvable seed entity is also found (seed resolution lives in + * relational-recall.ts). Patterns require the relation phrase and the entity + * to be adjacent, so "who invested TIME in learning Rust" does not match the + * "who invested in " pattern. + * + * Vocabulary (D2): the default bank covers the common archetypes; a schema + * pack can extend it with `extraVerbs`. Every emitted link_type is validated + * against KNOWN_LINK_TYPES so the query side can't drift from what ingest + * actually produces (see link-extraction.ts:inferLinkType). intro/connects + * traverse type-agnostically (linkTypes = null) because gbrain has no + * `introduced`/`knows` edge — any edge touching the seed is the signal. + * + * ReDoS: seed captures are length-bounded (`.{1,80}?`) and every pattern is + * anchored, so there is no catastrophic-backtracking surface. + * + * Tested in test/relational-intent.test.ts. + */ + +export type RelationalKind = 'who_rel' | 'who_at' | 'connects' | 'intro'; +export type RelationDirection = 'in' | 'out' | 'both'; + +export interface RelationalQuery { + /** Which archetype matched. */ + kind: RelationalKind; + /** Raw entity phrases to resolve, in query order. 1 for most, 2 for connects. */ + seeds: string[]; + /** Typed edges to traverse, or null for type-agnostic traversal. */ + linkTypes: string[] | null; + /** Traversal direction from the seed. */ + direction: RelationDirection; + /** The matched relation phrase, for telemetry / --explain. */ + relationPhrase: string; +} + +/** Schema-pack vocab extension (D2=B). */ +export interface RelationVerbSpec { + /** A regex-source alternation of phrasings, e.g. `acquired|bought`. */ + verb: string; + /** Edges this verb maps to. MUST be a subset of KNOWN_LINK_TYPES. */ + linkTypes: string[]; + /** Direction from the seed entity named after the verb. */ + direction: RelationDirection; +} + +export interface RelationVocab { + extraVerbs?: RelationVerbSpec[]; +} + +/** + * Link types ingest can actually produce (link-extraction.ts + frontmatter + * map + schema packs). The query parser may only emit a SUBSET of these, so a + * relation phrase can never traverse an edge type that ingest never writes. + * `validateVocab` enforces this for pack-supplied verbs. + */ +export const KNOWN_LINK_TYPES: ReadonlySet = new Set([ + 'founded', + 'invested_in', + 'advises', + 'works_at', + 'attended', + 'yc_partner', + 'led_round', + 'mentions', + 'image_of', + 'discussed_in', + 'source', + 'related_to', + 'wikilink_basename', +]); + +// Seeds that are pronouns / generic nouns, not entities. If a pattern's seed +// cleans down to one of these, the parse is rejected (precision-first). +const STOPWORD_SEEDS: ReadonlySet = new Set([ + 'it', 'that', 'this', 'them', 'these', 'those', 'here', 'there', + 'everyone', 'anyone', 'someone', 'anybody', 'somebody', 'people', + 'things', 'us', 'me', 'him', 'her', 'you', 'who', 'what', 'which', +]); + +interface CompiledPattern { + re: RegExp; + kind: RelationalKind; + linkTypes: string[] | null; + direction: RelationDirection; + /** Number of seed capture groups (1, or 2 for connects). */ + seedGroups: 1 | 2; +} + +// Bounded seed capture: 1–80 chars, lazy, so the trailing anchor decides the +// boundary without catastrophic backtracking. +const SEED = '(.{1,80}?)'; + +// ── who_rel verb bank: "who " → traverse INTO the seed ── +// Each entry is explicit (linkTypes inline) so there is no second lookup. +const WHO_REL_VERBS: Array<{ verb: string; linkTypes: string[]; direction: RelationDirection }> = [ + { verb: 'invested in|invests in|funded|backed|backs|led the round in|led the seed in|led the series [a-z] in', linkTypes: ['invested_in', 'led_round'], direction: 'in' }, + { verb: 'founded|co-?founded|started', linkTypes: ['founded'], direction: 'in' }, + { verb: 'advises|advised', linkTypes: ['advises'], direction: 'in' }, + { verb: 'works at|worked at|works for', linkTypes: ['works_at'], direction: 'in' }, + { verb: 'attended', linkTypes: ['attended'], direction: 'in' }, +]; + +function buildPatterns(vocab?: RelationVocab): CompiledPattern[] { + const patterns: CompiledPattern[] = []; + + // connects — two seeds, type-agnostic. Most specific, checked first. + patterns.push({ + re: new RegExp( + `\\b(?:what|which)\\s+(?:companies?|people|things|entities|deals?)?\\s*(?:connects?|links?|ties? together|is (?:the )?(?:connection|link|relationship) between)\\s+${SEED}\\s+(?:and|&)\\s+${SEED}\\s*\\??$`, + 'i', + ), + kind: 'connects', linkTypes: null, direction: 'both', seedGroups: 2, + }); + patterns.push({ + re: new RegExp( + `\\bhow\\s+(?:are|is|do|does)\\s+${SEED}\\s+(?:and|&)\\s+${SEED}\\s+(?:connected|related|linked|associated)\\b`, + 'i', + ), + kind: 'connects', linkTypes: null, direction: 'both', seedGroups: 2, + }); + + // intro — type-agnostic walk around the named person (no `introduced` edge). + patterns.push({ + re: new RegExp( + `\\bwho\\s+(?:introduced|connected|referred)\\s+(?:me|us|him|her|them)\\s+to\\s+${SEED}\\s*\\??$`, + 'i', + ), + kind: 'intro', linkTypes: null, direction: 'both', seedGroups: 1, + }); + + // who_at — entity in the middle: "who at acme works on payments". + patterns.push({ + re: new RegExp( + `\\bwho\\s+(?:at|from|in)\\s+${SEED}\\s+(?:works? on|works?|leads?|runs?|builds?|owns?|handles?|manages?)\\b`, + 'i', + ), + kind: 'who_at', linkTypes: ['works_at'], direction: 'in', seedGroups: 1, + }); + + // who_rel — "who ". + for (const v of WHO_REL_VERBS) { + patterns.push({ + re: new RegExp(`\\bwho\\s+(?:${v.verb})\\s+${SEED}\\s*\\??$`, 'i'), + kind: 'who_rel', linkTypes: v.linkTypes, direction: v.direction, seedGroups: 1, + }); + } + + // outgoing variants — "what did invest in", "where does work". + patterns.push({ + re: new RegExp( + `\\bwhat\\s+(?:companies?|startups?|deals?)?\\s*(?:has|have|did|does)?\\s*${SEED}\\s+(?:invest(?:ed)? in)\\b`, + 'i', + ), + kind: 'who_rel', linkTypes: ['invested_in', 'led_round'], direction: 'out', seedGroups: 1, + }); + patterns.push({ + re: new RegExp(`\\bwhere\\s+(?:does|did|has)\\s+${SEED}\\s+work\\b`, 'i'), + kind: 'who_rel', linkTypes: ['works_at'], direction: 'out', seedGroups: 1, + }); + + // schema-pack extensions: "who " for each extra verb. + for (const v of vocab?.extraVerbs ?? []) { + patterns.push({ + re: new RegExp(`\\bwho\\s+(?:${v.verb})\\s+${SEED}\\s*\\??$`, 'i'), + kind: 'who_rel', linkTypes: v.linkTypes, direction: v.direction, seedGroups: 1, + }); + } + + return patterns; +} + +/** Trim, drop a leading article and surrounding quotes, strip trailing `?`. */ +function cleanSeed(raw: string): string { + return raw + .trim() + .replace(/\?+$/, '') + .replace(/^["'`]|["'`]$/g, '') + .replace(/^(?:the|a|an)\s+/i, '') + .trim(); +} + +function validSeed(s: string): boolean { + if (s.length === 0 || s.length > 80) return false; + if (STOPWORD_SEEDS.has(s.toLowerCase())) return false; + return true; +} + +/** + * Validate that every link_type a vocab emits is one ingest can produce. + * Throws on an unknown type so a misconfigured schema pack fails loudly at + * load time rather than silently traversing an edge that never exists. + */ +export function validateVocab(vocab: RelationVocab): void { + for (const v of vocab.extraVerbs ?? []) { + for (const lt of v.linkTypes) { + if (!KNOWN_LINK_TYPES.has(lt)) { + throw new Error( + `relational vocab: unknown link_type "${lt}" for verb /${v.verb}/ — must be one of ${[...KNOWN_LINK_TYPES].join(', ')}`, + ); + } + } + } +} + +/** + * Parse a query into a RelationalQuery, or null if it isn't relational. + * First matching pattern wins (patterns are ordered specific → general). + */ +export function parseRelationalQuery(query: string, vocab?: RelationVocab): RelationalQuery | null { + if (!query || query.length > 512) return null; // bound work; real queries are short + const patterns = buildPatterns(vocab); + + for (const p of patterns) { + const m = p.re.exec(query); + if (!m) continue; + + if (p.seedGroups === 2) { + const a = cleanSeed(m[1] ?? ''); + const b = cleanSeed(m[2] ?? ''); + if (!validSeed(a) || !validSeed(b)) continue; + return { kind: p.kind, seeds: [a, b], linkTypes: p.linkTypes, direction: p.direction, relationPhrase: m[0].trim() }; + } + + const seed = cleanSeed(m[1] ?? ''); + if (!validSeed(seed)) continue; + return { kind: p.kind, seeds: [seed], linkTypes: p.linkTypes, direction: p.direction, relationPhrase: m[0].trim() }; + } + + return null; +} diff --git a/src/core/search/relational-recall.ts b/src/core/search/relational-recall.ts new file mode 100644 index 000000000..7a8850ba0 --- /dev/null +++ b/src/core/search/relational-recall.ts @@ -0,0 +1,232 @@ +/** + * Relational recall arm (typed-edge retrieval, v0.43). + * + * Turns a relational query into a ranked list of edge-derived candidates that + * hybridSearch injects as a FOURTH RRF arm (alongside keyword + vector), so a + * relationship answer competes for ranking instead of relying on lexical or + * vector similarity to surface it. + * + * Flow: parseRelationalQuery → resolve seed entity(ies) (scope-aware, + * confidence-gated) → engine.relationalFanout (within-source, deterministic) + * → batch-hydrate to SearchResult rows (reinforcing each page's REAL canonical + * chunk; page-level key for chunkless entity pages). + * + * Determinism: parses the ORIGINAL query (never an LLM-expanded variant); + * traversal + resolution are deterministic. Fail-open: any error returns an + * empty arm + an audit row, never breaking the search hot path. + * + * Federation (E2=A): resolves the seed in every in-scope source and fans out + * from each; traversal stays WITHIN each source (no cross-boundary edges in + * v1). Confidence gate (D3): a seed that only `fallback_slugify`-resolves is + * dropped, so the arm never traverses from an invented slug. The tier-2 + * resolution-margin gate is a filed TODO. + * + * Tested in test/relational-recall.test.ts. + */ + +import type { BrainEngine } from '../engine.ts'; +import type { SearchResult, PageType, RelationalFanoutRow } from '../types.ts'; +import { createAuditWriter } from '../audit/audit-writer.ts'; +import { resolveEntitySlugWithSource } from '../entities/resolve.ts'; +import { parseRelationalQuery, type RelationalQuery, type RelationVocab } from './relational-intent.ts'; + +export interface RelationalArmOpts { + sourceId?: string; + sourceIds?: string[]; + depth?: number; + limit?: number; + vocab?: RelationVocab; + onMeta?: (meta: RelationalArmMeta) => void; +} + +export interface RelationalArmMeta { + fired: boolean; + kind: RelationalQuery['kind'] | null; + seeds_resolved: number; + candidates: number; + errored: boolean; + duration_ms: number; +} + +interface RelationalFailureEvent { + ts: string; + error_summary: string; + query_kind: string; +} + +const failureWriter = createAuditWriter({ + featureName: 'relational-recall-failures', + errorLabel: 'gbrain', + errorTrailer: '; search continues', +}); + +/** Recent relational-arm fail-open events — consumed by doctor + search stats. */ +export function readRecentRelationalFailures(days = 7, now: Date = new Date()): RelationalFailureEvent[] { + return failureWriter.readRecent(days, now); +} + +function truncate(msg: string, max = 200): string { + return msg.length <= max ? msg : msg.slice(0, max - 1) + '…'; +} + +/** Sources to resolve a seed against. Federated → the set; scalar → [id]; + * unscoped/__all__ → ['default'] (single-source brains; multi-source + * enumeration under __all__ is a v1 limitation). */ +function scopeSources(opts: RelationalArmOpts): string[] { + if (opts.sourceIds && opts.sourceIds.length > 0) return opts.sourceIds; + if (opts.sourceId && opts.sourceId !== '__all__') return [opts.sourceId]; + return ['default']; +} + +/** Resolve a seed phrase to all in-scope (source_id, slug) pairs that + * resolve to a REAL page (confidence gate D3 tier-1: drop fallback_slugify). */ +async function resolveSeedScoped( + engine: BrainEngine, + sources: string[], + phrase: string, +): Promise> { + const out: Array<{ source_id: string; slug: string }> = []; + const seen = new Set(); + for (const sid of sources) { + const r = await resolveEntitySlugWithSource(engine, sid, phrase); + if (!r || r.source === 'fallback_slugify') continue; + const key = `${sid}:${r.slug}`; + if (seen.has(key)) continue; + seen.add(key); + out.push({ source_id: sid, slug: r.slug }); + } + return out; +} + +/** Batch-hydrate fanout rows into SearchResult rows in fanout (ranked) order. */ +async function hydrate( + engine: BrainEngine, + rows: RelationalFanoutRow[], + seedSlug: string, +): Promise { + if (rows.length === 0) return []; + const slugs = Array.from(new Set(rows.map(r => r.slug))); + const pageRows = await engine.executeRaw<{ + page_id: number; slug: string; source_id: string; title: string; type: string; synopsis: string | null; + }>( + `SELECT p.id AS page_id, p.slug, p.source_id, p.title, p.type, + LEFT(p.compiled_truth, 240) AS synopsis + FROM pages p + WHERE p.slug = ANY($1::text[]) AND p.deleted_at IS NULL`, + [slugs], + ); + const byKey = new Map(); + for (const pr of pageRows) byKey.set(`${pr.source_id}:${pr.slug}`, pr); + + const out: SearchResult[] = []; + for (const r of rows) { + const pr = byKey.get(`${r.source_id}:${r.slug}`); + if (!pr) continue; + out.push({ + slug: r.slug, + page_id: pr.page_id, + title: pr.title, + type: pr.type as PageType, + chunk_text: pr.synopsis ?? r.slug, + chunk_source: 'compiled_truth', + // E1: reinforce the page's REAL canonical chunk; F3: chunkless entity + // pages key page-level (chunk_id 0 → rrfKey `source:slug:0`, stable and + // collision-safe; SERIAL chunk ids start at 1 so 0 never aliases a real chunk). + chunk_id: r.canonical_chunk_id ?? 0, + chunk_index: 0, + score: 0, // rank-based: RRF derives score from list position + stale: false, + source_id: r.source_id, + relational_via_link_types: r.via_link_types, + relational_seed: seedSlug, + relational_hop: r.hop, + relational_path: r.path, + }); + } + return out; +} + +/** + * Build the relational recall arm. Returns an empty list (pure no-op) when the + * query isn't relational or no seed resolves. Never throws. + */ +export async function buildRelationalArm( + engine: BrainEngine, + query: string, + opts: RelationalArmOpts = {}, +): Promise { + const startedAt = Date.now(); + const meta: RelationalArmMeta = { + fired: false, kind: null, seeds_resolved: 0, candidates: 0, errored: false, duration_ms: 0, + }; + const finish = (list: SearchResult[]) => { + meta.candidates = list.length; + meta.duration_ms = Date.now() - startedAt; + opts.onMeta?.(meta); + return list; + }; + + const parsed = parseRelationalQuery(query, opts.vocab); + if (!parsed) return finish([]); + meta.kind = parsed.kind; + + try { + const sources = scopeSources(opts); + const fanoutOpts = { + linkTypes: parsed.linkTypes, + direction: parsed.direction, + depth: opts.depth, + limit: opts.limit, + }; + + if (parsed.kind === 'connects' && parsed.seeds.length === 2) { + // Resolve both endpoints; both must resolve or the arm no-ops. + const resA = await resolveSeedScoped(engine, sources, parsed.seeds[0]); + const resB = await resolveSeedScoped(engine, sources, parsed.seeds[1]); + if (resA.length === 0 || resB.length === 0) return finish([]); + meta.seeds_resolved = resA.length + resB.length; + + const perSource = (rs: typeof resA) => ({ + sourceId: rs.length === 1 ? rs[0].source_id : undefined, + sourceIds: rs.length > 1 ? Array.from(new Set(rs.map(x => x.source_id))) : undefined, + slugs: Array.from(new Set(rs.map(x => x.slug))), + }); + const a = perSource(resA); + const b = perSource(resB); + const fanA = await engine.relationalFanout(a.slugs, { ...fanoutOpts, sourceId: a.sourceId, sourceIds: a.sourceIds }); + const fanB = await engine.relationalFanout(b.slugs, { ...fanoutOpts, sourceId: b.sourceId, sourceIds: b.sourceIds }); + // Shared midpoints: nodes reachable from BOTH endpoints (exclude the + // endpoints themselves). Ordered by combined hop. + const bByKey = new Map(fanB.map(r => [`${r.source_id}:${r.slug}`, r] as const)); + const endpointSlugs = new Set([...a.slugs, ...b.slugs]); + const shared = fanA + .filter(r => bByKey.has(`${r.source_id}:${r.slug}`) && !endpointSlugs.has(r.slug)) + .map(r => ({ row: r, combined: r.hop + bByKey.get(`${r.source_id}:${r.slug}`)!.hop })) + .sort((x, y) => x.combined - y.combined || x.row.slug.localeCompare(y.row.slug)) + .map(x => x.row); + const list = await hydrate(engine, shared, parsed.seeds.join(' ↔ ')); + meta.fired = list.length > 0; + return finish(list); + } + + // who_rel / who_at / intro: single logical seed (may resolve in N sources). + const resolved = await resolveSeedScoped(engine, sources, parsed.seeds[0]); + if (resolved.length === 0) return finish([]); + meta.seeds_resolved = resolved.length; + const slugs = Array.from(new Set(resolved.map(r => r.slug))); + const srcIds = Array.from(new Set(resolved.map(r => r.source_id))); + const rows = await engine.relationalFanout(slugs, { + ...fanoutOpts, + sourceId: srcIds.length === 1 ? srcIds[0] : undefined, + sourceIds: srcIds.length > 1 ? srcIds : undefined, + }); + const list = await hydrate(engine, rows, resolved[0].slug); + meta.fired = list.length > 0; + return finish(list); + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + failureWriter.log({ error_summary: truncate(msg), query_kind: parsed.kind }); + meta.errored = true; + return finish([]); + } +} diff --git a/src/core/types.ts b/src/core/types.ts index 2933c42a5..16488b5ff 100644 --- a/src/core/types.ts +++ b/src/core/types.ts @@ -719,6 +719,20 @@ export interface SearchResult { graph_session_demoted?: boolean; /** Slug prefix used for the session-diversification grouping. */ graph_session_prefix?: string; + /** + * v0.43 relational recall arm — set when this result was surfaced by + * typed-edge traversal (not lexical/vector). Drives `gbrain search + * --explain` attribution ("surfaced via invested_in edge from widget-co"). + * Absent for organic keyword/vector results. + */ + /** Edge types the result was reached by (e.g. ['invested_in']). */ + relational_via_link_types?: string[]; + /** The resolved seed entity slug the traversal started from. */ + relational_seed?: string; + /** Shortest hop distance from the seed (1 = direct neighbor). */ + relational_hop?: number; + /** Shortest connecting slug path seed→…→result (for "how I know this"). */ + relational_path?: string[]; /** * v0.40.4 full attribution (D12=A) — per-stage score deltas for the * `gbrain search --explain` formatter. Every boost stage stamps its @@ -1089,6 +1103,14 @@ export interface SearchOpts { * would resolve to the same mode default and the gate would be a no-op. */ graph_signals?: boolean; + /** + * v0.43 — relational recall arm per-call override. Per-call wins over the + * `search.relational_retrieval` config key wins over the mode bundle. Eval + * A/B gates need explicit per-call control or both branches resolve to the + * same mode default. `relationalRetrievalDepth` caps traversal hops (1..3). + */ + relationalRetrieval?: boolean; + relationalRetrievalDepth?: number; } /** @@ -1177,6 +1199,43 @@ export interface GraphPath { depth: number; } +/** + * One reached node from a typed-edge relational fan-out (v0.43). The recall + * arm hydrates these into SearchResult rows and injects them as a fourth RRF + * arm. Aggregated to the page level: `hop` is the shortest distance from any + * seed, `edge_count` is a connection-richness proxy (distinct edge types via + * which the node is reached), `via_link_types` names them, `path` is the + * shortest connecting slug chain (for --explain), and `canonical_chunk_id` is + * the page's lowest-ordinal chunk (null for frontmatter-only entity pages). + */ +export interface RelationalFanoutRow { + source_id: string; + slug: string; + hop: number; + edge_count: number; + via_link_types: string[]; + path: string[]; + canonical_chunk_id: number | null; +} + +/** Options for BrainEngine.relationalFanout. */ +export interface RelationalFanoutOpts { + /** Edge types to traverse; null/empty = type-agnostic. */ + linkTypes?: string[] | null; + /** Direction from each seed. Default 'both'. */ + direction?: 'in' | 'out' | 'both'; + /** Max hops. Default 2, hard-capped at 3. */ + depth?: number; + /** Include `link_source='mentions'` edges. Default false (typed edges only). */ + includeMentions?: boolean; + /** Single-source scope. */ + sourceId?: string; + /** Federated scope; traversal stays WITHIN each seed's own source. */ + sourceIds?: string[]; + /** Hard cap on returned candidate nodes. Default 50. */ + limit?: number; +} + // Timeline export interface TimelineEntry { id: number; diff --git a/src/eval/retrieval-quality/harness.ts b/src/eval/retrieval-quality/harness.ts index e39b81452..780bfb1f8 100644 --- a/src/eval/retrieval-quality/harness.ts +++ b/src/eval/retrieval-quality/harness.ts @@ -33,6 +33,15 @@ export interface NamedThingQuestion { /** slugs that must NOT appear in top-k (hard-negative family). */ forbidden?: string[]; notes?: string; + /** + * v0.43 relational metadata (graph-relationship family). Optional, typed + * (not buried in `notes`) so the relational parser/path/determinism tests + * can machine-check the parse. `seed` is the entity the answer relates to; + * `linkTypes` the edge(s) expected; `kind` the archetype. + */ + seed?: string; + linkTypes?: string[]; + kind?: 'who_rel' | 'who_at' | 'connects' | 'intro'; } /** Ranked slugs for a query, best-first. */ @@ -46,6 +55,11 @@ export interface QuestionResult { reciprocal_rank: number; // 0 if no relevant slug in results /** hard-negative only: true when NO forbidden slug appeared in top-3. */ negative_clean?: boolean; + /** v0.43 — fraction of relevant slugs found in top-K (K=3). 0 for hard-negative. */ + recall_at_k: number; + /** v0.43 — fraction of relevant slugs found in top-10. The relational + * headline metric (a relationship query often has several valid answers). */ + recall_at_10: number; } export interface FamilyReport { @@ -54,6 +68,9 @@ export interface FamilyReport { hit_at_1: number; // rate hit_at_3: number; // rate mrr: number; + /** v0.43 — mean recall@K / recall@10 across the family's questions. */ + recall_at_k: number; + recall_at_10: number; } export interface RetrievalQualityReport { @@ -71,14 +88,20 @@ export function scoreQuestion(q: NamedThingQuestion, ranked: string[]): Question const forbidden = new Set(q.forbidden ?? []); const topK = ranked.slice(0, K); const clean = !topK.some(s => forbidden.has(s)); - return { family: q.family, query: q.query, hit_at_1: clean, hit_at_3: clean, reciprocal_rank: clean ? 1 : 0, negative_clean: clean }; + return { family: q.family, query: q.query, hit_at_1: clean, hit_at_3: clean, reciprocal_rank: clean ? 1 : 0, negative_clean: clean, recall_at_k: 0, recall_at_10: 0 }; } const relevant = new Set(q.relevant ?? []); const firstRelevantIdx = ranked.findIndex(s => relevant.has(s)); const hit1 = firstRelevantIdx === 0; const hit3 = firstRelevantIdx >= 0 && firstRelevantIdx < K; const rr = firstRelevantIdx >= 0 ? 1 / (firstRelevantIdx + 1) : 0; - return { family: q.family, query: q.query, hit_at_1: hit1, hit_at_3: hit3, reciprocal_rank: rr }; + const recallAt = (k: number): number => { + if (relevant.size === 0) return 0; + const top = ranked.slice(0, k); + const found = top.filter(s => relevant.has(s)).length; + return found / relevant.size; + }; + return { family: q.family, query: q.query, hit_at_1: hit1, hit_at_3: hit3, reciprocal_rank: rr, recall_at_k: recallAt(K), recall_at_10: recallAt(10) }; } export async function runRetrievalQuality( @@ -106,6 +129,8 @@ export async function runRetrievalQuality( hit_at_1: n ? list.filter(r => r.hit_at_1).length / n : 0, hit_at_3: n ? list.filter(r => r.hit_at_3).length / n : 0, mrr: n ? list.reduce((s, r) => s + r.reciprocal_rank, 0) / n : 0, + recall_at_k: n ? list.reduce((s, r) => s + r.recall_at_k, 0) / n : 0, + recall_at_10: n ? list.reduce((s, r) => s + r.recall_at_10, 0) / n : 0, }); } families.sort((a, b) => a.family.localeCompare(b.family)); diff --git a/test/cache-scope-key.test.ts b/test/cache-scope-key.test.ts new file mode 100644 index 000000000..dac58c239 --- /dev/null +++ b/test/cache-scope-key.test.ts @@ -0,0 +1,42 @@ +/** + * Query-cache scope key (federation hardening). + * + * A federated search reads a different graph than a single-source one, so + * the semantic cache must key them apart. `cacheScopeKey` produces an + * order-independent key for federated scopes and leaves single-source + * brains on their existing key (scalar id or 'default'), so single-source + * cache hit-rate is unchanged. + */ + +import { describe, test, expect } from 'bun:test'; +import { cacheScopeKey } from '../src/core/search/hybrid.ts'; + +describe('cacheScopeKey', () => { + test('unscoped → default (single-source unchanged)', () => { + expect(cacheScopeKey(undefined)).toBe('default'); + expect(cacheScopeKey({})).toBe('default'); + }); + + test('scalar sourceId → itself (single-source unchanged)', () => { + expect(cacheScopeKey({ sourceId: 'host' })).toBe('host'); + }); + + test('federated sourceIds → order-independent set key', () => { + const k1 = cacheScopeKey({ sourceIds: ['team-b', 'team-a', 'host'] }); + const k2 = cacheScopeKey({ sourceIds: ['host', 'team-a', 'team-b'] }); + expect(k1).toBe(k2); // order does not matter + expect(k1).toBe('__set__:host,team-a,team-b'); + }); + + test('different source-sets do NOT share a key', () => { + const a = cacheScopeKey({ sourceIds: ['host', 'team-a'] }); + const b = cacheScopeKey({ sourceIds: ['host', 'team-b'] }); + expect(a).not.toBe(b); + }); + + test('federated set key is distinct from any single scalar key', () => { + const set = cacheScopeKey({ sourceIds: ['host'] }); + const scalar = cacheScopeKey({ sourceId: 'host' }); + expect(set).not.toBe(scalar); // a 1-element set still cannot serve a scalar read + }); +}); diff --git a/test/cross-modal-phase1.test.ts b/test/cross-modal-phase1.test.ts index e699c8789..eaa245540 100644 --- a/test/cross-modal-phase1.test.ts +++ b/test/cross-modal-phase1.test.ts @@ -136,14 +136,15 @@ describe('D2 — knobsHash differs across cross-modal knob values', () => { return resolveSearchMode({ mode: 'balanced' }); } - test('KNOBS_HASH_VERSION is 9 (cross-modal still appended; 8→9 archive-demote #1777)', () => { + test('KNOBS_HASH_VERSION is 10 (cross-modal still appended; 9→10 relational recall)', () => { // v0.35 ladder: 1→2 reranker, 2→3 floor_ratio. v0.36 piggybacks on v=3 // with 7 cross-modal knobs + column/provider context. v0.40.4 (salem) + // v0.39 T21 (master) bump to v=4 for graph_signals + schema-pack fields. // v0.40.3.0 D8 bumps to v=5 (sequenced behind salem's v=4 graph-signals). // v0.41.22.0 (type-unification): 5→6 for alias_resolved post-fusion boost. // T2: 6→7 title_boost. v0.42.3.0: 7→8 autocut. issue #1777: 8→9 archive/ demote. - expect(KNOBS_HASH_VERSION).toBe(9); + // v0.43: 9→10 relational recall arm. + expect(KNOBS_HASH_VERSION).toBe(10); }); test('flipping unified_multimodal changes the hash', () => { diff --git a/test/e2e/engine-parity.test.ts b/test/e2e/engine-parity.test.ts index cce3e3ecc..d03c765c2 100644 --- a/test/e2e/engine-parity.test.ts +++ b/test/e2e/engine-parity.test.ts @@ -544,3 +544,71 @@ describeBoth('Engine parity — Postgres vs PGLite', () => { expect(pg.find((r) => r.slug === 'ep/ec-alice')!.inbound_count).toBe(2); }); }); + +// ── relationalFanout parity (v0.43) ───────────────────────────────────── +async function seedRelational(eng: BrainEngine) { + const pages: Array<[string, 'company' | 'person']> = [ + ['companies/ep-widget', 'company'], + ['companies/ep-other', 'company'], + ['people/ep-inv-a', 'person'], + ['people/ep-inv-b', 'person'], + ['people/ep-emp-c', 'person'], + ['people/ep-mentioner', 'person'], + ]; + for (const [slug, type] of pages) { + await eng.putPage(slug, { type, title: slug, compiled_truth: `${slug} body`, timeline: '' }); + } + await eng.upsertChunks('people/ep-inv-b', [{ + chunk_index: 0, chunk_text: 'b', chunk_source: 'compiled_truth', + embedding: basisEmbedding(2), token_count: 1, + }] satisfies ChunkInput[]); + await eng.addLink('people/ep-inv-a', 'companies/ep-widget', '', 'invested_in', 'manual'); + await eng.addLink('people/ep-inv-b', 'companies/ep-widget', '', 'invested_in', 'manual'); + await eng.addLink('people/ep-emp-c', 'companies/ep-widget', '', 'works_at', 'manual'); + await eng.addLink('people/ep-mentioner', 'companies/ep-widget', '', 'mentions', 'mentions'); + await eng.addLink('people/ep-inv-a', 'companies/ep-other', '', 'invested_in', 'manual'); +} + +describeBoth('Engine parity — relationalFanout', () => { + let pgEngine: BrainEngine; + let pgliteEngine: PGLiteEngine; + + beforeAll(async () => { + pgEngine = await setupDB(); + await seedRelational(pgEngine); + pgliteEngine = new PGLiteEngine(); + await pgliteEngine.connect({}); + await pgliteEngine.initSchema(); + await seedRelational(pgliteEngine); + }, 90_000); + + afterAll(async () => { + await pgliteEngine.disconnect(); + await teardownDB(); + }, 30_000); + + const shape = (rows: Awaited>) => + rows.map(r => `${r.source_id}:${r.slug}:${r.hop}:${r.edge_count}:${r.via_link_types.join(',')}:${r.path.join('>')}:${r.canonical_chunk_id ?? 'null'}`); + + test('typed-edge fan-out is identical across engines', async () => { + const opts = { direction: 'in' as const, linkTypes: ['invested_in'] }; + const pg = await pgEngine.relationalFanout(['companies/ep-widget'], opts); + const pglite = await pgliteEngine.relationalFanout(['companies/ep-widget'], opts); + expect(shape(pg)).toEqual(shape(pglite)); + expect(pg.map(r => r.slug).sort()).toEqual(['people/ep-inv-a', 'people/ep-inv-b']); + }); + + test('type-agnostic + mentions-exclusion identical across engines', async () => { + const pg = await pgEngine.relationalFanout(['companies/ep-widget'], { direction: 'in' }); + const pglite = await pgliteEngine.relationalFanout(['companies/ep-widget'], { direction: 'in' }); + expect(shape(pg)).toEqual(shape(pglite)); + expect(pg.map(r => r.slug)).not.toContain('people/ep-mentioner'); + }); + + test('connects (multi-seed, both) identical across engines', async () => { + const seeds = ['companies/ep-widget', 'companies/ep-other']; + const pg = await pgEngine.relationalFanout(seeds, { direction: 'both' }); + const pglite = await pgliteEngine.relationalFanout(seeds, { direction: 'both' }); + expect(shape(pg)).toEqual(shape(pglite)); + }); +}); diff --git a/test/fixtures/retrieval-quality/relational/corpus.ts b/test/fixtures/retrieval-quality/relational/corpus.ts new file mode 100644 index 000000000..df9388aed --- /dev/null +++ b/test/fixtures/retrieval-quality/relational/corpus.ts @@ -0,0 +1,195 @@ +/** + * Relational benchmark corpus (v0.43). + * + * A small entity graph whose relational answers are LEXICALLY UNRECOVERABLE: + * every page body is generic and NEVER names the entity it relates to. Only + * the typed edge connects them, so keyword + vector retrieval cannot surface + * the answer — isolating the contribution of the typed-edge recall arm. + * + * Canonical source for BOTH the seed loader and the question set, so the + * corpus and the gold answers can't drift. `relational.jsonl` is generated + * from RELATIONAL_QUESTIONS (a drift test pins them equal). + */ + +import type { BrainEngine } from '../../../../src/core/engine.ts'; +import type { ChunkInput } from '../../../../src/core/types.ts'; +import type { NamedThingQuestion } from '../../../../src/eval/retrieval-quality/harness.ts'; + +// edge maps: target entity → entities related to it via that edge type. +// invested_in / led_round point INTO the company (people/funds → company). +const INVESTMENTS: Record = { + 'companies/widget-co': ['people/alice-example', 'people/bob-example', 'funds/fund-a'], + 'companies/acme-co': ['people/carol-example', 'people/alice-example', 'funds/fund-b'], + 'companies/novapay': ['people/bob-example', 'funds/fund-a'], + 'companies/mindbridge': ['people/dave-example', 'people/carol-example'], + 'companies/helio': ['people/erin-example', 'funds/fund-b'], + 'companies/quanta': ['people/frank-example', 'people/alice-example'], + 'companies/zenith': ['people/grace-example', 'people/heidi-example'], + 'companies/orbital': ['people/ivan-example', 'people/alice-example'], +}; +const EMPLOYMENT: Record = { + 'companies/widget-co': ['people/erin-example', 'people/grace-example'], + 'companies/acme-co': ['people/dave-example', 'people/frank-example'], + 'companies/novapay': ['people/grace-example'], + 'companies/helio': ['people/heidi-example'], + 'companies/zenith': ['people/ivan-example'], +}; +const FOUNDED: Record = { + 'companies/mindbridge': ['people/carol-example'], + 'companies/quanta': ['people/frank-example'], + 'companies/widget-co': ['people/ivan-example'], + 'companies/orbital': ['people/grace-example'], +}; +const ADVISES: Record = { + 'companies/novapay': ['people/frank-example'], + 'companies/helio': ['people/alice-example'], +}; + +// Generic, cross-mention-free bodies keyed by slug prefix. +function bodyFor(slug: string): string { + if (slug.startsWith('companies/')) return 'A privately held company operating in its sector. Founded some years ago; details are sparse in this note.'; + if (slug.startsWith('funds/')) return 'An early-stage venture fund. Writes first checks; portfolio is not enumerated here.'; + return 'An individual in the network. Background and current focus are not described in this note.'; +} + +function titleFor(slug: string): string { + const tail = slug.split('/')[1]; + return tail.replace(/-/g, ' ').replace(/\b\w/g, c => c.toUpperCase()); +} + +/** Every distinct slug in the graph. */ +function allSlugs(): string[] { + const set = new Set(); + for (const map of [INVESTMENTS, EMPLOYMENT, FOUNDED, ADVISES]) { + for (const [company, members] of Object.entries(map)) { + set.add(company); + for (const m of members) set.add(m); + } + } + return [...set].sort(); +} + +function pageType(slug: string): 'company' | 'person' { + // funds + people render as person-ish entity pages; companies as company. + return slug.startsWith('companies/') ? 'company' : 'person'; +} + +// Deterministic per-slug basis embedding so pages are properly indexed +// (chunked) like a real brain. The query has no embedding in CI (no API key), +// so vector search can't connect entities anyway — but the pages must carry a +// chunk to be searchable at all. Body text stays generic + cross-mention-free. +// `dim` MUST match the schema's embedding column, which tracks the configured +// gateway default (1280 ZE / 1536 OpenAI) and can shift with shard order — +// so callers probe the real width via probeEmbeddingDim rather than hardcode. +function basisEmbedding(slug: string, dim: number): Float32Array { + let h = 0; + for (let i = 0; i < slug.length; i++) h = (h * 31 + slug.charCodeAt(i)) >>> 0; + const e = new Float32Array(dim); + e[h % dim] = 1.0; + return e; +} + +/** + * Probe the actual `content_chunks.embedding` column width. pgvector stores + * the dimension in `atttypmod` directly. Tests must size fixtures to this, not + * a hardcoded 1536 — the column tracks the gateway default and a prior test in + * the same shard can leave it at 1280 (ZE). Mirrors pglite-engine.test.ts. + */ +export async function probeEmbeddingDim(engine: BrainEngine): Promise { + const db = (engine as unknown as { db: { query: (sql: string) => Promise<{ rows: Array<{ atttypmod: number }> }> } }).db; + const r = await db.query( + `SELECT atttypmod FROM pg_attribute + WHERE attrelid = 'content_chunks'::regclass AND attname = 'embedding'`, + ); + return r.rows[0].atttypmod; +} + +/** Seed the corpus into a fresh brain. Bodies never name related entities. */ +export async function seedRelationalCorpus(engine: BrainEngine): Promise { + const dim = await probeEmbeddingDim(engine); + for (const slug of allSlugs()) { + const body = bodyFor(slug); + await engine.putPage(slug, { + type: pageType(slug), + title: titleFor(slug), + compiled_truth: body, + timeline: '', + }); + await engine.upsertChunks(slug, [{ + chunk_index: 0, + chunk_text: body, + chunk_source: 'compiled_truth', + embedding: basisEmbedding(slug, dim), + token_count: body.split(/\s+/).length, + }] satisfies ChunkInput[]); + } + const addAll = async (map: Record, linkType: string) => { + for (const [company, members] of Object.entries(map)) { + for (const m of members) { + // edge points member → company (member invested_in / works_at / founded / advises company) + await engine.addLink(m, company, '', linkType, 'manual'); + } + } + }; + await addAll(INVESTMENTS, 'invested_in'); + await addAll(EMPLOYMENT, 'works_at'); + await addAll(FOUNDED, 'founded'); + await addAll(ADVISES, 'advises'); +} + +function bareName(slug: string): string { + return slug.split('/')[1]; +} + +/** Generate the gold question set from the same maps used to seed. */ +export function buildRelationalQuestions(): NamedThingQuestion[] { + const qs: NamedThingQuestion[] = []; + + for (const [company, investors] of Object.entries(INVESTMENTS)) { + qs.push({ + family: 'graph-relationship', kind: 'who_rel', seed: company, linkTypes: ['invested_in', 'led_round'], + query: `who invested in ${bareName(company)}`, relevant: investors, + }); + } + for (const [company, employees] of Object.entries(EMPLOYMENT)) { + qs.push({ + family: 'graph-relationship', kind: 'who_rel', seed: company, linkTypes: ['works_at'], + query: `who works at ${bareName(company)}`, relevant: employees, + }); + } + for (const [company, founders] of Object.entries(FOUNDED)) { + qs.push({ + family: 'graph-relationship', kind: 'who_rel', seed: company, linkTypes: ['founded'], + query: `who founded ${bareName(company)}`, relevant: founders, + }); + } + for (const [company, advisors] of Object.entries(ADVISES)) { + qs.push({ + family: 'graph-relationship', kind: 'who_rel', seed: company, linkTypes: ['advises'], + query: `who advises ${bareName(company)}`, relevant: advisors, + }); + } + + // connects: company pairs that share at least one related entity (any edge). + const relatedTo = (company: string): Set => { + const s = new Set(); + for (const map of [INVESTMENTS, EMPLOYMENT, FOUNDED, ADVISES]) for (const m of map[company] ?? []) s.add(m); + return s; + }; + const companies = Object.keys(INVESTMENTS); + for (let i = 0; i < companies.length; i++) { + for (let j = i + 1; j < companies.length; j++) { + const a = companies[i], b = companies[j]; + const shared = [...relatedTo(a)].filter(x => relatedTo(b).has(x)); + if (shared.length === 0) continue; + qs.push({ + family: 'graph-relationship', kind: 'connects', seed: `${a} ↔ ${b}`, linkTypes: undefined, + query: `what connects ${bareName(a)} and ${bareName(b)}`, relevant: shared.sort(), + }); + } + } + + return qs; +} + +export const RELATIONAL_QUESTIONS: NamedThingQuestion[] = buildRelationalQuestions(); diff --git a/test/fixtures/retrieval-quality/relational/relational.jsonl b/test/fixtures/retrieval-quality/relational/relational.jsonl new file mode 100644 index 000000000..06a4bf28f --- /dev/null +++ b/test/fixtures/retrieval-quality/relational/relational.jsonl @@ -0,0 +1,42 @@ +// Relational benchmark — graph-relationship family. GENERATED from +// test/fixtures/retrieval-quality/relational/corpus.ts (canonical source). +// Answers are lexically unrecoverable — only the typed edge connects +// query to answer. Regenerate: bun run scripts/_gen_relational_jsonl.ts +{"family":"graph-relationship","kind":"who_rel","seed":"companies/widget-co","linkTypes":["invested_in","led_round"],"query":"who invested in widget-co","relevant":["people/alice-example","people/bob-example","funds/fund-a"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/acme-co","linkTypes":["invested_in","led_round"],"query":"who invested in acme-co","relevant":["people/carol-example","people/alice-example","funds/fund-b"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/novapay","linkTypes":["invested_in","led_round"],"query":"who invested in novapay","relevant":["people/bob-example","funds/fund-a"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/mindbridge","linkTypes":["invested_in","led_round"],"query":"who invested in mindbridge","relevant":["people/dave-example","people/carol-example"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/helio","linkTypes":["invested_in","led_round"],"query":"who invested in helio","relevant":["people/erin-example","funds/fund-b"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/quanta","linkTypes":["invested_in","led_round"],"query":"who invested in quanta","relevant":["people/frank-example","people/alice-example"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/zenith","linkTypes":["invested_in","led_round"],"query":"who invested in zenith","relevant":["people/grace-example","people/heidi-example"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/orbital","linkTypes":["invested_in","led_round"],"query":"who invested in orbital","relevant":["people/ivan-example","people/alice-example"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/widget-co","linkTypes":["works_at"],"query":"who works at widget-co","relevant":["people/erin-example","people/grace-example"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/acme-co","linkTypes":["works_at"],"query":"who works at acme-co","relevant":["people/dave-example","people/frank-example"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/novapay","linkTypes":["works_at"],"query":"who works at novapay","relevant":["people/grace-example"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/helio","linkTypes":["works_at"],"query":"who works at helio","relevant":["people/heidi-example"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/zenith","linkTypes":["works_at"],"query":"who works at zenith","relevant":["people/ivan-example"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/mindbridge","linkTypes":["founded"],"query":"who founded mindbridge","relevant":["people/carol-example"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/quanta","linkTypes":["founded"],"query":"who founded quanta","relevant":["people/frank-example"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/widget-co","linkTypes":["founded"],"query":"who founded widget-co","relevant":["people/ivan-example"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/orbital","linkTypes":["founded"],"query":"who founded orbital","relevant":["people/grace-example"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/novapay","linkTypes":["advises"],"query":"who advises novapay","relevant":["people/frank-example"]} +{"family":"graph-relationship","kind":"who_rel","seed":"companies/helio","linkTypes":["advises"],"query":"who advises helio","relevant":["people/alice-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/acme-co","query":"what connects widget-co and acme-co","relevant":["people/alice-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/novapay","query":"what connects widget-co and novapay","relevant":["funds/fund-a","people/bob-example","people/grace-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/helio","query":"what connects widget-co and helio","relevant":["people/alice-example","people/erin-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/quanta","query":"what connects widget-co and quanta","relevant":["people/alice-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/zenith","query":"what connects widget-co and zenith","relevant":["people/grace-example","people/ivan-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/orbital","query":"what connects widget-co and orbital","relevant":["people/alice-example","people/grace-example","people/ivan-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/novapay","query":"what connects acme-co and novapay","relevant":["people/frank-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/mindbridge","query":"what connects acme-co and mindbridge","relevant":["people/carol-example","people/dave-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/helio","query":"what connects acme-co and helio","relevant":["funds/fund-b","people/alice-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/quanta","query":"what connects acme-co and quanta","relevant":["people/alice-example","people/frank-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/orbital","query":"what connects acme-co and orbital","relevant":["people/alice-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/novapay ↔ companies/quanta","query":"what connects novapay and quanta","relevant":["people/frank-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/novapay ↔ companies/zenith","query":"what connects novapay and zenith","relevant":["people/grace-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/novapay ↔ companies/orbital","query":"what connects novapay and orbital","relevant":["people/grace-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/helio ↔ companies/quanta","query":"what connects helio and quanta","relevant":["people/alice-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/helio ↔ companies/zenith","query":"what connects helio and zenith","relevant":["people/heidi-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/helio ↔ companies/orbital","query":"what connects helio and orbital","relevant":["people/alice-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/quanta ↔ companies/orbital","query":"what connects quanta and orbital","relevant":["people/alice-example"]} +{"family":"graph-relationship","kind":"connects","seed":"companies/zenith ↔ companies/orbital","query":"what connects zenith and orbital","relevant":["people/grace-example","people/ivan-example"]} diff --git a/test/relational-ab.test.ts b/test/relational-ab.test.ts new file mode 100644 index 000000000..b1c747f32 --- /dev/null +++ b/test/relational-ab.test.ts @@ -0,0 +1,73 @@ +/** + * Relational A/B proof + no-regression gate (PGLite, default CI). + * + * Seeds the lexically-unrecoverable relational corpus, then runs the gold + * question set twice through bare hybridSearch — relational arm OFF vs ON — + * and asserts: + * 1. recall@10 on the graph-relationship family jumps materially with the + * arm on (the answers are unreachable by keyword/vector). + * 2. a non-relational content query returns the SAME results with the arm + * on vs off (the arm is a true no-op off-target — the regression gate). + * + * This is the headline proof artifact for the feature: relational queries + * that the corpus can't surface lexically jump from ~0 to high recall. + */ + +import { describe, test, expect, beforeAll, afterAll } from 'bun:test'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import { hybridSearch } from '../src/core/search/hybrid.ts'; +import { runRetrievalQuality, parseQuestionsJsonl, type SearchFn } from '../src/eval/retrieval-quality/harness.ts'; +import { seedRelationalCorpus, RELATIONAL_QUESTIONS } from './fixtures/retrieval-quality/relational/corpus.ts'; +import { readFileSync } from 'fs'; +import { join } from 'path'; + +let eng: PGLiteEngine; + +beforeAll(async () => { + eng = new PGLiteEngine(); + await eng.connect({}); + await eng.initSchema(); + await seedRelationalCorpus(eng); +}, 60_000); + +afterAll(async () => { await eng.disconnect(); }); + +const searchFnWith = (relationalRetrieval: boolean): SearchFn => async (q) => { + const results = await hybridSearch(eng, q, { limit: 10, relationalRetrieval, expansion: false }); + return results.map(r => r.slug); +}; + +describe('relational A/B', () => { + test('corpus has a meaningful question set', () => { + expect(RELATIONAL_QUESTIONS.length).toBeGreaterThanOrEqual(30); + }); + + test('relational.jsonl matches the corpus module (no drift)', () => { + const path = join(import.meta.dir, 'fixtures/retrieval-quality/relational/relational.jsonl'); + const fromFile = parseQuestionsJsonl(readFileSync(path, 'utf8')); + expect(JSON.stringify(fromFile)).toBe(JSON.stringify(RELATIONAL_QUESTIONS)); + }); + + test('recall@10 lifts materially with the relational arm ON', async () => { + const off = await runRetrievalQuality(RELATIONAL_QUESTIONS, searchFnWith(false)); + const on = await runRetrievalQuality(RELATIONAL_QUESTIONS, searchFnWith(true)); + + const offFam = off.families.find(f => f.family === 'graph-relationship')!; + const onFam = on.families.find(f => f.family === 'graph-relationship')!; + + // Answers are lexically unrecoverable → baseline recall is near zero. + expect(offFam.recall_at_10).toBeLessThan(0.25); + // Typed-edge arm surfaces them → large lift. + expect(onFam.recall_at_10).toBeGreaterThan(0.75); + expect(onFam.recall_at_10 - offFam.recall_at_10).toBeGreaterThan(0.5); + // Hit@3 should also rise sharply. + expect(onFam.hit_at_3).toBeGreaterThan(offFam.hit_at_3); + }, 120_000); + + test('no-regression: non-relational query is identical arm-on vs arm-off', async () => { + const q = 'early-stage venture fund first checks'; + const off = await searchFnWith(false)(q); + const on = await searchFnWith(true)(q); + expect(on).toEqual(off); + }, 60_000); +}); diff --git a/test/relational-fanout.test.ts b/test/relational-fanout.test.ts new file mode 100644 index 000000000..69862ecda --- /dev/null +++ b/test/relational-fanout.test.ts @@ -0,0 +1,115 @@ +/** + * relationalFanout unit tests (PGLite, default CI). + * + * Exercises the typed-edge fan-out SQL directly so the engine method is + * covered even when the DATABASE_URL-gated parity test does not run. Pins: + * typed-edge filtering, mentions-excluded-by-default, deleted_at exclusion, + * hop/edge_count aggregation, canonical_chunk_id, multi-seed + connects, + * and determinism. + */ + +import { describe, test, expect, beforeAll, afterAll } from 'bun:test'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import type { ChunkInput } from '../src/core/types.ts'; +import { probeEmbeddingDim } from './fixtures/retrieval-quality/relational/corpus.ts'; + +function emb(idx: number, dim: number): Float32Array { + const e = new Float32Array(dim); + e[idx % dim] = 1.0; + return e; +} + +let eng: PGLiteEngine; +let DIM = 0; + +beforeAll(async () => { + eng = new PGLiteEngine(); + await eng.connect({}); + await eng.initSchema(); + DIM = await probeEmbeddingDim(eng); // match the schema's column width (1280 ZE / 1536 OpenAI) + + const pages: Array<[string, string, string]> = [ + ['companies/widget-co', 'company', 'Widget Co'], + ['companies/other-co', 'company', 'Other Co'], + ['people/investor-a', 'person', 'Investor A'], + ['people/investor-b', 'person', 'Investor B'], + ['people/employee-c', 'person', 'Employee C'], + ['people/mentioner', 'person', 'Mentioner'], + ['people/deleted-investor', 'person', 'Deleted Investor'], + ]; + for (const [slug, type, title] of pages) { + await eng.putPage(slug, { type: type as 'company' | 'person', title, compiled_truth: `${title} body`, timeline: '' }); + } + // investor-b carries a chunk (→ canonical_chunk_id non-null); investor-a stays chunkless. + const chunk: ChunkInput[] = [{ chunk_index: 0, chunk_text: 'b', chunk_source: 'compiled_truth', embedding: emb(2, DIM), token_count: 1 }]; + await eng.upsertChunks('people/investor-b', chunk); + + // Edges into widget-co. + await eng.addLink('people/investor-a', 'companies/widget-co', '', 'invested_in', 'manual'); + await eng.addLink('people/investor-b', 'companies/widget-co', '', 'invested_in', 'manual'); + await eng.addLink('people/employee-c', 'companies/widget-co', '', 'works_at', 'manual'); + await eng.addLink('people/mentioner', 'companies/widget-co', '', 'mentions', 'mentions'); + await eng.addLink('people/deleted-investor', 'companies/widget-co', '', 'invested_in', 'manual'); + // investor-a also invested in other-co → widget-co and other-co connect via investor-a. + await eng.addLink('people/investor-a', 'companies/other-co', '', 'invested_in', 'manual'); + + // Soft-delete one investor; it must never surface. + await eng.executeRaw(`UPDATE pages SET deleted_at = now() WHERE slug = $1`, ['people/deleted-investor']); +}, 60_000); + +afterAll(async () => { + await eng.disconnect(); +}); + +describe('relationalFanout', () => { + test('typed-edge: who invested in widget-co', async () => { + const rows = await eng.relationalFanout(['companies/widget-co'], { direction: 'in', linkTypes: ['invested_in'] }); + const slugs = rows.map(r => r.slug).sort(); + expect(slugs).toEqual(['people/investor-a', 'people/investor-b']); + for (const r of rows) { + expect(r.hop).toBe(1); + expect(r.via_link_types).toEqual(['invested_in']); + } + }); + + test('deleted pages are excluded', async () => { + const rows = await eng.relationalFanout(['companies/widget-co'], { direction: 'in', linkTypes: ['invested_in'] }); + expect(rows.map(r => r.slug)).not.toContain('people/deleted-investor'); + }); + + test('mentions excluded by default, included on opt-in', async () => { + const off = await eng.relationalFanout(['companies/widget-co'], { direction: 'in' }); + expect(off.map(r => r.slug)).not.toContain('people/mentioner'); + // type-agnostic also picks up the works_at neighbor + expect(off.map(r => r.slug).sort()).toEqual(['people/employee-c', 'people/investor-a', 'people/investor-b']); + + const on = await eng.relationalFanout(['companies/widget-co'], { direction: 'in', includeMentions: true }); + expect(on.map(r => r.slug)).toContain('people/mentioner'); + }); + + test('canonical_chunk_id: non-null for chunked page, null for chunkless', async () => { + const rows = await eng.relationalFanout(['companies/widget-co'], { direction: 'in', linkTypes: ['invested_in'] }); + const a = rows.find(r => r.slug === 'people/investor-a')!; + const b = rows.find(r => r.slug === 'people/investor-b')!; + expect(a.canonical_chunk_id).toBeNull(); + expect(b.canonical_chunk_id).not.toBeNull(); + }); + + test('connects: shared midpoint reachable from both seeds with a path', async () => { + const rows = await eng.relationalFanout(['companies/widget-co', 'companies/other-co'], { direction: 'both' }); + const a = rows.find(r => r.slug === 'people/investor-a'); + expect(a).toBeDefined(); + expect(a!.path.length).toBeGreaterThanOrEqual(2); // [seed, ..., node] + expect(a!.path[a!.path.length - 1]).toBe('people/investor-a'); + }); + + test('empty seeds → []', async () => { + expect(await eng.relationalFanout([])).toEqual([]); + }); + + test('deterministic: same call twice is byte-identical', async () => { + const r1 = await eng.relationalFanout(['companies/widget-co'], { direction: 'in' }); + const r2 = await eng.relationalFanout(['companies/widget-co'], { direction: 'in' }); + expect(JSON.stringify(r1)).toBe(JSON.stringify(r2)); + }); +}); diff --git a/test/relational-intent.test.ts b/test/relational-intent.test.ts new file mode 100644 index 000000000..813969bab --- /dev/null +++ b/test/relational-intent.test.ts @@ -0,0 +1,136 @@ +/** + * Relational-query parser unit tests. + * + * Covers the four archetypes, the no-match path, the precision-first + * false-positive guard, schema-pack vocab extension, subset validation, and + * the length bound (ReDoS surface). + */ + +import { describe, test, expect } from 'bun:test'; +import { + parseRelationalQuery, + validateVocab, + KNOWN_LINK_TYPES, + type RelationVocab, +} from '../src/core/search/relational-intent.ts'; + +describe('parseRelationalQuery — archetypes', () => { + test('who_rel: who invested in widget-co', () => { + const r = parseRelationalQuery('who invested in widget-co'); + expect(r).not.toBeNull(); + expect(r!.kind).toBe('who_rel'); + expect(r!.seeds).toEqual(['widget-co']); + expect(r!.linkTypes).toEqual(['invested_in', 'led_round']); + expect(r!.direction).toBe('in'); + }); + + test('who_rel: who founded acme (article stripped)', () => { + const r = parseRelationalQuery('who founded the acme corporation?'); + expect(r!.kind).toBe('who_rel'); + expect(r!.seeds).toEqual(['acme corporation']); + expect(r!.linkTypes).toEqual(['founded']); + }); + + test('who_at: who at acme works on payments', () => { + const r = parseRelationalQuery('who at acme works on payments'); + expect(r!.kind).toBe('who_at'); + expect(r!.seeds).toEqual(['acme']); + expect(r!.linkTypes).toEqual(['works_at']); + expect(r!.direction).toBe('in'); + }); + + test('intro: who introduced me to alice-example (type-agnostic)', () => { + const r = parseRelationalQuery('who introduced me to alice-example?'); + expect(r!.kind).toBe('intro'); + expect(r!.seeds).toEqual(['alice-example']); + expect(r!.linkTypes).toBeNull(); + expect(r!.direction).toBe('both'); + }); + + test('connects: what connects fund-a and fund-b (two seeds)', () => { + const r = parseRelationalQuery('what connects fund-a and fund-b'); + expect(r!.kind).toBe('connects'); + expect(r!.seeds).toEqual(['fund-a', 'fund-b']); + expect(r!.linkTypes).toBeNull(); + expect(r!.direction).toBe('both'); + }); + + test('connects: how are X and Y related', () => { + const r = parseRelationalQuery('how are widget-co and acme related'); + expect(r!.kind).toBe('connects'); + expect(r!.seeds).toEqual(['widget-co', 'acme']); + }); + + test('outgoing: what did alice invest in', () => { + const r = parseRelationalQuery('what did alice invest in'); + expect(r!.kind).toBe('who_rel'); + expect(r!.seeds).toEqual(['alice']); + expect(r!.direction).toBe('out'); + }); +}); + +describe('parseRelationalQuery — precision-first / no-match', () => { + test('non-relational content query → null', () => { + expect(parseRelationalQuery('what is the capital structure of a seed round')).toBeNull(); + expect(parseRelationalQuery('notes from the offsite')).toBeNull(); + expect(parseRelationalQuery('summarize the q3 board deck')).toBeNull(); + }); + + test('false-positive: "who invested TIME in learning Rust" does NOT match', () => { + // "invested time in" is not "invested in" — adjacency guard. + expect(parseRelationalQuery('who invested time in learning Rust')).toBeNull(); + }); + + test('pronoun / stopword seed is rejected', () => { + expect(parseRelationalQuery('who invested in it')).toBeNull(); + expect(parseRelationalQuery('who founded them?')).toBeNull(); + }); + + test('empty / overlong input → null', () => { + expect(parseRelationalQuery('')).toBeNull(); + expect(parseRelationalQuery('who invested in ' + 'x'.repeat(600))).toBeNull(); + }); +}); + +describe('schema-pack vocab extension (D2=B)', () => { + const vocab: RelationVocab = { + extraVerbs: [{ verb: 'related to|associated with', linkTypes: ['related_to'], direction: 'both' }], + }; + + test('extra verb extends detection', () => { + expect(parseRelationalQuery('who related to widget-co', vocab)!.linkTypes).toEqual(['related_to']); + // same query without the pack does NOT match the extra verb + expect(parseRelationalQuery('who related to widget-co')).toBeNull(); + }); + + test('validateVocab passes for known types', () => { + expect(() => validateVocab(vocab)).not.toThrow(); + }); + + test('validateVocab throws on unknown link_type', () => { + expect(() => + validateVocab({ extraVerbs: [{ verb: 'pwns', linkTypes: ['not_a_real_edge'], direction: 'in' }] }), + ).toThrow(/unknown link_type/); + }); +}); + +describe('default bank emits only known link types (no drift)', () => { + test('every parsed linkType is a subset of KNOWN_LINK_TYPES', () => { + const queries = [ + 'who invested in widget-co', + 'who founded acme', + 'who advises bob-example', + 'who works at acme', + 'who at acme leads payments', + 'what did alice invest in', + 'where does alice work', + ]; + for (const q of queries) { + const r = parseRelationalQuery(q); + if (!r || r.linkTypes === null) continue; + for (const lt of r.linkTypes) { + expect(KNOWN_LINK_TYPES.has(lt)).toBe(true); + } + } + }); +}); diff --git a/test/relational-recall.test.ts b/test/relational-recall.test.ts new file mode 100644 index 000000000..61eff1777 --- /dev/null +++ b/test/relational-recall.test.ts @@ -0,0 +1,72 @@ +/** + * Relational recall arm integration tests (PGLite, default CI). + * + * End-to-end through buildRelationalArm: parse → resolve seed → fanout → + * hydrate. Pins the lexically-unrecoverable win (the investor page never names + * the company; only the invested_in edge connects them), the non-relational + * no-op, attribution stamping, and fail-open. + */ + +import { describe, test, expect, beforeAll, afterAll } from 'bun:test'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import { buildRelationalArm } from '../src/core/search/relational-recall.ts'; +import { probeEmbeddingDim } from './fixtures/retrieval-quality/relational/corpus.ts'; +import type { ChunkInput } from '../src/core/types.ts'; + +let eng: PGLiteEngine; + +beforeAll(async () => { + eng = new PGLiteEngine(); + await eng.connect({}); + await eng.initSchema(); + const dim = await probeEmbeddingDim(eng); // match schema column width (1280 ZE / 1536 OpenAI) + + await eng.putPage('companies/widget-co', { type: 'company', title: 'Widget Co', compiled_truth: 'A payments company.', timeline: '' }); + // The investor's body deliberately NEVER mentions Widget Co — only the edge connects them. + await eng.putPage('people/alice-example', { type: 'person', title: 'Alice Example', compiled_truth: 'Alice is a seed-stage investor based in Lisbon.', timeline: '' }); + await eng.upsertChunks('people/alice-example', [{ + chunk_index: 0, chunk_text: 'Alice is a seed-stage investor based in Lisbon.', + chunk_source: 'compiled_truth', embedding: new Float32Array(dim), token_count: 8, + }] satisfies ChunkInput[]); + await eng.addLink('people/alice-example', 'companies/widget-co', '', 'invested_in', 'manual'); +}, 60_000); + +afterAll(async () => { await eng.disconnect(); }); + +describe('buildRelationalArm', () => { + test('surfaces the edge answer that lexical search would miss', async () => { + const list = await buildRelationalArm(eng, 'who invested in widget-co'); + const alice = list.find(r => r.slug === 'people/alice-example'); + expect(alice).toBeDefined(); + expect(alice!.relational_via_link_types).toEqual(['invested_in']); + expect(alice!.relational_hop).toBe(1); + expect(alice!.relational_seed).toBe('companies/widget-co'); + // chunk-backed page → reinforces a REAL chunk id (not synthetic 0). + expect(alice!.chunk_id).toBeGreaterThan(0); + }); + + test('non-relational query is a pure no-op', async () => { + const meta: { fired?: boolean } = {}; + const list = await buildRelationalArm(eng, 'summary of the payments roadmap', { onMeta: m => { meta.fired = m.fired; } }); + expect(list).toEqual([]); + expect(meta.fired).toBe(false); + }); + + test('unresolvable seed → no-op (never traverse from a guess)', async () => { + const list = await buildRelationalArm(eng, 'who invested in nonexistent-phantom-xyz'); + expect(list).toEqual([]); + }); + + test('fail-open: fanout error returns [] + errored meta, never throws', async () => { + const original = eng.relationalFanout.bind(eng); + let captured: { errored?: boolean } = {}; + eng.relationalFanout = async () => { throw new Error('boom'); }; + try { + const list = await buildRelationalArm(eng, 'who invested in widget-co', { onMeta: m => { captured = m; } }); + expect(list).toEqual([]); + expect(captured.errored).toBe(true); + } finally { + eng.relationalFanout = original; + } + }); +}); diff --git a/test/rrf-source-key.test.ts b/test/rrf-source-key.test.ts new file mode 100644 index 000000000..245682d25 --- /dev/null +++ b/test/rrf-source-key.test.ts @@ -0,0 +1,70 @@ +/** + * RRF fusion key is source-aware (federation hardening). + * + * Pre-fix the RRF/dedup key was `slug:chunk_id`, which collapsed two + * same-slug pages living in different federated sources into one fusion + * entry — a cross-source recall bug. The key is now + * `(source_id, slug, chunk_id)`, matching `dedup.ts:pageKey`'s composite + * discipline at chunk granularity. These tests pin that behavior for both + * `rrfFusion` and `rrfFusionWeighted` and confirm single-source ranking is + * unchanged. + */ + +import { describe, test, expect } from 'bun:test'; +import { rrfFusion, rrfFusionWeighted } from '../src/core/search/hybrid.ts'; +import type { SearchResult } from '../src/core/types.ts'; + +function makeResult(overrides: Partial = {}): SearchResult { + return { + slug: 'test-page', + page_id: 1, + title: 'Test', + type: 'concept', + chunk_text: 'unique chunk text', + chunk_source: 'timeline', // avoid compiled_truth 2x boost noise + chunk_id: 1, + chunk_index: 0, + score: 0.5, + stale: false, + ...overrides, + }; +} + +describe('RRF key is source-aware', () => { + test('same slug + chunk_id in DIFFERENT sources do NOT collapse', () => { + const a = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-a', chunk_text: 'x' }); + const b = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-b', chunk_text: 'y' }); + const out = rrfFusion([[a, b]], 60); + expect(out.length).toBe(2); + expect(new Set(out.map(r => r.source_id))).toEqual(new Set(['team-a', 'team-b'])); + }); + + test('same slug + chunk_id + SAME source DO collapse (reinforce across lists)', () => { + const a = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-a', chunk_text: 'x' }); + const b = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-a', chunk_text: 'x' }); + const out = rrfFusion([[a], [b]], 60); + expect(out.length).toBe(1); + }); + + test('single-source (no source_id → "default") ranking unchanged', () => { + const a = makeResult({ slug: 'a', chunk_id: 1, chunk_text: 'aaa' }); + const b = makeResult({ slug: 'b', chunk_id: 2, chunk_text: 'bbb' }); + const out = rrfFusion([[a, b]], 60); + expect(out.length).toBe(2); + expect(out[0].slug).toBe('a'); // rank-0 wins, as before the key change + }); + + test('chunkless rows (null chunk_id) stay distinct per source via text prefix', () => { + const a = makeResult({ slug: 'people/alice', chunk_id: undefined as unknown as number, source_id: 'team-a', chunk_text: 'alice page' }); + const b = makeResult({ slug: 'people/alice', chunk_id: undefined as unknown as number, source_id: 'team-b', chunk_text: 'alice page' }); + const out = rrfFusion([[a, b]], 60); + expect(out.length).toBe(2); + }); + + test('weighted RRF is also source-aware', () => { + const a = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-a' }); + const b = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-b' }); + const out = rrfFusionWeighted([{ list: [a, b], k: 60 }]); + expect(out.length).toBe(2); + }); +}); diff --git a/test/search-alias-resolved-boost.test.ts b/test/search-alias-resolved-boost.test.ts index 8c9eb1110..e67bb0300 100644 --- a/test/search-alias-resolved-boost.test.ts +++ b/test/search-alias-resolved-boost.test.ts @@ -89,7 +89,7 @@ describe('alias_resolved boost stage', () => { }); describe('KNOBS_HASH_VERSION', () => { - it('is 9 (8→9 archive-demote invalidates archive-excluded cache rows, #1777)', () => { - expect(KNOBS_HASH_VERSION).toBe(9); + it('is 10 (9→10 relational recall arm invalidates rel-off cache rows, v0.43)', () => { + expect(KNOBS_HASH_VERSION).toBe(10); }); }); diff --git a/test/search-mode.test.ts b/test/search-mode.test.ts index cec3ba4b1..c2a3516e1 100644 --- a/test/search-mode.test.ts +++ b/test/search-mode.test.ts @@ -77,6 +77,9 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => { // v0.42.3.0 — autocut OFF for conservative (no reranker). autocut: false, autocut_jump: 0.2, + // v0.43 — relational recall OFF for conservative. + relationalRetrieval: false, + relational_retrieval_depth: 2, }); }); @@ -106,6 +109,9 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => { // v0.42.3.0 — autocut ON. autocut: true, autocut_jump: 0.2, + // v0.43 — relational recall ON for balanced. + relationalRetrieval: true, + relational_retrieval_depth: 2, }); }); @@ -133,6 +139,9 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => { // v0.42.3.0 — autocut ON. autocut: true, autocut_jump: 0.2, + // v0.43 — relational recall ON for tokenmax. + relationalRetrieval: true, + relational_retrieval_depth: 2, }); }); @@ -392,7 +401,9 @@ describe('knobsHash determinism + cross-mode separation (CDX-4)', () => { // archive/ demote (search-exclude policy change isn't in the hash, so the // version bump is what invalidates archive-excluded cache rows). A query // must not be served from a cache row written before the policy change. - expect(KNOBS_HASH_VERSION).toBe(9); + // v0.43: bumped 9→10 for the relational recall arm (rel=/reld=) — a + // relational-on write must not be served to a relational-off lookup. + expect(KNOBS_HASH_VERSION).toBe(10); }); test('T1 (codex): floor_ratio set vs unset produces DIFFERENT hashes (cache contamination prevention)', () => { @@ -557,8 +568,8 @@ describe('v0.40.4 — graph_signals knob', () => { }); describe('v0.42.3.0 — autocut knobs', () => { - test('KNOBS_HASH_VERSION is 9 (8→9 archive-demote, issue #1777)', () => { - expect(KNOBS_HASH_VERSION).toBe(9); + test('KNOBS_HASH_VERSION is 10 (9→10 relational recall arm, v0.43)', () => { + expect(KNOBS_HASH_VERSION).toBe(10); }); test('bundle defaults: conservative off, balanced/tokenmax on @0.20', () => { @@ -621,3 +632,44 @@ describe('v0.42.3.0 — autocut knobs', () => { expect(attr.value).toBe(false); }); }); + +describe('v0.43 — relational recall knobs', () => { + test('bundle defaults: conservative off, balanced/tokenmax on; depth 2', () => { + expect(MODE_BUNDLES.conservative.relationalRetrieval).toBe(false); + expect(MODE_BUNDLES.balanced.relationalRetrieval).toBe(true); + expect(MODE_BUNDLES.tokenmax.relationalRetrieval).toBe(true); + for (const m of ['conservative', 'balanced', 'tokenmax'] as const) { + expect(MODE_BUNDLES[m].relational_retrieval_depth).toBe(2); + } + }); + + test('per-call relationalRetrieval:false overrides bundle/config true', () => { + // bundle default true (balanced); per-call false wins + expect(resolveSearchMode({ mode: 'balanced', perCall: { relationalRetrieval: false } }).relationalRetrieval).toBe(false); + // config override true on conservative (bundle false); then per-call false beats config + expect( + resolveSearchMode({ mode: 'conservative', overrides: { relationalRetrieval: true }, perCall: { relationalRetrieval: false } }).relationalRetrieval, + ).toBe(false); + // config override alone flips conservative on + expect(resolveSearchMode({ mode: 'conservative', overrides: { relationalRetrieval: true } }).relationalRetrieval).toBe(true); + }); + + test('loadOverridesFromConfig reads search.relational_retrieval(+_depth)', () => { + const ov = loadOverridesFromConfig({ 'search.relational_retrieval': 'true', 'search.relational_retrieval_depth': '3' }); + expect(ov.relationalRetrieval).toBe(true); + expect(ov.relational_retrieval_depth).toBe(3); + // out-of-range depth is ignored (falls through to bundle) + expect(loadOverridesFromConfig({ 'search.relational_retrieval_depth': '9' }).relational_retrieval_depth).toBeUndefined(); + }); + + test('SEARCH_MODE_CONFIG_KEYS includes the relational keys', () => { + expect(SEARCH_MODE_CONFIG_KEYS).toContain('search.relational_retrieval'); + expect(SEARCH_MODE_CONFIG_KEYS).toContain('search.relational_retrieval_depth'); + }); + + test('knobsHash: relational-on vs off differ (cache isolation)', () => { + const on = knobsHash(resolveSearchMode({ mode: 'balanced' })); // relational true + const off = knobsHash(resolveSearchMode({ mode: 'balanced', perCall: { relationalRetrieval: false } })); + expect(on).not.toBe(off); + }); +}); diff --git a/test/search/knobs-hash-reranker.test.ts b/test/search/knobs-hash-reranker.test.ts index fac2f23eb..509e98249 100644 --- a/test/search/knobs-hash-reranker.test.ts +++ b/test/search/knobs-hash-reranker.test.ts @@ -43,7 +43,7 @@ function baseKnobs(): ResolvedSearchKnobs { } describe('KNOBS_HASH_VERSION + version invariants', () => { - test('version is 9 (…; 6→7 title_boost; 7→8 autocut; 8→9 archive-demote #1777)', () => { + test('version is 10 (…; 7→8 autocut; 8→9 archive-demote #1777; 9→10 relational recall)', () => { // v0.35.0.0: 1→2 to fold reranker fields. v0.35.6.0: 2→3 to fold // floor_ratio. v0.36 wave: piggybacks on v=3 with 7 cross-modal knobs // (D2) PLUS column + provider context (D8/CDX-2 cross-column isolation). @@ -57,7 +57,8 @@ describe('KNOBS_HASH_VERSION + version invariants', () => { // cannot leak past the new stage. T2: 6→7 title_boost. v0.42.3.0: 7→8 // autocut. issue #1777: 8→9 archive/ demote (search-exclude policy change // isn't in the hash, so the bump invalidates archive-excluded cache rows). - expect(KNOBS_HASH_VERSION).toBe(9); + // v0.43: 9→10 relational recall arm (rel=/reld=). + expect(KNOBS_HASH_VERSION).toBe(10); }); test('hash is 16 hex chars regardless of reranker config', () => {