v0.42.34.0 feat(search): typed-edge relational retrieval — relationship questions get relationship answers (#1959)

* feat(search): deterministic relational-query parser

Pure, ReDoS-bounded parser that detects relationship queries ("who invested
in X", "who at X works on Y", "who introduced me to X", "what connects A and
B") and maps them to typed edges. Schema-pack-extensible vocab with subset
validation against the link types ingest produces, so query-side and
ingest-side relation vocabularies can't drift. No-match / pronoun-seed /
adjacency guards keep it precision-first (a candidate only; the arm fires
only when a real seed also resolves).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(engine): relationalFanout typed-edge fan-out (both engines)

Generalizes traversePaths to a SEED ARRAY, aggregating to ranked NODES
(shortest hop, edge-richness count, via-link-types, shortest connecting
path, canonical chunk id) instead of edges. Within-source traversal (never
crosses a source boundary even across a federated scope), link_source=
'mentions' excluded by default, deleted_at filtered at seed + every neighbor
+ every node, bounded depth (<=3) + candidate cap. Adds RelationalFanoutRow
/ RelationalFanoutOpts + the relational SearchResult/SearchOpts fields to
types.

Lands in lockstep in postgres + pglite engines, pinned by a DATABASE_URL-
gated parity block in engine-parity.test.ts; a PGLite unit test exercises
the SQL (typed-edge filter, mentions exclusion, deleted_at, canonical chunk,
multi-seed connects, determinism) in default CI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(search): relational recall arm + federation key hardening

Wires edge-derived candidates into bare hybridSearch as a FOURTH RRF arm
(relational-recall.ts): parse the original query -> scope-aware,
confidence-gated seed resolution (drops fallback_slugify; never traverses
from a guess) -> relationalFanout -> batch-hydrate, reinforcing each page's
REAL canonical chunk (page-level key for chunkless entity pages) ->
--explain attribution + fail-open audit row. Text-only (no-op in image
mode); pure no-op for non-relational queries; rides every downstream stage
(cosine, post-fusion boosts, dedup, reranker, autocut, token budget).

Mode wiring: relationalRetrieval + relational_retrieval_depth knobs
(conservative off; balanced/tokenmax on; depth 2), per-call thread-through
in both bare + cached paths, KNOBS_HASH_VERSION 9->10 (rel=/reld=), config
keys, modes-dashboard descriptions, and a `relational` param on the query op.

Federation hardening (structural, engine-wide): the RRF/dedup key now
carries source_id via a shared rrfKey() (fixes a latent cross-source
collapse where same-slug pages in different sources merged), and the query
cache scopes by a canonical source-set key (cacheScopeKey) so a federated
read can't be served a single-source row.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(eval): relational benchmark + recall@k harness metrics

NamedThingBench harness gains recall@k / recall@10 (the relational headline
metric) on QuestionResult + FamilyReport, plus typed seed/linkTypes/kind on
NamedThingQuestion so the graph-relationship family is machine-checkable.

Adds the relational benchmark corpus (test/fixtures/retrieval-quality/
relational/): a small entity graph whose answers are LEXICALLY UNRECOVERABLE
— every page body is generic and never names the entity it relates to, so
only the typed edge connects query to answer. corpus.ts is the canonical
source for both the seed loader and the 38-question gold set; relational.jsonl
is generated from it (a drift test pins them equal).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(eval): relational A/B proof + arm fires on all retrieval paths

Fixes the integration bug the eval caught: the relational arm was only
injected on the main RRF path, so it silently did nothing whenever vector
was unavailable — no embedding provider configured (the default in many
deployments) OR embed failure. The arm is now built once and fused via RRF
on ALL THREE hybridSearch return paths (no-embedding-provider, embed-failed
keyword fallback, main path). Without this it would have been dead in
exactly the setups that most need it.

Adds `gbrain eval retrieval-quality --ab-relational`: runs the gold set
twice (arm off vs on) in a fixed mode and reports the graph-relationship
recall@10 lift + Hit@3 + latency add. The CI A/B test pins the headline
result — recall@10 jumps from <25% (lexically unrecoverable) to >75% with
the arm on — and a non-relational query returns identical results arm-on vs
off (the no-op / no-regression gate).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.42.34.0)

Relational retrieval feature: typed-edge recall arm + federation key
hardening. Also updates the KNOBS_HASH_VERSION 9→10 assertions across the
remaining search test files (the bump invalidates relational-off cache rows).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: document typed-edge relational retrieval (v0.42.34.0)

CLAUDE.md Search Mode: add relationalRetrieval to the knob table, the
knobs_hash v=9→10 note, and a relational-retrieval summary. RETRIEVAL.md:
add the relational recall arm to the pipeline diagram. Regenerate llms
bundles (build:llms).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(test): size relational fixtures to the actual embedding-column dim

CI shard runs with the ZeroEntropy gateway default (1280-d), but the
relational test fixtures hardcoded 1536-d embeddings, so chunk inserts were
rejected with "expected 1280 dimensions, not 1536" (CheckExpectedDim) — the
`test (6)` shard + `test-status` failures. The column width tracks the
configured gateway default and can shift with shard order, so fixtures now
probe `content_chunks.embedding`'s actual `atttypmod` after initSchema and
size embeddings to it (the pglite-engine.test.ts pattern), via a shared
`probeEmbeddingDim` helper. Verified passing at a forced 1280-d column.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-06-07 22:05:31 -07:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 7cf46230e5
commit 099d9a8f55
31 changed files with 1895 additions and 28 deletions
+42
View File
@@ -0,0 +1,42 @@
/**
* Query-cache scope key (federation hardening).
*
* A federated search reads a different graph than a single-source one, so
* the semantic cache must key them apart. `cacheScopeKey` produces an
* order-independent key for federated scopes and leaves single-source
* brains on their existing key (scalar id or 'default'), so single-source
* cache hit-rate is unchanged.
*/
import { describe, test, expect } from 'bun:test';
import { cacheScopeKey } from '../src/core/search/hybrid.ts';
describe('cacheScopeKey', () => {
test('unscoped → default (single-source unchanged)', () => {
expect(cacheScopeKey(undefined)).toBe('default');
expect(cacheScopeKey({})).toBe('default');
});
test('scalar sourceId → itself (single-source unchanged)', () => {
expect(cacheScopeKey({ sourceId: 'host' })).toBe('host');
});
test('federated sourceIds → order-independent set key', () => {
const k1 = cacheScopeKey({ sourceIds: ['team-b', 'team-a', 'host'] });
const k2 = cacheScopeKey({ sourceIds: ['host', 'team-a', 'team-b'] });
expect(k1).toBe(k2); // order does not matter
expect(k1).toBe('__set__:host,team-a,team-b');
});
test('different source-sets do NOT share a key', () => {
const a = cacheScopeKey({ sourceIds: ['host', 'team-a'] });
const b = cacheScopeKey({ sourceIds: ['host', 'team-b'] });
expect(a).not.toBe(b);
});
test('federated set key is distinct from any single scalar key', () => {
const set = cacheScopeKey({ sourceIds: ['host'] });
const scalar = cacheScopeKey({ sourceId: 'host' });
expect(set).not.toBe(scalar); // a 1-element set still cannot serve a scalar read
});
});
+3 -2
View File
@@ -136,14 +136,15 @@ describe('D2 — knobsHash differs across cross-modal knob values', () => {
return resolveSearchMode({ mode: 'balanced' });
}
test('KNOBS_HASH_VERSION is 9 (cross-modal still appended; 8→9 archive-demote #1777)', () => {
test('KNOBS_HASH_VERSION is 10 (cross-modal still appended; 9→10 relational recall)', () => {
// v0.35 ladder: 1→2 reranker, 2→3 floor_ratio. v0.36 piggybacks on v=3
// with 7 cross-modal knobs + column/provider context. v0.40.4 (salem) +
// v0.39 T21 (master) bump to v=4 for graph_signals + schema-pack fields.
// v0.40.3.0 D8 bumps to v=5 (sequenced behind salem's v=4 graph-signals).
// v0.41.22.0 (type-unification): 5→6 for alias_resolved post-fusion boost.
// T2: 6→7 title_boost. v0.42.3.0: 7→8 autocut. issue #1777: 8→9 archive/ demote.
expect(KNOBS_HASH_VERSION).toBe(9);
// v0.43: 9→10 relational recall arm.
expect(KNOBS_HASH_VERSION).toBe(10);
});
test('flipping unified_multimodal changes the hash', () => {
+68
View File
@@ -544,3 +544,71 @@ describeBoth('Engine parity — Postgres vs PGLite', () => {
expect(pg.find((r) => r.slug === 'ep/ec-alice')!.inbound_count).toBe(2);
});
});
// ── relationalFanout parity (v0.43) ─────────────────────────────────────
async function seedRelational(eng: BrainEngine) {
const pages: Array<[string, 'company' | 'person']> = [
['companies/ep-widget', 'company'],
['companies/ep-other', 'company'],
['people/ep-inv-a', 'person'],
['people/ep-inv-b', 'person'],
['people/ep-emp-c', 'person'],
['people/ep-mentioner', 'person'],
];
for (const [slug, type] of pages) {
await eng.putPage(slug, { type, title: slug, compiled_truth: `${slug} body`, timeline: '' });
}
await eng.upsertChunks('people/ep-inv-b', [{
chunk_index: 0, chunk_text: 'b', chunk_source: 'compiled_truth',
embedding: basisEmbedding(2), token_count: 1,
}] satisfies ChunkInput[]);
await eng.addLink('people/ep-inv-a', 'companies/ep-widget', '', 'invested_in', 'manual');
await eng.addLink('people/ep-inv-b', 'companies/ep-widget', '', 'invested_in', 'manual');
await eng.addLink('people/ep-emp-c', 'companies/ep-widget', '', 'works_at', 'manual');
await eng.addLink('people/ep-mentioner', 'companies/ep-widget', '', 'mentions', 'mentions');
await eng.addLink('people/ep-inv-a', 'companies/ep-other', '', 'invested_in', 'manual');
}
describeBoth('Engine parity — relationalFanout', () => {
let pgEngine: BrainEngine;
let pgliteEngine: PGLiteEngine;
beforeAll(async () => {
pgEngine = await setupDB();
await seedRelational(pgEngine);
pgliteEngine = new PGLiteEngine();
await pgliteEngine.connect({});
await pgliteEngine.initSchema();
await seedRelational(pgliteEngine);
}, 90_000);
afterAll(async () => {
await pgliteEngine.disconnect();
await teardownDB();
}, 30_000);
const shape = (rows: Awaited<ReturnType<BrainEngine['relationalFanout']>>) =>
rows.map(r => `${r.source_id}:${r.slug}:${r.hop}:${r.edge_count}:${r.via_link_types.join(',')}:${r.path.join('>')}:${r.canonical_chunk_id ?? 'null'}`);
test('typed-edge fan-out is identical across engines', async () => {
const opts = { direction: 'in' as const, linkTypes: ['invested_in'] };
const pg = await pgEngine.relationalFanout(['companies/ep-widget'], opts);
const pglite = await pgliteEngine.relationalFanout(['companies/ep-widget'], opts);
expect(shape(pg)).toEqual(shape(pglite));
expect(pg.map(r => r.slug).sort()).toEqual(['people/ep-inv-a', 'people/ep-inv-b']);
});
test('type-agnostic + mentions-exclusion identical across engines', async () => {
const pg = await pgEngine.relationalFanout(['companies/ep-widget'], { direction: 'in' });
const pglite = await pgliteEngine.relationalFanout(['companies/ep-widget'], { direction: 'in' });
expect(shape(pg)).toEqual(shape(pglite));
expect(pg.map(r => r.slug)).not.toContain('people/ep-mentioner');
});
test('connects (multi-seed, both) identical across engines', async () => {
const seeds = ['companies/ep-widget', 'companies/ep-other'];
const pg = await pgEngine.relationalFanout(seeds, { direction: 'both' });
const pglite = await pgliteEngine.relationalFanout(seeds, { direction: 'both' });
expect(shape(pg)).toEqual(shape(pglite));
});
});
+195
View File
@@ -0,0 +1,195 @@
/**
* Relational benchmark corpus (v0.43).
*
* A small entity graph whose relational answers are LEXICALLY UNRECOVERABLE:
* every page body is generic and NEVER names the entity it relates to. Only
* the typed edge connects them, so keyword + vector retrieval cannot surface
* the answer — isolating the contribution of the typed-edge recall arm.
*
* Canonical source for BOTH the seed loader and the question set, so the
* corpus and the gold answers can't drift. `relational.jsonl` is generated
* from RELATIONAL_QUESTIONS (a drift test pins them equal).
*/
import type { BrainEngine } from '../../../../src/core/engine.ts';
import type { ChunkInput } from '../../../../src/core/types.ts';
import type { NamedThingQuestion } from '../../../../src/eval/retrieval-quality/harness.ts';
// edge maps: target entity → entities related to it via that edge type.
// invested_in / led_round point INTO the company (people/funds → company).
const INVESTMENTS: Record<string, string[]> = {
'companies/widget-co': ['people/alice-example', 'people/bob-example', 'funds/fund-a'],
'companies/acme-co': ['people/carol-example', 'people/alice-example', 'funds/fund-b'],
'companies/novapay': ['people/bob-example', 'funds/fund-a'],
'companies/mindbridge': ['people/dave-example', 'people/carol-example'],
'companies/helio': ['people/erin-example', 'funds/fund-b'],
'companies/quanta': ['people/frank-example', 'people/alice-example'],
'companies/zenith': ['people/grace-example', 'people/heidi-example'],
'companies/orbital': ['people/ivan-example', 'people/alice-example'],
};
const EMPLOYMENT: Record<string, string[]> = {
'companies/widget-co': ['people/erin-example', 'people/grace-example'],
'companies/acme-co': ['people/dave-example', 'people/frank-example'],
'companies/novapay': ['people/grace-example'],
'companies/helio': ['people/heidi-example'],
'companies/zenith': ['people/ivan-example'],
};
const FOUNDED: Record<string, string[]> = {
'companies/mindbridge': ['people/carol-example'],
'companies/quanta': ['people/frank-example'],
'companies/widget-co': ['people/ivan-example'],
'companies/orbital': ['people/grace-example'],
};
const ADVISES: Record<string, string[]> = {
'companies/novapay': ['people/frank-example'],
'companies/helio': ['people/alice-example'],
};
// Generic, cross-mention-free bodies keyed by slug prefix.
function bodyFor(slug: string): string {
if (slug.startsWith('companies/')) return 'A privately held company operating in its sector. Founded some years ago; details are sparse in this note.';
if (slug.startsWith('funds/')) return 'An early-stage venture fund. Writes first checks; portfolio is not enumerated here.';
return 'An individual in the network. Background and current focus are not described in this note.';
}
function titleFor(slug: string): string {
const tail = slug.split('/')[1];
return tail.replace(/-/g, ' ').replace(/\b\w/g, c => c.toUpperCase());
}
/** Every distinct slug in the graph. */
function allSlugs(): string[] {
const set = new Set<string>();
for (const map of [INVESTMENTS, EMPLOYMENT, FOUNDED, ADVISES]) {
for (const [company, members] of Object.entries(map)) {
set.add(company);
for (const m of members) set.add(m);
}
}
return [...set].sort();
}
function pageType(slug: string): 'company' | 'person' {
// funds + people render as person-ish entity pages; companies as company.
return slug.startsWith('companies/') ? 'company' : 'person';
}
// Deterministic per-slug basis embedding so pages are properly indexed
// (chunked) like a real brain. The query has no embedding in CI (no API key),
// so vector search can't connect entities anyway — but the pages must carry a
// chunk to be searchable at all. Body text stays generic + cross-mention-free.
// `dim` MUST match the schema's embedding column, which tracks the configured
// gateway default (1280 ZE / 1536 OpenAI) and can shift with shard order —
// so callers probe the real width via probeEmbeddingDim rather than hardcode.
function basisEmbedding(slug: string, dim: number): Float32Array {
let h = 0;
for (let i = 0; i < slug.length; i++) h = (h * 31 + slug.charCodeAt(i)) >>> 0;
const e = new Float32Array(dim);
e[h % dim] = 1.0;
return e;
}
/**
* Probe the actual `content_chunks.embedding` column width. pgvector stores
* the dimension in `atttypmod` directly. Tests must size fixtures to this, not
* a hardcoded 1536 — the column tracks the gateway default and a prior test in
* the same shard can leave it at 1280 (ZE). Mirrors pglite-engine.test.ts.
*/
export async function probeEmbeddingDim(engine: BrainEngine): Promise<number> {
const db = (engine as unknown as { db: { query: (sql: string) => Promise<{ rows: Array<{ atttypmod: number }> }> } }).db;
const r = await db.query(
`SELECT atttypmod FROM pg_attribute
WHERE attrelid = 'content_chunks'::regclass AND attname = 'embedding'`,
);
return r.rows[0].atttypmod;
}
/** Seed the corpus into a fresh brain. Bodies never name related entities. */
export async function seedRelationalCorpus(engine: BrainEngine): Promise<void> {
const dim = await probeEmbeddingDim(engine);
for (const slug of allSlugs()) {
const body = bodyFor(slug);
await engine.putPage(slug, {
type: pageType(slug),
title: titleFor(slug),
compiled_truth: body,
timeline: '',
});
await engine.upsertChunks(slug, [{
chunk_index: 0,
chunk_text: body,
chunk_source: 'compiled_truth',
embedding: basisEmbedding(slug, dim),
token_count: body.split(/\s+/).length,
}] satisfies ChunkInput[]);
}
const addAll = async (map: Record<string, string[]>, linkType: string) => {
for (const [company, members] of Object.entries(map)) {
for (const m of members) {
// edge points member → company (member invested_in / works_at / founded / advises company)
await engine.addLink(m, company, '', linkType, 'manual');
}
}
};
await addAll(INVESTMENTS, 'invested_in');
await addAll(EMPLOYMENT, 'works_at');
await addAll(FOUNDED, 'founded');
await addAll(ADVISES, 'advises');
}
function bareName(slug: string): string {
return slug.split('/')[1];
}
/** Generate the gold question set from the same maps used to seed. */
export function buildRelationalQuestions(): NamedThingQuestion[] {
const qs: NamedThingQuestion[] = [];
for (const [company, investors] of Object.entries(INVESTMENTS)) {
qs.push({
family: 'graph-relationship', kind: 'who_rel', seed: company, linkTypes: ['invested_in', 'led_round'],
query: `who invested in ${bareName(company)}`, relevant: investors,
});
}
for (const [company, employees] of Object.entries(EMPLOYMENT)) {
qs.push({
family: 'graph-relationship', kind: 'who_rel', seed: company, linkTypes: ['works_at'],
query: `who works at ${bareName(company)}`, relevant: employees,
});
}
for (const [company, founders] of Object.entries(FOUNDED)) {
qs.push({
family: 'graph-relationship', kind: 'who_rel', seed: company, linkTypes: ['founded'],
query: `who founded ${bareName(company)}`, relevant: founders,
});
}
for (const [company, advisors] of Object.entries(ADVISES)) {
qs.push({
family: 'graph-relationship', kind: 'who_rel', seed: company, linkTypes: ['advises'],
query: `who advises ${bareName(company)}`, relevant: advisors,
});
}
// connects: company pairs that share at least one related entity (any edge).
const relatedTo = (company: string): Set<string> => {
const s = new Set<string>();
for (const map of [INVESTMENTS, EMPLOYMENT, FOUNDED, ADVISES]) for (const m of map[company] ?? []) s.add(m);
return s;
};
const companies = Object.keys(INVESTMENTS);
for (let i = 0; i < companies.length; i++) {
for (let j = i + 1; j < companies.length; j++) {
const a = companies[i], b = companies[j];
const shared = [...relatedTo(a)].filter(x => relatedTo(b).has(x));
if (shared.length === 0) continue;
qs.push({
family: 'graph-relationship', kind: 'connects', seed: `${a}${b}`, linkTypes: undefined,
query: `what connects ${bareName(a)} and ${bareName(b)}`, relevant: shared.sort(),
});
}
}
return qs;
}
export const RELATIONAL_QUESTIONS: NamedThingQuestion[] = buildRelationalQuestions();
@@ -0,0 +1,42 @@
// Relational benchmark — graph-relationship family. GENERATED from
// test/fixtures/retrieval-quality/relational/corpus.ts (canonical source).
// Answers are lexically unrecoverable — only the typed edge connects
// query to answer. Regenerate: bun run scripts/_gen_relational_jsonl.ts
{"family":"graph-relationship","kind":"who_rel","seed":"companies/widget-co","linkTypes":["invested_in","led_round"],"query":"who invested in widget-co","relevant":["people/alice-example","people/bob-example","funds/fund-a"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/acme-co","linkTypes":["invested_in","led_round"],"query":"who invested in acme-co","relevant":["people/carol-example","people/alice-example","funds/fund-b"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/novapay","linkTypes":["invested_in","led_round"],"query":"who invested in novapay","relevant":["people/bob-example","funds/fund-a"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/mindbridge","linkTypes":["invested_in","led_round"],"query":"who invested in mindbridge","relevant":["people/dave-example","people/carol-example"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/helio","linkTypes":["invested_in","led_round"],"query":"who invested in helio","relevant":["people/erin-example","funds/fund-b"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/quanta","linkTypes":["invested_in","led_round"],"query":"who invested in quanta","relevant":["people/frank-example","people/alice-example"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/zenith","linkTypes":["invested_in","led_round"],"query":"who invested in zenith","relevant":["people/grace-example","people/heidi-example"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/orbital","linkTypes":["invested_in","led_round"],"query":"who invested in orbital","relevant":["people/ivan-example","people/alice-example"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/widget-co","linkTypes":["works_at"],"query":"who works at widget-co","relevant":["people/erin-example","people/grace-example"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/acme-co","linkTypes":["works_at"],"query":"who works at acme-co","relevant":["people/dave-example","people/frank-example"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/novapay","linkTypes":["works_at"],"query":"who works at novapay","relevant":["people/grace-example"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/helio","linkTypes":["works_at"],"query":"who works at helio","relevant":["people/heidi-example"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/zenith","linkTypes":["works_at"],"query":"who works at zenith","relevant":["people/ivan-example"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/mindbridge","linkTypes":["founded"],"query":"who founded mindbridge","relevant":["people/carol-example"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/quanta","linkTypes":["founded"],"query":"who founded quanta","relevant":["people/frank-example"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/widget-co","linkTypes":["founded"],"query":"who founded widget-co","relevant":["people/ivan-example"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/orbital","linkTypes":["founded"],"query":"who founded orbital","relevant":["people/grace-example"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/novapay","linkTypes":["advises"],"query":"who advises novapay","relevant":["people/frank-example"]}
{"family":"graph-relationship","kind":"who_rel","seed":"companies/helio","linkTypes":["advises"],"query":"who advises helio","relevant":["people/alice-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/acme-co","query":"what connects widget-co and acme-co","relevant":["people/alice-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/novapay","query":"what connects widget-co and novapay","relevant":["funds/fund-a","people/bob-example","people/grace-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/helio","query":"what connects widget-co and helio","relevant":["people/alice-example","people/erin-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/quanta","query":"what connects widget-co and quanta","relevant":["people/alice-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/zenith","query":"what connects widget-co and zenith","relevant":["people/grace-example","people/ivan-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/orbital","query":"what connects widget-co and orbital","relevant":["people/alice-example","people/grace-example","people/ivan-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/novapay","query":"what connects acme-co and novapay","relevant":["people/frank-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/mindbridge","query":"what connects acme-co and mindbridge","relevant":["people/carol-example","people/dave-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/helio","query":"what connects acme-co and helio","relevant":["funds/fund-b","people/alice-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/quanta","query":"what connects acme-co and quanta","relevant":["people/alice-example","people/frank-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/orbital","query":"what connects acme-co and orbital","relevant":["people/alice-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/novapay ↔ companies/quanta","query":"what connects novapay and quanta","relevant":["people/frank-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/novapay ↔ companies/zenith","query":"what connects novapay and zenith","relevant":["people/grace-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/novapay ↔ companies/orbital","query":"what connects novapay and orbital","relevant":["people/grace-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/helio ↔ companies/quanta","query":"what connects helio and quanta","relevant":["people/alice-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/helio ↔ companies/zenith","query":"what connects helio and zenith","relevant":["people/heidi-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/helio ↔ companies/orbital","query":"what connects helio and orbital","relevant":["people/alice-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/quanta ↔ companies/orbital","query":"what connects quanta and orbital","relevant":["people/alice-example"]}
{"family":"graph-relationship","kind":"connects","seed":"companies/zenith ↔ companies/orbital","query":"what connects zenith and orbital","relevant":["people/grace-example","people/ivan-example"]}
+73
View File
@@ -0,0 +1,73 @@
/**
* Relational A/B proof + no-regression gate (PGLite, default CI).
*
* Seeds the lexically-unrecoverable relational corpus, then runs the gold
* question set twice through bare hybridSearch — relational arm OFF vs ON —
* and asserts:
* 1. recall@10 on the graph-relationship family jumps materially with the
* arm on (the answers are unreachable by keyword/vector).
* 2. a non-relational content query returns the SAME results with the arm
* on vs off (the arm is a true no-op off-target — the regression gate).
*
* This is the headline proof artifact for the feature: relational queries
* that the corpus can't surface lexically jump from ~0 to high recall.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { hybridSearch } from '../src/core/search/hybrid.ts';
import { runRetrievalQuality, parseQuestionsJsonl, type SearchFn } from '../src/eval/retrieval-quality/harness.ts';
import { seedRelationalCorpus, RELATIONAL_QUESTIONS } from './fixtures/retrieval-quality/relational/corpus.ts';
import { readFileSync } from 'fs';
import { join } from 'path';
let eng: PGLiteEngine;
beforeAll(async () => {
eng = new PGLiteEngine();
await eng.connect({});
await eng.initSchema();
await seedRelationalCorpus(eng);
}, 60_000);
afterAll(async () => { await eng.disconnect(); });
const searchFnWith = (relationalRetrieval: boolean): SearchFn => async (q) => {
const results = await hybridSearch(eng, q, { limit: 10, relationalRetrieval, expansion: false });
return results.map(r => r.slug);
};
describe('relational A/B', () => {
test('corpus has a meaningful question set', () => {
expect(RELATIONAL_QUESTIONS.length).toBeGreaterThanOrEqual(30);
});
test('relational.jsonl matches the corpus module (no drift)', () => {
const path = join(import.meta.dir, 'fixtures/retrieval-quality/relational/relational.jsonl');
const fromFile = parseQuestionsJsonl(readFileSync(path, 'utf8'));
expect(JSON.stringify(fromFile)).toBe(JSON.stringify(RELATIONAL_QUESTIONS));
});
test('recall@10 lifts materially with the relational arm ON', async () => {
const off = await runRetrievalQuality(RELATIONAL_QUESTIONS, searchFnWith(false));
const on = await runRetrievalQuality(RELATIONAL_QUESTIONS, searchFnWith(true));
const offFam = off.families.find(f => f.family === 'graph-relationship')!;
const onFam = on.families.find(f => f.family === 'graph-relationship')!;
// Answers are lexically unrecoverable → baseline recall is near zero.
expect(offFam.recall_at_10).toBeLessThan(0.25);
// Typed-edge arm surfaces them → large lift.
expect(onFam.recall_at_10).toBeGreaterThan(0.75);
expect(onFam.recall_at_10 - offFam.recall_at_10).toBeGreaterThan(0.5);
// Hit@3 should also rise sharply.
expect(onFam.hit_at_3).toBeGreaterThan(offFam.hit_at_3);
}, 120_000);
test('no-regression: non-relational query is identical arm-on vs arm-off', async () => {
const q = 'early-stage venture fund first checks';
const off = await searchFnWith(false)(q);
const on = await searchFnWith(true)(q);
expect(on).toEqual(off);
}, 60_000);
});
+115
View File
@@ -0,0 +1,115 @@
/**
* relationalFanout unit tests (PGLite, default CI).
*
* Exercises the typed-edge fan-out SQL directly so the engine method is
* covered even when the DATABASE_URL-gated parity test does not run. Pins:
* typed-edge filtering, mentions-excluded-by-default, deleted_at exclusion,
* hop/edge_count aggregation, canonical_chunk_id, multi-seed + connects,
* and determinism.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import type { ChunkInput } from '../src/core/types.ts';
import { probeEmbeddingDim } from './fixtures/retrieval-quality/relational/corpus.ts';
function emb(idx: number, dim: number): Float32Array {
const e = new Float32Array(dim);
e[idx % dim] = 1.0;
return e;
}
let eng: PGLiteEngine;
let DIM = 0;
beforeAll(async () => {
eng = new PGLiteEngine();
await eng.connect({});
await eng.initSchema();
DIM = await probeEmbeddingDim(eng); // match the schema's column width (1280 ZE / 1536 OpenAI)
const pages: Array<[string, string, string]> = [
['companies/widget-co', 'company', 'Widget Co'],
['companies/other-co', 'company', 'Other Co'],
['people/investor-a', 'person', 'Investor A'],
['people/investor-b', 'person', 'Investor B'],
['people/employee-c', 'person', 'Employee C'],
['people/mentioner', 'person', 'Mentioner'],
['people/deleted-investor', 'person', 'Deleted Investor'],
];
for (const [slug, type, title] of pages) {
await eng.putPage(slug, { type: type as 'company' | 'person', title, compiled_truth: `${title} body`, timeline: '' });
}
// investor-b carries a chunk (→ canonical_chunk_id non-null); investor-a stays chunkless.
const chunk: ChunkInput[] = [{ chunk_index: 0, chunk_text: 'b', chunk_source: 'compiled_truth', embedding: emb(2, DIM), token_count: 1 }];
await eng.upsertChunks('people/investor-b', chunk);
// Edges into widget-co.
await eng.addLink('people/investor-a', 'companies/widget-co', '', 'invested_in', 'manual');
await eng.addLink('people/investor-b', 'companies/widget-co', '', 'invested_in', 'manual');
await eng.addLink('people/employee-c', 'companies/widget-co', '', 'works_at', 'manual');
await eng.addLink('people/mentioner', 'companies/widget-co', '', 'mentions', 'mentions');
await eng.addLink('people/deleted-investor', 'companies/widget-co', '', 'invested_in', 'manual');
// investor-a also invested in other-co → widget-co and other-co connect via investor-a.
await eng.addLink('people/investor-a', 'companies/other-co', '', 'invested_in', 'manual');
// Soft-delete one investor; it must never surface.
await eng.executeRaw(`UPDATE pages SET deleted_at = now() WHERE slug = $1`, ['people/deleted-investor']);
}, 60_000);
afterAll(async () => {
await eng.disconnect();
});
describe('relationalFanout', () => {
test('typed-edge: who invested in widget-co', async () => {
const rows = await eng.relationalFanout(['companies/widget-co'], { direction: 'in', linkTypes: ['invested_in'] });
const slugs = rows.map(r => r.slug).sort();
expect(slugs).toEqual(['people/investor-a', 'people/investor-b']);
for (const r of rows) {
expect(r.hop).toBe(1);
expect(r.via_link_types).toEqual(['invested_in']);
}
});
test('deleted pages are excluded', async () => {
const rows = await eng.relationalFanout(['companies/widget-co'], { direction: 'in', linkTypes: ['invested_in'] });
expect(rows.map(r => r.slug)).not.toContain('people/deleted-investor');
});
test('mentions excluded by default, included on opt-in', async () => {
const off = await eng.relationalFanout(['companies/widget-co'], { direction: 'in' });
expect(off.map(r => r.slug)).not.toContain('people/mentioner');
// type-agnostic also picks up the works_at neighbor
expect(off.map(r => r.slug).sort()).toEqual(['people/employee-c', 'people/investor-a', 'people/investor-b']);
const on = await eng.relationalFanout(['companies/widget-co'], { direction: 'in', includeMentions: true });
expect(on.map(r => r.slug)).toContain('people/mentioner');
});
test('canonical_chunk_id: non-null for chunked page, null for chunkless', async () => {
const rows = await eng.relationalFanout(['companies/widget-co'], { direction: 'in', linkTypes: ['invested_in'] });
const a = rows.find(r => r.slug === 'people/investor-a')!;
const b = rows.find(r => r.slug === 'people/investor-b')!;
expect(a.canonical_chunk_id).toBeNull();
expect(b.canonical_chunk_id).not.toBeNull();
});
test('connects: shared midpoint reachable from both seeds with a path', async () => {
const rows = await eng.relationalFanout(['companies/widget-co', 'companies/other-co'], { direction: 'both' });
const a = rows.find(r => r.slug === 'people/investor-a');
expect(a).toBeDefined();
expect(a!.path.length).toBeGreaterThanOrEqual(2); // [seed, ..., node]
expect(a!.path[a!.path.length - 1]).toBe('people/investor-a');
});
test('empty seeds → []', async () => {
expect(await eng.relationalFanout([])).toEqual([]);
});
test('deterministic: same call twice is byte-identical', async () => {
const r1 = await eng.relationalFanout(['companies/widget-co'], { direction: 'in' });
const r2 = await eng.relationalFanout(['companies/widget-co'], { direction: 'in' });
expect(JSON.stringify(r1)).toBe(JSON.stringify(r2));
});
});
+136
View File
@@ -0,0 +1,136 @@
/**
* Relational-query parser unit tests.
*
* Covers the four archetypes, the no-match path, the precision-first
* false-positive guard, schema-pack vocab extension, subset validation, and
* the length bound (ReDoS surface).
*/
import { describe, test, expect } from 'bun:test';
import {
parseRelationalQuery,
validateVocab,
KNOWN_LINK_TYPES,
type RelationVocab,
} from '../src/core/search/relational-intent.ts';
describe('parseRelationalQuery — archetypes', () => {
test('who_rel: who invested in widget-co', () => {
const r = parseRelationalQuery('who invested in widget-co');
expect(r).not.toBeNull();
expect(r!.kind).toBe('who_rel');
expect(r!.seeds).toEqual(['widget-co']);
expect(r!.linkTypes).toEqual(['invested_in', 'led_round']);
expect(r!.direction).toBe('in');
});
test('who_rel: who founded acme (article stripped)', () => {
const r = parseRelationalQuery('who founded the acme corporation?');
expect(r!.kind).toBe('who_rel');
expect(r!.seeds).toEqual(['acme corporation']);
expect(r!.linkTypes).toEqual(['founded']);
});
test('who_at: who at acme works on payments', () => {
const r = parseRelationalQuery('who at acme works on payments');
expect(r!.kind).toBe('who_at');
expect(r!.seeds).toEqual(['acme']);
expect(r!.linkTypes).toEqual(['works_at']);
expect(r!.direction).toBe('in');
});
test('intro: who introduced me to alice-example (type-agnostic)', () => {
const r = parseRelationalQuery('who introduced me to alice-example?');
expect(r!.kind).toBe('intro');
expect(r!.seeds).toEqual(['alice-example']);
expect(r!.linkTypes).toBeNull();
expect(r!.direction).toBe('both');
});
test('connects: what connects fund-a and fund-b (two seeds)', () => {
const r = parseRelationalQuery('what connects fund-a and fund-b');
expect(r!.kind).toBe('connects');
expect(r!.seeds).toEqual(['fund-a', 'fund-b']);
expect(r!.linkTypes).toBeNull();
expect(r!.direction).toBe('both');
});
test('connects: how are X and Y related', () => {
const r = parseRelationalQuery('how are widget-co and acme related');
expect(r!.kind).toBe('connects');
expect(r!.seeds).toEqual(['widget-co', 'acme']);
});
test('outgoing: what did alice invest in', () => {
const r = parseRelationalQuery('what did alice invest in');
expect(r!.kind).toBe('who_rel');
expect(r!.seeds).toEqual(['alice']);
expect(r!.direction).toBe('out');
});
});
describe('parseRelationalQuery — precision-first / no-match', () => {
test('non-relational content query → null', () => {
expect(parseRelationalQuery('what is the capital structure of a seed round')).toBeNull();
expect(parseRelationalQuery('notes from the offsite')).toBeNull();
expect(parseRelationalQuery('summarize the q3 board deck')).toBeNull();
});
test('false-positive: "who invested TIME in learning Rust" does NOT match', () => {
// "invested time in" is not "invested in" — adjacency guard.
expect(parseRelationalQuery('who invested time in learning Rust')).toBeNull();
});
test('pronoun / stopword seed is rejected', () => {
expect(parseRelationalQuery('who invested in it')).toBeNull();
expect(parseRelationalQuery('who founded them?')).toBeNull();
});
test('empty / overlong input → null', () => {
expect(parseRelationalQuery('')).toBeNull();
expect(parseRelationalQuery('who invested in ' + 'x'.repeat(600))).toBeNull();
});
});
describe('schema-pack vocab extension (D2=B)', () => {
const vocab: RelationVocab = {
extraVerbs: [{ verb: 'related to|associated with', linkTypes: ['related_to'], direction: 'both' }],
};
test('extra verb extends detection', () => {
expect(parseRelationalQuery('who related to widget-co', vocab)!.linkTypes).toEqual(['related_to']);
// same query without the pack does NOT match the extra verb
expect(parseRelationalQuery('who related to widget-co')).toBeNull();
});
test('validateVocab passes for known types', () => {
expect(() => validateVocab(vocab)).not.toThrow();
});
test('validateVocab throws on unknown link_type', () => {
expect(() =>
validateVocab({ extraVerbs: [{ verb: 'pwns', linkTypes: ['not_a_real_edge'], direction: 'in' }] }),
).toThrow(/unknown link_type/);
});
});
describe('default bank emits only known link types (no drift)', () => {
test('every parsed linkType is a subset of KNOWN_LINK_TYPES', () => {
const queries = [
'who invested in widget-co',
'who founded acme',
'who advises bob-example',
'who works at acme',
'who at acme leads payments',
'what did alice invest in',
'where does alice work',
];
for (const q of queries) {
const r = parseRelationalQuery(q);
if (!r || r.linkTypes === null) continue;
for (const lt of r.linkTypes) {
expect(KNOWN_LINK_TYPES.has(lt)).toBe(true);
}
}
});
});
+72
View File
@@ -0,0 +1,72 @@
/**
* Relational recall arm integration tests (PGLite, default CI).
*
* End-to-end through buildRelationalArm: parse → resolve seed → fanout →
* hydrate. Pins the lexically-unrecoverable win (the investor page never names
* the company; only the invested_in edge connects them), the non-relational
* no-op, attribution stamping, and fail-open.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { buildRelationalArm } from '../src/core/search/relational-recall.ts';
import { probeEmbeddingDim } from './fixtures/retrieval-quality/relational/corpus.ts';
import type { ChunkInput } from '../src/core/types.ts';
let eng: PGLiteEngine;
beforeAll(async () => {
eng = new PGLiteEngine();
await eng.connect({});
await eng.initSchema();
const dim = await probeEmbeddingDim(eng); // match schema column width (1280 ZE / 1536 OpenAI)
await eng.putPage('companies/widget-co', { type: 'company', title: 'Widget Co', compiled_truth: 'A payments company.', timeline: '' });
// The investor's body deliberately NEVER mentions Widget Co — only the edge connects them.
await eng.putPage('people/alice-example', { type: 'person', title: 'Alice Example', compiled_truth: 'Alice is a seed-stage investor based in Lisbon.', timeline: '' });
await eng.upsertChunks('people/alice-example', [{
chunk_index: 0, chunk_text: 'Alice is a seed-stage investor based in Lisbon.',
chunk_source: 'compiled_truth', embedding: new Float32Array(dim), token_count: 8,
}] satisfies ChunkInput[]);
await eng.addLink('people/alice-example', 'companies/widget-co', '', 'invested_in', 'manual');
}, 60_000);
afterAll(async () => { await eng.disconnect(); });
describe('buildRelationalArm', () => {
test('surfaces the edge answer that lexical search would miss', async () => {
const list = await buildRelationalArm(eng, 'who invested in widget-co');
const alice = list.find(r => r.slug === 'people/alice-example');
expect(alice).toBeDefined();
expect(alice!.relational_via_link_types).toEqual(['invested_in']);
expect(alice!.relational_hop).toBe(1);
expect(alice!.relational_seed).toBe('companies/widget-co');
// chunk-backed page → reinforces a REAL chunk id (not synthetic 0).
expect(alice!.chunk_id).toBeGreaterThan(0);
});
test('non-relational query is a pure no-op', async () => {
const meta: { fired?: boolean } = {};
const list = await buildRelationalArm(eng, 'summary of the payments roadmap', { onMeta: m => { meta.fired = m.fired; } });
expect(list).toEqual([]);
expect(meta.fired).toBe(false);
});
test('unresolvable seed → no-op (never traverse from a guess)', async () => {
const list = await buildRelationalArm(eng, 'who invested in nonexistent-phantom-xyz');
expect(list).toEqual([]);
});
test('fail-open: fanout error returns [] + errored meta, never throws', async () => {
const original = eng.relationalFanout.bind(eng);
let captured: { errored?: boolean } = {};
eng.relationalFanout = async () => { throw new Error('boom'); };
try {
const list = await buildRelationalArm(eng, 'who invested in widget-co', { onMeta: m => { captured = m; } });
expect(list).toEqual([]);
expect(captured.errored).toBe(true);
} finally {
eng.relationalFanout = original;
}
});
});
+70
View File
@@ -0,0 +1,70 @@
/**
* RRF fusion key is source-aware (federation hardening).
*
* Pre-fix the RRF/dedup key was `slug:chunk_id`, which collapsed two
* same-slug pages living in different federated sources into one fusion
* entry — a cross-source recall bug. The key is now
* `(source_id, slug, chunk_id)`, matching `dedup.ts:pageKey`'s composite
* discipline at chunk granularity. These tests pin that behavior for both
* `rrfFusion` and `rrfFusionWeighted` and confirm single-source ranking is
* unchanged.
*/
import { describe, test, expect } from 'bun:test';
import { rrfFusion, rrfFusionWeighted } from '../src/core/search/hybrid.ts';
import type { SearchResult } from '../src/core/types.ts';
function makeResult(overrides: Partial<SearchResult> = {}): SearchResult {
return {
slug: 'test-page',
page_id: 1,
title: 'Test',
type: 'concept',
chunk_text: 'unique chunk text',
chunk_source: 'timeline', // avoid compiled_truth 2x boost noise
chunk_id: 1,
chunk_index: 0,
score: 0.5,
stale: false,
...overrides,
};
}
describe('RRF key is source-aware', () => {
test('same slug + chunk_id in DIFFERENT sources do NOT collapse', () => {
const a = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-a', chunk_text: 'x' });
const b = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-b', chunk_text: 'y' });
const out = rrfFusion([[a, b]], 60);
expect(out.length).toBe(2);
expect(new Set(out.map(r => r.source_id))).toEqual(new Set(['team-a', 'team-b']));
});
test('same slug + chunk_id + SAME source DO collapse (reinforce across lists)', () => {
const a = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-a', chunk_text: 'x' });
const b = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-a', chunk_text: 'x' });
const out = rrfFusion([[a], [b]], 60);
expect(out.length).toBe(1);
});
test('single-source (no source_id → "default") ranking unchanged', () => {
const a = makeResult({ slug: 'a', chunk_id: 1, chunk_text: 'aaa' });
const b = makeResult({ slug: 'b', chunk_id: 2, chunk_text: 'bbb' });
const out = rrfFusion([[a, b]], 60);
expect(out.length).toBe(2);
expect(out[0].slug).toBe('a'); // rank-0 wins, as before the key change
});
test('chunkless rows (null chunk_id) stay distinct per source via text prefix', () => {
const a = makeResult({ slug: 'people/alice', chunk_id: undefined as unknown as number, source_id: 'team-a', chunk_text: 'alice page' });
const b = makeResult({ slug: 'people/alice', chunk_id: undefined as unknown as number, source_id: 'team-b', chunk_text: 'alice page' });
const out = rrfFusion([[a, b]], 60);
expect(out.length).toBe(2);
});
test('weighted RRF is also source-aware', () => {
const a = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-a' });
const b = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-b' });
const out = rrfFusionWeighted([{ list: [a, b], k: 60 }]);
expect(out.length).toBe(2);
});
});
+2 -2
View File
@@ -89,7 +89,7 @@ describe('alias_resolved boost stage', () => {
});
describe('KNOBS_HASH_VERSION', () => {
it('is 9 (8→9 archive-demote invalidates archive-excluded cache rows, #1777)', () => {
expect(KNOBS_HASH_VERSION).toBe(9);
it('is 10 (9→10 relational recall arm invalidates rel-off cache rows, v0.43)', () => {
expect(KNOBS_HASH_VERSION).toBe(10);
});
});
+55 -3
View File
@@ -77,6 +77,9 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => {
// v0.42.3.0 — autocut OFF for conservative (no reranker).
autocut: false,
autocut_jump: 0.2,
// v0.43 — relational recall OFF for conservative.
relationalRetrieval: false,
relational_retrieval_depth: 2,
});
});
@@ -106,6 +109,9 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => {
// v0.42.3.0 — autocut ON.
autocut: true,
autocut_jump: 0.2,
// v0.43 — relational recall ON for balanced.
relationalRetrieval: true,
relational_retrieval_depth: 2,
});
});
@@ -133,6 +139,9 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => {
// v0.42.3.0 — autocut ON.
autocut: true,
autocut_jump: 0.2,
// v0.43 — relational recall ON for tokenmax.
relationalRetrieval: true,
relational_retrieval_depth: 2,
});
});
@@ -392,7 +401,9 @@ describe('knobsHash determinism + cross-mode separation (CDX-4)', () => {
// archive/ demote (search-exclude policy change isn't in the hash, so the
// version bump is what invalidates archive-excluded cache rows). A query
// must not be served from a cache row written before the policy change.
expect(KNOBS_HASH_VERSION).toBe(9);
// v0.43: bumped 9→10 for the relational recall arm (rel=/reld=) — a
// relational-on write must not be served to a relational-off lookup.
expect(KNOBS_HASH_VERSION).toBe(10);
});
test('T1 (codex): floor_ratio set vs unset produces DIFFERENT hashes (cache contamination prevention)', () => {
@@ -557,8 +568,8 @@ describe('v0.40.4 — graph_signals knob', () => {
});
describe('v0.42.3.0 — autocut knobs', () => {
test('KNOBS_HASH_VERSION is 9 (8→9 archive-demote, issue #1777)', () => {
expect(KNOBS_HASH_VERSION).toBe(9);
test('KNOBS_HASH_VERSION is 10 (9→10 relational recall arm, v0.43)', () => {
expect(KNOBS_HASH_VERSION).toBe(10);
});
test('bundle defaults: conservative off, balanced/tokenmax on @0.20', () => {
@@ -621,3 +632,44 @@ describe('v0.42.3.0 — autocut knobs', () => {
expect(attr.value).toBe(false);
});
});
describe('v0.43 — relational recall knobs', () => {
test('bundle defaults: conservative off, balanced/tokenmax on; depth 2', () => {
expect(MODE_BUNDLES.conservative.relationalRetrieval).toBe(false);
expect(MODE_BUNDLES.balanced.relationalRetrieval).toBe(true);
expect(MODE_BUNDLES.tokenmax.relationalRetrieval).toBe(true);
for (const m of ['conservative', 'balanced', 'tokenmax'] as const) {
expect(MODE_BUNDLES[m].relational_retrieval_depth).toBe(2);
}
});
test('per-call relationalRetrieval:false overrides bundle/config true', () => {
// bundle default true (balanced); per-call false wins
expect(resolveSearchMode({ mode: 'balanced', perCall: { relationalRetrieval: false } }).relationalRetrieval).toBe(false);
// config override true on conservative (bundle false); then per-call false beats config
expect(
resolveSearchMode({ mode: 'conservative', overrides: { relationalRetrieval: true }, perCall: { relationalRetrieval: false } }).relationalRetrieval,
).toBe(false);
// config override alone flips conservative on
expect(resolveSearchMode({ mode: 'conservative', overrides: { relationalRetrieval: true } }).relationalRetrieval).toBe(true);
});
test('loadOverridesFromConfig reads search.relational_retrieval(+_depth)', () => {
const ov = loadOverridesFromConfig({ 'search.relational_retrieval': 'true', 'search.relational_retrieval_depth': '3' });
expect(ov.relationalRetrieval).toBe(true);
expect(ov.relational_retrieval_depth).toBe(3);
// out-of-range depth is ignored (falls through to bundle)
expect(loadOverridesFromConfig({ 'search.relational_retrieval_depth': '9' }).relational_retrieval_depth).toBeUndefined();
});
test('SEARCH_MODE_CONFIG_KEYS includes the relational keys', () => {
expect(SEARCH_MODE_CONFIG_KEYS).toContain('search.relational_retrieval');
expect(SEARCH_MODE_CONFIG_KEYS).toContain('search.relational_retrieval_depth');
});
test('knobsHash: relational-on vs off differ (cache isolation)', () => {
const on = knobsHash(resolveSearchMode({ mode: 'balanced' })); // relational true
const off = knobsHash(resolveSearchMode({ mode: 'balanced', perCall: { relationalRetrieval: false } }));
expect(on).not.toBe(off);
});
});
+3 -2
View File
@@ -43,7 +43,7 @@ function baseKnobs(): ResolvedSearchKnobs {
}
describe('KNOBS_HASH_VERSION + version invariants', () => {
test('version is 9 (…; 6→7 title_boost; 7→8 autocut; 8→9 archive-demote #1777)', () => {
test('version is 10 (…; 7→8 autocut; 8→9 archive-demote #1777; 9→10 relational recall)', () => {
// v0.35.0.0: 1→2 to fold reranker fields. v0.35.6.0: 2→3 to fold
// floor_ratio. v0.36 wave: piggybacks on v=3 with 7 cross-modal knobs
// (D2) PLUS column + provider context (D8/CDX-2 cross-column isolation).
@@ -57,7 +57,8 @@ describe('KNOBS_HASH_VERSION + version invariants', () => {
// cannot leak past the new stage. T2: 6→7 title_boost. v0.42.3.0: 7→8
// autocut. issue #1777: 8→9 archive/ demote (search-exclude policy change
// isn't in the hash, so the bump invalidates archive-excluded cache rows).
expect(KNOBS_HASH_VERSION).toBe(9);
// v0.43: 9→10 relational recall arm (rel=/reld=).
expect(KNOBS_HASH_VERSION).toBe(10);
});
test('hash is 16 hex chars regardless of reranker config', () => {