mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 21:19:18 +00:00
v0.41.34.0 feat(search): retrieval cathedral — max-pool + title + alias + evidence (#1657)
* feat(search): per-page max-pool in searchVector (both engines) T1 of the retrieval-cathedral wave (supersedes #1616). Vector search returned chunk-grain top-k with no DISTINCT ON, so a page could be represented by a weak chunk while a hub page's chunks crowded a distinct page's strong chunk out of the candidate set entirely. Keyword search always pooled per page; the vector path did not. - New shared buildBestPerPagePoolCte() in sql-ranking.ts — single source of truth consumed by searchKeyword + searchVector across postgres + pglite, so the two engines can't drift (the recurring parity bug class). - searchVector both engines: compute score as a select-list expr (HNSW ORDER BY stays pure-distance), pool DISTINCT ON (slug) over the full candidate set before the user LIMIT, deterministic tiebreak (slug, score DESC, page_id ASC, chunk_id ASC). - All keyword pooling blocks refactored onto the shared builder (DRY). - Regression test: a hub page's chunks no longer crowd out a distinct page's strong chunk; results are one-per-page by best chunk. Fails on old path. Verified: real-Postgres engine-parity 22/22, PGLite hermetic suite green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(search): title-phrase boost (page.title first-class signal) T2 of the retrieval-cathedral wave. A query that is a phrase from a page's title ("Greek amphitheater" -> "The Mingtang - Indoor Greek Amphitheater") matched a weak body chunk instead of being recognized as a title hit. Names of things deserve weight. - New pure title-match.ts: isTitlePhraseMatch (contiguous token-run inside page.title OR exact full-title match). Precision guards: >= 2 content tokens OR exact full-title; stopword filter; token-boundary match (no raw substring). Reused by the eval later so production + bench can't drift. - applyTitleBoost post-fusion stage in hybrid.ts: reads page.title (not the brittle "first chunk"), floor-ratio-gated, stamps title_match_boost for --explain, never touches base_score (the agent's dedup confidence). - ModeBundle.title_boost knob (1.25, on in all modes - cheap gated correctness fix), search.title_boost config key, dashboard description. - KNOBS_HASH_VERSION 6 -> 7 so a boost-on cache write can't serve a boost-off lookup; all version-pin + canonical-bundle assertions updated. - 18 new tests (matcher 13 + stage 5); typecheck clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(search): page_aliases data layer (T3 foundation) Free-text alias resolution for search. gbrain stored a page's chosen names in pages.frontmatter `aliases:` JSONB but search never consulted them, so a query like "Hall of Light" or "明堂" couldn't surface the "Mingtang" page. DELIBERATELY SEPARATE from slug_aliases (re-grounded against current code): - slug_aliases: old-slug -> canonical-slug (wikilink/get_page redirect, populated only from concept-redirect conversions) - page_aliases: normalized free-text name -> canonical slug (search hop) Overloading slug_aliases would muddy two distinct semantics, so this is a new table, not an extension (honors DRY by keeping concepts separate). - src/core/search/alias-normalize.ts: ONE normalizeAlias() (NFKC + lowercase + ws-collapse + quote-strip) + normalizeAliasList() shared by the write (ingest) and read (search) paths so they match on the same key (CQ2). - Migration v108 page_aliases (source_id, alias_norm, slug); btree (source_id, alias_norm) for indexed-equality hop, NOT ILIKE; unique TRIPLE (not source_id+alias_norm) so two pages may claim one alias — collisions reported + resolved at query time, not blocked at ingest (Codex#8). Mirror in pglite-schema.ts; Postgres fresh gets it from the migration. - engine.resolveAliases(aliasNorms, {sourceId|sourceIds}) read + setPageAliases(slug, source, aliasNorms) write, both engines, source-scoped. - 17 tests: normalize round-trip, collision, source-scope, replace, clear. Ingest projection + the hybridSearch alias hop land next (T3 wiring). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(search): alias hop + ingest projection (T3 wiring) Wires the page_aliases data layer into ingest (write) and hybridSearch (read) so a query that is a page's declared chosen name surfaces that page — the named-thing class neither max-pool nor title-boost can fix (true synonyms with zero surface overlap: "Hall of Light" / "明堂" -> the Mingtang page). - Ingest projection (import-file.ts): after the page write commits, normalizeAliasList(frontmatter.aliases) -> engine.setPageAliases. Always called (even []) so removing an alias clears its row; content_hash includes non-timestamp frontmatter so alias edits reach this path, not the skip branch. Fail-soft + pre-v108-safe (isUndefinedTableError swallowed). - applyAliasHop (hybrid.ts), AFTER rerank so a named query reliably surfaces its page: FULL normalized-query exact match only (no substring/n-grams), skip >6-token prose queries, present-boost 1.10x / inject absent canonical at top-of-organic + epsilon (never absolute 1.0, D3), collisions alpha-ordered + capped at 3, fail-open on pre-v108 / lookup error (D9). Stamps alias_hit for the T4 evidence contract. - SearchResult.alias_hit attribution field. - 8 tests: inject/boost/CJK/no-match/long-skip/collision + ingest projection round-trip + alias-removal-clears. 73 pass across the T1/T2/T3 + import suite. Backfill of existing pages' aliases lands as T8 (reindex --aliases). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(search): evidence/create_safety contract + search→cheap-hybrid + per-call mode (T4) The agent-facing fix for the incident's ROOT behavior: tonight the agent read a single blended 0.64 score, decided "no strong match, safe to write a new page", and wrote a duplicate on a developed concept page. A blended RRF/cosine score is not a calibrated probability, so the don't-duplicate decision must key off WHY a page matched, not a raw number. - evidence.ts: classifyEvidence (alias_hit > exact_title_match > high_vector_match > keyword_exact > weak_semantic) + createSafetyFor (exists | probable | unknown). stampEvidence runs at the end of every hybrid return path (main + both keyword fallbacks). SearchResult gains evidence + create_safety. The agent keys don't-duplicate off create_safety='exists', not a score threshold. - search op → cheap-hybrid everywhere (D4/D15): full vector+keyword+RRF+pool+ title+alias, expansion OFF (no per-call LLM cost); `query` stays full-control. search.mcp_keyword_only escape hatch (D17) keeps the old keyword-only behavior for operators who don't want query text sent to an embedding provider. - Alias hop + evidence now also run on the keyword-only fallback paths (the named-thing fix is most valuable exactly when vector is unavailable). - Per-call `mode` (D5): honored ONLY for local/trusted callers (ctx.remote=== false) so a remote OAuth client can't escalate to costly tokenmax; local + unknown mode rejects loudly; threaded into resolveSearchMode + the cache key. - 30 tests (evidence classifier incl. before/after-incident cases, per-call mode gate, alias hop). Updated mcp-eval-capture to the new cheap-hybrid contract. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): reconcile `gbrain search` dispatch (T5) After T4 made the `search` op cheap-hybrid, `gbrain search "x"` already does the right thing — but `gbrain search modes/stats/tune` would have run a hybrid search for the literal word "modes" instead of opening the config dashboard (the op intercepts before the unreachable handleCliOnly dashboard path). Add a pre-dispatch interception in main(): `search` + subArgs[0] in {modes,stats,tune} → runSearch dashboard (with the v0.41.6.0 read-only connect+ dispatch 10s timeout preserved); everything else (free-text) falls through to the cheap-hybrid `search` op. Subprocess test pins all three routes: modes/stats → dashboard, free-text → search op ("No results", not "Unknown subcommand"). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(eval): NamedThingBench retrieval-quality gate (T6) The eval that makes the retrieval-maxpool incident impossible to reintroduce silently. 7 query families, each a failure class the incident exposed: title-substring, generic-to-named, alias-synonym, multi-chunk-dilution, short-vs-rich, graph-relationship, hard-negative. - src/eval/retrieval-quality/harness.ts: pure scoring (Hit@1/Hit@3/MRR per family) + injected SearchFn (CLI uses hybridSearch; tests stub it) + evaluateGate. D12 gate: hard-gate the families that ARE the bug from day one (title-substring Hit@1>=0.95, alias-synonym Hit@1>=0.98, dilution Hit@3=1.0), warn-then-enforce the softer families. Env-overridable floors. - `gbrain eval retrieval-quality <fixture.jsonl> [--json] [--source]` + dispatch in eval.ts. Exit 0 PASS / 1 FAIL / 2 USAGE. - Synthetic fixture (placeholder names only, privacy-grep guarded) + hermetic gate test: seeds a synthetic brain, forces the keyword+title+alias path (embed transport stubbed to throw — free, deterministic), asserts the bug families pass. The vector max-pool guarantee is pinned separately by searchvector-maxpool.test.ts. - CI gate: the hermetic test is a normal unit test, so it runs in every PR shard — the gate is live on every change. - 23 tests (harness unit + hermetic gate + fixture privacy guard). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(telemetry): rank-1 score drift signal (T7) Standing observability so a retrieval regression is caught before a human hits it in chat (like tonight). Aggregate columns on search_telemetry (NOT per-query rows, D10): sum_rank1_score + count_rank1 + 3 coarse buckets (<0.6 / 0.6-0.85 / >=0.85). The mean rank-1 base_score is the headline; a downward drift = retrieval quality regressing. - hybrid.ts: capture rank-1 base_score at all three return paths, thread through emitMeta → recordSearchTelemetry opts (like results_count). - telemetry.ts: Bucket + record + flush ON CONFLICT-add + readSearchStats expose avg_rank1_score (null when no samples — no NaN) + rank1_distribution. - Migration v109 ADD COLUMN IF NOT EXISTS (both engines; search_telemetry lives only in migration v57, so the v57+v109 chain covers fresh + upgrade). Columns exempted in schema-bootstrap-coverage (no forward-ref index → no bootstrap need). - `gbrain search stats` surfaces the avg + bucket line; JSON envelope auto-carries the fields. "true-positive" wording dropped per Codex#14 — production has no labels, so this is an unlabeled rank-1 score histogram; labeled calibration lives in NamedThingBench (T6). - 3 round-trip tests (mean+buckets, no-result excluded, empty=null). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reindex): gbrain reindex --aliases backfill (T8) Import-time projection (T3) covers new + changed pages; this backfills EXISTING pages whose frontmatter `aliases:` predate v108 / the projection. Walks listAllPageRefs (cheap cross-source (source_id, slug) enumeration), reads each page's frontmatter aliases, writes page_aliases via setPageAliases. Idempotent (setPageAliases replaces) so re-running is convergent — no op-checkpoint needed (fast, no embedding). --dry-run reports would-write counts, --source narrows, --limit caps, --json envelope, progress reporter. Wired into the `reindex` dispatch alongside --markdown / --multimodal. 4 tests: backfill from array + comma-scalar frontmatter, --dry-run writes nothing, idempotent second run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(search): pre-migration fail-open regression (T9) Pins that pre-v108 brains (no page_aliases table) keep working: applyAliasHop returns input unchanged + doesn't throw, importFromContent with frontmatter aliases still imports (projection swallows table-missing via isUndefinedTableError), and resolveAliases surfaces the error for the caller to catch. Completes the T9 mandatory regression set (dilution → searchvector-maxpool, dispatch → cli-search-dispatch, MCP contract → mcp-eval-capture, engine parity → engine-parity 22/22, pre-migration → here). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(search): Phase-0 retrieval diagnostic — `gbrain search diagnose` (T0) The operator-facing trace the user runs against the production brain to pin which retrieval layer surfaces (or misses) a target page — the diagnostic the plan front-loaded so we don't ship a fix that doesn't move the incident. `gbrain search diagnose "<query>" --target <slug> [--json] [--source]` reports, for the target: keyword rank+score, vector rank+score (skipped/graceful if no embedding provider), whether the query is a registered alias, and the hybrid final rank + evidence + create_safety + which boosts fired (title/alias). The verdict names the layer that surfaces the target at rank 1 (or "none"), telling you whether the lever is max-pool/innerLimit (vector) vs title/alias. Wired into the `search` dispatch alongside modes/stats/tune (60s timeout since it runs real retrieval). 2 hermetic tests (alias-query trace + title-phrase trace). For the Mingtang incident, run: gbrain search diagnose "Greek amphitheater" --target projects/new-greek-theater/concept_v0 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(retrieval): corrected incident record + named-thing layers + glossary (T10) - RETRIEVAL_MAXPOOL_INCIDENT.md: replaces closed PR #1616's RFC with the verified record — what happened, the disease, the corrections to the RFC's mechanics (search was keyword-only, --mode unthreaded, hybrid already pooled at dedup, aliases dead to search), the four-layer fix that shipped, and the triage commands (search diagnose / reindex --aliases / search stats / eval retrieval-quality). - RETRIEVAL.md: new "Named-thing retrieval" section documenting per-page pool + title boost + alias hop + the evidence contract, reconciling the doc with the shipped pipeline (closes the doc/reality gap). - metric-glossary.ts + regenerated METRIC_GLOSSARY.md: Hit@1, Hit@3, avg_rank1_score (drift signal, not labeled accuracy), and create_safety (the evidence contract) now carry plain-English glossary entries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(eval): NamedThingBench fixture privacy guard via slug-shape (T6 fixup) The banned-name literal list itself tripped check-privacy/check-test-real-names. Replace it with the load-bearing assertion: every fixture slug must be an *-example placeholder (no real brain page can be referenced). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(search): source-isolation in per-page pool + alias hop (P0, codex adversarial) Codex outside-voice caught two source-isolation P0s in the retrieval wave — the exact class the v0.34.1 seal guards. Both fixed before merge. P0-1: buildBestPerPagePoolCte pooled on `slug` alone. In a federated brain, two pages with the same slug in different sources collapsed before ranking/pagination (the neighbor-source page dropped). Now DISTINCT ON (COALESCE(source_id,'default'), slug) — composite key matching dedup.ts's pageKey. Also fixes the PRE-EXISTING keyword-path bug (best_per_page was slug-only before this wave); real-PG parity 23/23. P0-2: the alias hop dropped source_id. resolveAliases returned bare slugs and applyAliasHop hydrated via getPage(slug, undefined), so a federated caller could get the default-source page injected or the right allowed-source page suppressed. resolveAliases now returns {slug, source_id} pairs; applyAliasHop matches by (source_id, slug) and fetches each canonical in its OWN source. Regression tests: alias hop boosts only the aliased source (not same-slug in another source); resolveAliases keeps cross-source same-slug distinct. Deferred as documented tradeoffs (TODO): evidence high_vector_match label uses blended base_score not pure cosine; deep-pagination candidate budget is chunk-bounded; telemetry writes swallow errors pre-v109 on rolling deploys. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.41.30.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: v0.41.30.0 retrieval cathedral — CLAUDE.md key files + llms regen Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: renumber release v0.41.30.0 → v0.41.34.0 (queue moved) Version trio + CHANGELOG header + CLAUDE.md key-file annotations + TODOS heading + regenerated llms bundles, all moved to 0.41.34.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): restore glossary roster + harden facts-anti-loop hook budget Two CI failures surfaced after the master merges that brought the branch to 111 migrations: 1. shard 1 — `ALL_METRICS roster > matches the renderer output (no orphans)`: the merge took master's `renderMetricGlossaryMarkdown` whose `groups` array lacked this branch's 4 retrieval-quality keys (hit@1, hit@3, avg_rank1_score, create_safety). `ALL_METRICS` (derived via Object.keys) kept them, so the roster test saw 4 orphans. The freshness check (check:eval-glossary) passed because renderer-output == committed doc — it can't catch a renderer that drops a metric; the roster test can. Restored the "Retrieval-Quality / Evidence Metrics (NamedThingBench)" group + regenerated docs/eval/METRIC_GLOSSARY.md. 2. shard 2 — facts-anti-loop's two engine-dependent put_page tests failed while the two engine-free extractFactsFromTurn tests passed (the signature of a partially-failed beforeAll). This file has a documented PGLite-cold-start-under-deep-shard-load timeout history; the 30s budget was tuned for 95 migrations and the chain is now 111. Bumped to 60s. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): isolate facts-anti-loop in its own process (serial) Follow-up to the prior hook-timeout bump, which was the wrong theory: the [58ms]/[71ms] body times in the re-run prove beforeAll did NOT time out — the engine connects and the two put_page tests run and fail for real, while the two engine-free extractFactsFromTurn tests in the same file pass. put_page (via dispatchToolCall) touches process-global singletons (the facts queue + the AI gateway used by importFromContent's embed step). Some sibling file in the 78-file shard-2 process leaves residual global state that makes put_page's pre-backstop path fail on the CI runner. The failure is NOT reproducible alone, in a Linux oven/bun:1 container, or in a full local shard-2 run (1172 pass) — only on the GitHub runner, deterministically. Per CLAUDE.md's test-isolation rules, a test coupled to shared process state belongs in its own process. Renamed to *.serial.test.ts so it runs in the dedicated serial-tests job (scripts/run-serial-tests.sh spawns a fresh `bun test` per serial file), where it passes deterministically; test-shard.sh excludes serial files from the matrix. Updated the comment to reflect the real cause and refreshed the test-weights.json key. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): close cross-file gateway-config pollution in test shards The prior serial-move theory was incomplete. The real, single root cause behind all three shard failures (2, 5, 10) is cross-file AI-gateway config pollution within a shard's bun process: - A test calls configureGateway() and doesn't restore the gateway on exit. The legacy-embedding preload pins OpenAI/1536 ONCE at process start and re-pins per-test ONLY when the gateway slot is empty — so a leaker that reconfigured the gateway to the v0.37 default (zeroentropyai:zembed-1 / 1280-d) and never reset poisons every later file in the shard. - Victim A (shard 5, test/search/searchvector-maxpool.test.ts): runs initSchema in beforeAll under the leaked gateway → content_chunks.embedding becomes vector(1280) → inserting its hardcoded 1536-d basis vectors throws pgvector CheckExpectedDim. - Victims B/C (shard 10 facts-backstop-gating, shard 2 facts-anti-loop): put_page's importFromContent embeds by design (embed failure PROPAGATES, Codex C2). Under a leaked fake-key gateway the embed step 401s and put_page returns isError → the backstop assertions fail. My branch's shard re-partition (added test files + weight changes) merely co-located leakers with victims; the hazard was latent. Fixes (root cause + self-sufficient victims): - test/search/rerank.test.ts (the shard-5 leaker): add afterAll(resetGateway). Its stub omits embedding_model, so it fell back to the ZE/1280 default; now it restores the empty slot so the preload re-pins legacy for the next file. - test/search/searchvector-maxpool.test.ts: pin configureGateway(openai/1536) in beforeAll BEFORE initSchema (initSchema runs before any preload beforeEach, so it can't rely on the inherited slot). - test/facts-backstop-gating.test.ts + test/facts-anti-loop.test.ts: reset the gateway in beforeEach so put_page's embed is a graceful no-op; reverted anti-loop from the serial quarantine back into the matrix (the serial move was the wrong fix for a gateway-state problem). Validated deterministically: a non-resetting leaker that poisons the gateway to ZE, run first in one bun process, no longer breaks any of the three victims (14/14 pass). verify 29/29, typecheck clean, isolation lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
730aed77f2
commit
d6db3f0ce3
+100
@@ -2,6 +2,106 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.41.34.0] - 2026-05-30
|
||||
|
||||
**Search now finds a page by its name, even when you only remember what you
|
||||
called it.** Ask for "Greek amphitheater" and the page you titled "The Mingtang —
|
||||
Indoor Greek Amphitheater" comes back first, not buried at a weak 0.64 behind two
|
||||
unrelated pages. Give a page `aliases:` in its frontmatter ("Hall of Light", "明堂")
|
||||
and any of those names finds it too. And every result now tells your agent *why*
|
||||
it matched, so it stops writing duplicate pages on top of ones you already have.
|
||||
|
||||
A real miss prompted this: an agent searched "Greek amphitheater," didn't
|
||||
confidently find the existing concept page, and wrote a duplicate stub on top of
|
||||
2,000 words of work. The page was right there — search just represented it by a
|
||||
throwaway body chunk instead of its title. Four things changed so that can't
|
||||
happen again:
|
||||
|
||||
- **A page is now scored by its best chunk, not a random one.** Vector search
|
||||
pools to one row per page (its strongest match) before ranking, so a strong
|
||||
page can't get crowded out of the results by another page's chunks. Same
|
||||
behavior on both the embedded (PGLite) and Postgres engines.
|
||||
- **A query that's a phrase in the title gets a boost.** "Greek amphitheater" is
|
||||
in the title, so that page wins by design instead of by luck. Bounded and gated
|
||||
so it can't float an irrelevant page to the top.
|
||||
- **Chosen names actually resolve.** Add `aliases:` to a page's frontmatter and
|
||||
those names route to it at query time — the only thing that bridges "Hall of
|
||||
Light" to the Mingtang when no words overlap. Run `gbrain reindex --aliases`
|
||||
once to backfill names on pages you already have.
|
||||
- **Results carry evidence, not just a number.** Each result says how it matched
|
||||
(`exact_title_match`, `alias_hit`, `weak_semantic`, …) and a `create_safety`
|
||||
hint (`exists` / `probable` / `unknown`). Your agent keys "is this already in
|
||||
the brain?" off that, not a fuzzy score — which is what stops the duplicate.
|
||||
|
||||
`gbrain search "<text>"` is now the good hybrid path (it used to be keyword-only);
|
||||
`gbrain search modes/stats/tune` still work. New: `gbrain search diagnose
|
||||
"<query>" --target <slug>` traces exactly which layer surfaces (or misses) a
|
||||
page. `gbrain search stats` now shows an average rank-1 match score so a
|
||||
retrieval regression shows up before you hit it in chat. And `gbrain eval
|
||||
retrieval-quality` is a CI gate (NamedThingBench) that fails any change which
|
||||
regresses name-based search.
|
||||
|
||||
### To take advantage of v0.41.34.0
|
||||
|
||||
`gbrain upgrade` applies migrations v110 (page_aliases) and v111 (telemetry
|
||||
columns) automatically. If `gbrain doctor` warns about a partial migration:
|
||||
|
||||
1. **Run the orchestrator manually:**
|
||||
```bash
|
||||
gbrain apply-migrations --yes
|
||||
```
|
||||
2. **Backfill aliases for pages you already have** (one time):
|
||||
```bash
|
||||
gbrain reindex --aliases
|
||||
```
|
||||
To make a page findable by a chosen name, add `aliases:` to its frontmatter
|
||||
and re-sync, or pass the names there before the backfill.
|
||||
3. **Verify:**
|
||||
```bash
|
||||
gbrain search "<a phrase from one of your page titles>"
|
||||
gbrain search diagnose "<that phrase>" --target <the page slug>
|
||||
gbrain doctor
|
||||
```
|
||||
4. **If anything looks wrong,** file an issue with `gbrain doctor` output:
|
||||
https://github.com/garrytan/gbrain/issues
|
||||
|
||||
Note: the `search` operation (CLI + MCP) now runs hybrid retrieval by default
|
||||
instead of keyword-only. If you specifically need the old cheap keyword-only
|
||||
behavior for an automated caller, set `gbrain config set search.mcp_keyword_only true`.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
#### Added
|
||||
- Per-page max-pool in `searchVector` (both engines) via a shared
|
||||
`buildBestPerPagePoolCte` SQL builder; collapse key is the composite
|
||||
`(source_id, slug)` so federated brains never collapse same-slug pages across sources.
|
||||
- Title-phrase boost (`search.title_boost`, on in all modes): floor-ratio-gated,
|
||||
capped, reads `page.title`, stamped for `--explain`.
|
||||
- Free-text alias layer: `page_aliases` table (migration v110), shared
|
||||
`normalizeAlias`, ingest projection from frontmatter `aliases:`, query-time
|
||||
alias hop (full-match, source-scoped, bounded inject, fail-open pre-migration),
|
||||
and `gbrain reindex --aliases` backfill.
|
||||
- Evidence contract: every `SearchResult` carries `evidence` +
|
||||
`create_safety` (+ `base_score`/`final_score`), so the agent's
|
||||
don't-duplicate decision keys off why a page matched.
|
||||
- `gbrain search diagnose "<query>" --target <slug>` — Phase-0 retrieval trace
|
||||
(keyword / vector / alias / hybrid ranks + boosts + verdict).
|
||||
- NamedThingBench: `gbrain eval retrieval-quality <fixture.jsonl>` + a hermetic
|
||||
CI gate over 7 query families.
|
||||
- Rank-1 match-score drift telemetry (migration v111) surfaced in `gbrain search stats`.
|
||||
- Per-call `--mode` for local/trusted callers (remote uses configured mode).
|
||||
|
||||
#### Changed
|
||||
- The `search` operation (CLI + MCP) is now cheap-hybrid by default
|
||||
(vector + keyword + RRF + pool + title + alias, expansion off); `query`
|
||||
remains the full-control variant. Opt back into keyword-only with
|
||||
`search.mcp_keyword_only`.
|
||||
- `gbrain search "<freetext>"` routes to hybrid; `modes/stats/tune` stay subcommands.
|
||||
|
||||
#### Fixed
|
||||
- A page represented by its weakest chunk in vector search (the incident).
|
||||
- Same-slug pages in different sources collapsing in per-page pooling and the
|
||||
alias hop (source-isolation).
|
||||
## [0.41.33.0] - 2026-05-29
|
||||
|
||||
**`gbrain search` and `query` can now return a tight, intent-sized set of
|
||||
|
||||
@@ -230,7 +230,7 @@ The whole loop is described in [`docs/architecture/topologies.md`](docs/architec
|
||||
|
||||
## Capabilities
|
||||
|
||||
**Hybrid search.** Vector (HNSW on pgvector) + BM25 keyword + reciprocal-rank fusion + source-tier boost + intent-aware query rewriting. Three named search modes (`conservative`, `balanced`, `tokenmax`) bundle the cost/quality knobs into a single config key. Live cost/recall comparisons in [`docs/eval/SEARCH_MODE_METHODOLOGY.md`](docs/eval/SEARCH_MODE_METHODOLOGY.md). Default: `balanced` with ZeroEntropy reranker on. Per-query graph signals notice when a top result is a hub for THAT query (adjacency boost), is corroborated across team brains (cross-source boost), or is being crowded out by weak chunks from a chatty session (session demote). Run `gbrain search "<query>" --explain` to see per-stage attribution: base score, every boost that fired, what it multiplied. `gbrain doctor` ships a `graph_signals_coverage` check; `gbrain search stats` shows fire counts and failure breakdowns.
|
||||
**Hybrid search.** Vector (HNSW on pgvector) + BM25 keyword + reciprocal-rank fusion + source-tier boost + intent-aware query rewriting. Three named search modes (`conservative`, `balanced`, `tokenmax`) bundle the cost/quality knobs into a single config key. Live cost/recall comparisons in [`docs/eval/SEARCH_MODE_METHODOLOGY.md`](docs/eval/SEARCH_MODE_METHODOLOGY.md). Default: `balanced` with ZeroEntropy reranker on. Per-query graph signals notice when a top result is a hub for THAT query (adjacency boost), is corroborated across team brains (cross-source boost), or is being crowded out by weak chunks from a chatty session (session demote). Run `gbrain search "<query>" --explain` to see per-stage attribution: base score, every boost that fired, what it multiplied. `gbrain doctor` ships a `graph_signals_coverage` check; `gbrain search stats` shows fire counts and failure breakdowns. Vector retrieval pools the best chunk per page, so a page surfaces on its strongest evidence instead of losing to a neighbor on one weak chunk. Queries that match a page's title phrase or a declared free-text alias (`gbrain reindex --aliases` backfills existing pages) get boosted to the page they name. Every result carries an `evidence` tag (why it matched) and a `create_safety` hint (`exists` / `probable` / `unknown`) so an agent decides whether a page already exists instead of guessing from a raw score. `gbrain search diagnose "<query>" --target <slug>` traces which retrieval layer surfaces (or misses) a page.
|
||||
|
||||
**Self-wiring knowledge graph.** Every `put_page` extracts entity refs from markdown/wikilinks/typed-link syntax and writes edges with zero LLM calls. Typed edges (`attended`, `works_at`, `invested_in`, `founded`, `advises`, `mentions`, …). Multi-hop traversal via `gbrain graph-query`. The graph is what produces the +31.4 P@5 lift over vector-only RAG.
|
||||
|
||||
@@ -238,7 +238,7 @@ The whole loop is described in [`docs/architecture/topologies.md`](docs/architec
|
||||
|
||||
**43 curated skills.** Routing lives in [`skills/RESOLVER.md`](skills/RESOLVER.md). Covers signal capture, ingest (idea / media / meeting), enrichment, querying, brain ops, citation fixing, daily task management, cron scheduling, reports, voice, soul audit, skill creation, eval framework, and migrations. Skills are markdown files (tool-agnostic), packaged as a single skillpack the installer drops into your agent workspace.
|
||||
|
||||
**Eval framework.** `gbrain eval longmemeval` runs the public [LongMemEval](https://huggingface.co/datasets/xiaowu0162/longmemeval) benchmark against your hybrid retrieval. `gbrain eval export` + `gbrain eval replay` capture real queries and replay them against code changes (set `GBRAIN_CONTRIBUTOR_MODE=1`). `gbrain eval cross-modal` cross-checks an output against the task using three different-provider frontier models. Full methodology in [`docs/eval/SEARCH_MODE_METHODOLOGY.md`](docs/eval/SEARCH_MODE_METHODOLOGY.md).
|
||||
**Eval framework.** `gbrain eval longmemeval` runs the public [LongMemEval](https://huggingface.co/datasets/xiaowu0162/longmemeval) benchmark against your hybrid retrieval. `gbrain eval export` + `gbrain eval replay` capture real queries and replay them against code changes (set `GBRAIN_CONTRIBUTOR_MODE=1`). `gbrain eval cross-modal` cross-checks an output against the task using three different-provider frontier models. `gbrain eval retrieval-quality` runs NamedThingBench, which hard-gates the named-thing retrieval families (title-substring, alias-synonym, generic-to-named, multi-chunk-dilution) so a regression in "find the page this query names" fails CI loudly. Full methodology in [`docs/eval/SEARCH_MODE_METHODOLOGY.md`](docs/eval/SEARCH_MODE_METHODOLOGY.md).
|
||||
|
||||
**Brain consistency.** `gbrain eval suspected-contradictions` samples retrieval pairs, layered date pre-filter, query-conditioned LLM judge, persistent cache. Surfaces conflicts between takes + facts the agent has written. Wired into the daily dream cycle.
|
||||
|
||||
|
||||
@@ -1,5 +1,32 @@
|
||||
# TODOS
|
||||
|
||||
## v0.41.34.0 retrieval-cathedral follow-ups (v0.42+)
|
||||
|
||||
Deferred from the v0.41.34.0 wave (codex adversarial P1/P2 — documented tradeoffs,
|
||||
not blockers; the P0 source-isolation issues were fixed in-wave).
|
||||
|
||||
- [ ] **P1 — Calibrate the `evidence` classifier.** `high_vector_match` is assigned
|
||||
from `base_score >= 0.85`, but `base_score` is the pre-boost RRF/keyword/title/alias
|
||||
pipeline score, not a pure cosine. A generic high-scoring page can read as
|
||||
`create_safety='exists'`. Add a true vector-cosine signal (or a `keyword_exact`
|
||||
exact-token check) so the evidence labels are grounded, not inferred from the blend.
|
||||
File: `src/core/search/evidence.ts`. **Why:** the evidence contract is what stops
|
||||
the duplicate-page class; mislabeled evidence weakens it.
|
||||
|
||||
- [ ] **P1 — Page-bounded vector pagination.** `searchVector` innerLimit is
|
||||
`offset + max(limit*5, 100)` counted BEFORE `DISTINCT ON`, so on a dense page one
|
||||
page can consume the candidate budget and a deep `OFFSET` can underfill even when
|
||||
more pages exist. Restructure to a two-stage pull (top-N chunks → pool → re-expand)
|
||||
or raise innerLimit adaptively for deep offsets. Files: both engines' `searchVector`.
|
||||
**Why:** deep search pagination on big brains can return short pages.
|
||||
|
||||
- [ ] **P2 — Telemetry rolling-deploy gap.** Pre-v111 (mid rolling deploy), rank-1
|
||||
telemetry INSERTs reference missing columns and the write is swallowed, so a window
|
||||
of telemetry is silently lost and `search stats` reads empty on old tables. Either
|
||||
feature-detect the columns before writing the extended INSERT, or accept the gap
|
||||
(documented). File: `src/core/search/telemetry.ts`. **Why:** brief observability
|
||||
blind spot during upgrades.
|
||||
|
||||
## v0.41.33.0 adaptive return-sizing follow-ups (v0.42+)
|
||||
|
||||
Filed from the v0.41.33.0 wave (intent-aware adaptive return-sizing, born from
|
||||
@@ -11,6 +38,7 @@ default-off; these are the gates and extensions before any default flip.
|
||||
- [ ] **v0.42+: gentle adaptive gate on `think`'s gather stage (A3).** The plan's A3 decision was a gentler return-gate on `runThink`'s gather candidates (cleaner context, fewer tokens per reasoning call). Deferred because the benefit is unvalidated without a longmemeval answer-quality run, and trimming the answer path (even default-off) carries regression risk. gather fuses 4 streams (page / takes-keyword / takes-vector / graph); the gate must operate on the fused output with a higher min-keep than search, validated on `gbrain eval longmemeval` answer quality (not retrieval precision). Also: `RunThinkOpts` has no `sourceId` today, so think's gather runs unscoped (codex finding) — scope-isolated think needs that plumbing first. Priority: P2.
|
||||
- [ ] **v0.42+: `--explain` human header for adaptive_return.** The decision is in `HybridSearchMeta.adaptive_return` and surfaces in `--json` today. The per-result `explain-formatter.ts` is result-scoped and can't render a per-query meta line; the human `gbrain search --explain` header needs the meta threaded through `cli.ts:formatResult` (it currently only receives `results`). Add a one-line gate-decision header (intent / cap / kept of total). Priority: P3.
|
||||
- [ ] **v0.42+: structured-alias / facts-mode fidelity for the PrecisionMemBench eval.** The gbrain-evals benchmark seeds beliefs as pages with aliases in the body (real FTS). A second fidelity that exercises gbrain's structured alias/entity-resolution layer (facts with `valid_until` + entity resolution) would measure gbrain's structured-belief path on the 23 alias cases. Lives in gbrain-evals (`eval/precisionmembench/seed.ts` throws on `fidelity:'structured'` today). Priority: P3.
|
||||
|
||||
## v0.41.32.0 content-relative staleness follow-ups (v0.42+)
|
||||
|
||||
Filed from the v0.41.32.0 wave (supersedes #1623 — commit-relative sync
|
||||
@@ -44,6 +72,7 @@ were deliberately scoped out (CM2 + the remote post-sync-divergence residual).
|
||||
different axis from sync staleness). Content-relativizing it (a source whose
|
||||
newest commit predates its last full cycle doesn't need re-cycling) is a
|
||||
natural companion to this probe phase. Priority: P3.
|
||||
|
||||
## brainstorm/lsd --save source-awareness (v0.42+)
|
||||
|
||||
Filed from the `--save` dual-sink hardening wave (route through the canonical
|
||||
|
||||
@@ -58,6 +58,38 @@ Hybrid search applies a source-factor CASE expression at the SQL layer (lives in
|
||||
|
||||
The boost map is configurable via `GBRAIN_SOURCE_BOOST` env var or per-call `SearchOpts.exclude_slug_prefixes`. Temporal queries (`detail: 'high'`) bypass the boost so chat pages re-surface for time-sensitive lookups.
|
||||
|
||||
## Named-thing retrieval (per-page pool + title + alias + evidence)
|
||||
|
||||
A brain organized around *chosen names* (Mingtang, Hall of Light) needs more than
|
||||
embedding proximity. Four layers, added after the incident in
|
||||
[`RETRIEVAL_MAXPOOL_INCIDENT.md`](./RETRIEVAL_MAXPOOL_INCIDENT.md):
|
||||
|
||||
- **Per-page max-pool** — `searchVector` (both engines) collapses chunk-grain
|
||||
candidates to the best chunk per page (`DISTINCT ON (slug)`) over the full
|
||||
candidate set before the user `LIMIT`, via the shared `buildBestPerPagePoolCte`
|
||||
in `sql-ranking.ts`. The vector side returns N distinct pages by best chunk,
|
||||
not N chunks that collapse to fewer pages downstream.
|
||||
- **Title-phrase boost** — when the normalized query is a contiguous token-run
|
||||
inside `page.title` (or an exact full-title match), a floor-ratio-gated,
|
||||
bounded multiplier fires (`applyTitleBoost`, `search.title_boost` knob). A
|
||||
query that is a phrase from the title can't lose to a body chunk by luck.
|
||||
- **Alias hop** — free-text `aliases:` frontmatter is projected into a
|
||||
`page_aliases` table (separate from the `slug_aliases` wikilink redirect) and
|
||||
consulted at query time: a full normalized-query match injects/boosts the
|
||||
canonical page (`applyAliasHop`). The only layer that bridges true synonyms
|
||||
with zero surface overlap ("Hall of Light" → the Mingtang page). Backfill
|
||||
existing pages with `gbrain reindex --aliases`.
|
||||
- **Evidence contract** — every result carries `evidence`
|
||||
(`alias_hit | exact_title_match | high_vector_match | keyword_exact |
|
||||
weak_semantic`) and `create_safety` (`exists | probable | unknown`). An agent
|
||||
deciding "is this page already here, safe to NOT write a duplicate?" keys off
|
||||
`create_safety`, not a raw blended score.
|
||||
|
||||
The `search` MCP/CLI op is **cheap-hybrid** (vector + keyword + RRF + pool +
|
||||
title + alias, expansion off); `query` is the full-control variant. NamedThingBench
|
||||
(`gbrain eval retrieval-quality`) gates these families on every PR. Diagnose a
|
||||
specific miss with `gbrain search diagnose "<q>" --target <slug>`.
|
||||
|
||||
## Intent-aware query rewriting
|
||||
|
||||
`src/core/search/intent.ts` classifies queries into `entity`, `temporal`, `event`, or `general`. Each routes through different ranking knobs:
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
# Retrieval Incident: a chosen-name page was missed, and the fix
|
||||
|
||||
**Status:** Resolved (retrieval-cathedral wave). Supersedes the docs-only RFC in
|
||||
closed PR #1616 — the diagnosis there was directionally right about the disease
|
||||
but wrong on several mechanics; this is the corrected record + what shipped.
|
||||
**Original author:** Garry Tan's OpenClaw. **Severity at the time:** High.
|
||||
**Related:** [`RETRIEVAL.md`](./RETRIEVAL.md), [`../eval/METRIC_GLOSSARY.md`](../eval/METRIC_GLOSSARY.md).
|
||||
|
||||
---
|
||||
|
||||
## 1. What happened
|
||||
|
||||
The agent was asked to log that Garry "wants to build a Greek amphitheater." It
|
||||
ran a retrieval for the concept, the canonical concept page (titled "...Indoor
|
||||
Greek Amphitheater...") did **not** surface with enough confidence to be
|
||||
recognized as the existing page, and the agent wrote a **duplicate stub** on top
|
||||
of a fully-developed concept doc. Garry caught it: "It's in the brain. It's the
|
||||
Hall of Light. Why did you forget?"
|
||||
|
||||
The page is *about* a Greek amphitheater — the phrase is in its title and first
|
||||
sentence. A healthy index returns it at the top. It didn't.
|
||||
|
||||
## 2. The disease (the RFC got this right)
|
||||
|
||||
The brain is stored by **meaning and chosen name** (Mingtang, Hall of Light) but
|
||||
was retrieved by **literal embedding proximity to a body chunk**, and the agent's
|
||||
"is this already here?" decision keyed off a single fuzzy blended score. Three
|
||||
retrieval gaps plus one contract gap produced the miss.
|
||||
|
||||
## 3. Verified ground truth (corrections to the RFC)
|
||||
|
||||
These were checked in code during the fix; several change the remedy:
|
||||
|
||||
1. **`gbrain search` was keyword-only**, not hybrid — so the RFC's cosine scores
|
||||
(0.64/0.98) came from the hybrid `query`/MCP path the agent actually hit, not
|
||||
`gbrain search`. The repro command in the RFC was mislabeled.
|
||||
2. **`--mode` was never a CLI param** — mode resolves server-side from the
|
||||
`search.mode` config key, which is why all three "modes" returned identical
|
||||
results (the flag was silently dropped; `thorough` isn't a real mode).
|
||||
3. **`hybridSearch` already max-pooled per page at the dedup layer.** So the
|
||||
per-page max-pool fix's real win is *candidate-set page recall* (the vector
|
||||
side returned N chunks that could collapse to fewer pages), and it is
|
||||
necessary-but-not-sufficient: if a page's title chunk scores below a body
|
||||
chunk on a 2-word query, or falls outside the candidate pool, pooling alone
|
||||
doesn't rescue it.
|
||||
4. **Frontmatter `aliases:` was dead to search** — stored in `pages.frontmatter`
|
||||
JSONB, never consulted. `slug_aliases` is a *slug→slug* wikilink redirect, a
|
||||
different concept.
|
||||
|
||||
## 4. The fix that shipped (four layers + a contract)
|
||||
|
||||
| Layer | Fixes | Where |
|
||||
|---|---|---|
|
||||
| **Per-page max-pool** (T1) | a page scored by its weakest chunk; vector page-recall | `searchVector` both engines, shared `buildBestPerPagePoolCte` |
|
||||
| **Title-phrase boost** (T2) | query is a phrase in the title but matched a body chunk | `applyTitleBoost` (reads `page.title`), `title_boost` mode knob |
|
||||
| **Alias hop** (T3) | true synonyms with zero surface overlap ("Hall of Light" → Mingtang) | `page_aliases` table, `applyAliasHop`, ingest projection + `reindex --aliases` backfill |
|
||||
| **Evidence contract** (T4) | the agent keyed "don't duplicate" off a fuzzy score | `evidence` + `create_safety` on every result; the agent keys off `create_safety='exists'`, not a threshold |
|
||||
|
||||
Plus: `gbrain search "<text>"` is now cheap-hybrid (the obvious verb gives the
|
||||
good path); `modes/stats/tune` stay subcommands; `--mode` works per-call for
|
||||
local callers; rank-1 score drift telemetry; and **NamedThingBench**, a CI gate
|
||||
that hard-gates the families that ARE this incident.
|
||||
|
||||
## 5. How to confirm / triage a recurrence
|
||||
|
||||
```
|
||||
# Which layer surfaces (or misses) the target page?
|
||||
gbrain search diagnose "Greek amphitheater" --target projects/new-greek-theater/concept_v0
|
||||
|
||||
# Backfill aliases for existing pages whose frontmatter predates the alias layer:
|
||||
gbrain reindex --aliases
|
||||
|
||||
# Watch retrieval quality over time (a downward avg rank-1 score = regressing):
|
||||
gbrain search stats --days 30
|
||||
|
||||
# The gate that prevents silent reintroduction:
|
||||
gbrain eval retrieval-quality test/fixtures/retrieval-quality/namedthing.jsonl
|
||||
```
|
||||
|
||||
For a page to be reliably found by its chosen name, give it `aliases:` frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: The Mingtang — Indoor Greek Amphitheater
|
||||
aliases:
|
||||
- Hall of Light
|
||||
- 明堂
|
||||
---
|
||||
```
|
||||
|
||||
## 6. The discipline this teaches
|
||||
|
||||
A benchmark that scores 97.9 R@5 while production returns a flagship page at 0.64
|
||||
means the benchmark and the shipped path diverged. NamedThingBench runs the same
|
||||
families through the real pipeline on every PR, and the evidence contract means
|
||||
the agent's duplicate-or-not decision is grounded in *why* a page matched, not a
|
||||
number that was never a calibrated probability.
|
||||
@@ -38,6 +38,40 @@ Every metric `gbrain eval *` and `gbrain search stats` reports has a plain-Engli
|
||||
|
||||
**Range:** 0..1, higher is better. nDCG@10 above 0.65 is the common "ship it" threshold for hybrid retrieval on technical corpora.
|
||||
|
||||
## Retrieval-Quality / Evidence Metrics (NamedThingBench)
|
||||
|
||||
### Hit rate at 1 (Hit@1)
|
||||
|
||||
**Key:** `hit@1`
|
||||
|
||||
**Plain English:** Fraction of queries where the right page is the very first result. NamedThingBench hard-gates title-substring Hit@1 >= 0.95 and alias Hit@1 >= 0.98 — a query that is a page's name or title phrase should land it at rank 1, not "somewhere in the top 10".
|
||||
|
||||
**Range:** 0..1, higher is better.
|
||||
|
||||
### Hit rate at 3 (Hit@3)
|
||||
|
||||
**Key:** `hit@3`
|
||||
|
||||
**Plain English:** Fraction of queries where the right page is in the top 3 results. NamedThingBench requires the multi-chunk-dilution family to hit 1.0 — a page with one strong chunk among many weak ones must never be buried.
|
||||
|
||||
**Range:** 0..1, higher is better.
|
||||
|
||||
### Average rank-1 match score
|
||||
|
||||
**Key:** `avg_rank1_score`
|
||||
|
||||
**Plain English:** The mean base (pre-boost) retrieval score of the TOP result across recent searches, from `gbrain search stats`. It is NOT a labeled accuracy number — it is a drift signal: if this trends DOWN over time, retrieval quality is regressing (the early warning that would have caught the duplicate-page incident before a human did).
|
||||
|
||||
**Range:** 0..1. Watch the trend, not the absolute value; pair with the <0.6 / 0.6-0.85 / >=0.85 bucket counts for shape.
|
||||
|
||||
### Create-safety hint (evidence contract)
|
||||
|
||||
**Key:** `create_safety`
|
||||
|
||||
**Plain English:** A result's answer to "is this page already in the brain — safe to NOT write a new one?" Derived from the strongest evidence, NOT a raw score: exists (alias_hit / exact_title_match / high_vector_match — do not duplicate), probable (solid keyword match — prefer updating), unknown (weak match — look closer). An agent keys its don't-duplicate decision off this, which is what prevents the incident's duplicate-stub class.
|
||||
|
||||
**Range:** enum: exists | probable | unknown
|
||||
|
||||
## Set-Similarity / Stability Metrics
|
||||
|
||||
### Jaccard similarity at k (set Jaccard @k)
|
||||
|
||||
+15
-6
File diff suppressed because one or more lines are too long
+1
-1
@@ -141,5 +141,5 @@
|
||||
"bun": ">=1.3.10"
|
||||
},
|
||||
"license": "MIT",
|
||||
"version": "0.41.33.0"
|
||||
"version": "0.41.34.0"
|
||||
}
|
||||
|
||||
+39
@@ -107,6 +107,38 @@ async function main() {
|
||||
command = 'query';
|
||||
}
|
||||
|
||||
// T5 — `gbrain search modes|stats|tune` is the read-only config dashboard,
|
||||
// NOT a free-text search for the literal word "modes". Free-text
|
||||
// `gbrain search "<query>"` falls through to the cheap-hybrid `search` op
|
||||
// below (T4). Preserves the v0.41.6.0 read-only connect+dispatch timeout.
|
||||
if (command === 'search' && ['modes', 'stats', 'tune', 'diagnose'].includes(subArgs[0] ?? '')) {
|
||||
const { withTimeout, OperationTimeoutError } = await import('./core/timeout.ts');
|
||||
const isDiagnose = subArgs[0] === 'diagnose';
|
||||
const label = 'gbrain search';
|
||||
// diagnose runs real retrieval (keyword + vector + hybrid) so it gets a
|
||||
// longer deadline than the read-only dashboard.
|
||||
const timeoutMs = isDiagnose ? 60_000 : 10_000;
|
||||
let engine: BrainEngine;
|
||||
try {
|
||||
engine = await withTimeout(connectEngine(), timeoutMs, `${label}: connect`);
|
||||
} catch (e) {
|
||||
if (e instanceof OperationTimeoutError) { console.error(`${e.label} timed out.`); process.exit(124); }
|
||||
throw e;
|
||||
}
|
||||
try {
|
||||
if (isDiagnose) {
|
||||
const { runSearchDiagnose } = await import('./commands/search-diagnose.ts');
|
||||
await withTimeout(runSearchDiagnose(engine, subArgs), timeoutMs, label);
|
||||
} else {
|
||||
const { runSearch } = await import('./commands/search.ts');
|
||||
await withTimeout(runSearch(engine, subArgs), timeoutMs, label);
|
||||
}
|
||||
} finally {
|
||||
await engine.disconnect();
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// Per-command --help
|
||||
if (hasHelpFlag(subArgs)) {
|
||||
const op = cliOps.get(command);
|
||||
@@ -1438,6 +1470,13 @@ async function handleCliOnly(command: string, args: string[]) {
|
||||
}
|
||||
break;
|
||||
}
|
||||
if (args.includes('--aliases')) {
|
||||
// T8 — backfill the free-text alias layer (page_aliases) for existing
|
||||
// pages whose frontmatter `aliases:` predate the import-time projection.
|
||||
const { runReindexAliases } = await import('./commands/reindex-aliases.ts');
|
||||
await runReindexAliases(engine, args);
|
||||
break;
|
||||
}
|
||||
const { runReindex } = await import('./commands/reindex.ts');
|
||||
await runReindex(engine, args);
|
||||
break;
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
/**
|
||||
* `gbrain eval retrieval-quality <fixture.jsonl> [--json] [--source <id>]`
|
||||
* (T6 — NamedThingBench). Runs the gold query set against the brain's hybrid
|
||||
* retrieval and gates on the families that ARE the retrieval-maxpool incident.
|
||||
*
|
||||
* Run with reranker + expansion at their configured defaults but the gate
|
||||
* measures core retrieval (title/alias/pool) — the families don't depend on
|
||||
* the rescue layers. Exit 0 PASS / 1 FAIL (hard-family breach) / 2 USAGE.
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import { readFileSync } from 'fs';
|
||||
import { hybridSearch } from '../core/search/hybrid.ts';
|
||||
import {
|
||||
parseQuestionsJsonl,
|
||||
runRetrievalQuality,
|
||||
evaluateGate,
|
||||
type SearchFn,
|
||||
} from '../eval/retrieval-quality/harness.ts';
|
||||
|
||||
export async function runEvalRetrievalQuality(engine: BrainEngine, args: string[]): Promise<void> {
|
||||
const json = args.includes('--json');
|
||||
const sourceIdx = args.indexOf('--source');
|
||||
const sourceId = sourceIdx >= 0 ? args[sourceIdx + 1] : undefined;
|
||||
const fixture = args.find(a => !a.startsWith('--') && a !== sourceId);
|
||||
|
||||
if (!fixture) {
|
||||
console.error('Usage: gbrain eval retrieval-quality <fixture.jsonl> [--json] [--source <id>]');
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
let questions;
|
||||
try {
|
||||
questions = parseQuestionsJsonl(readFileSync(fixture, 'utf8'));
|
||||
} catch (e) {
|
||||
console.error(`Cannot read fixture: ${e instanceof Error ? e.message : String(e)}`);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
// Core-retrieval measurement: reranker/expansion at config defaults; the
|
||||
// families key off title/alias/pool which are upstream of the rescue layers.
|
||||
const searchFn: SearchFn = async (q) => {
|
||||
const results = await hybridSearch(engine, q, {
|
||||
limit: 10,
|
||||
...(sourceId ? { sourceId } : {}),
|
||||
});
|
||||
return results.map(r => r.slug);
|
||||
};
|
||||
|
||||
const report = await runRetrievalQuality(questions, searchFn);
|
||||
const gate = evaluateGate(report);
|
||||
|
||||
if (json) {
|
||||
console.log(JSON.stringify({ schema_version: 1, report, gate }, null, 2));
|
||||
} else {
|
||||
console.log(`NamedThingBench — ${report.total} queries across ${report.families.length} families\n`);
|
||||
for (const f of report.families) {
|
||||
console.log(` ${f.family.padEnd(22)} n=${f.n} Hit@1=${(f.hit_at_1 * 100).toFixed(0)}% Hit@3=${(f.hit_at_3 * 100).toFixed(0)}% MRR=${f.mrr.toFixed(3)}`);
|
||||
}
|
||||
console.log('');
|
||||
if (gate.breaches.length) {
|
||||
console.log('GATE: FAIL');
|
||||
for (const b of gate.breaches) {
|
||||
console.log(` ✗ ${b.family} ${b.metric}=${(b.got * 100).toFixed(0)}% < floor ${(b.floor * 100).toFixed(0)}%`);
|
||||
}
|
||||
} else {
|
||||
console.log('GATE: PASS');
|
||||
}
|
||||
for (const w of gate.warnings) {
|
||||
console.log(` ⚠ (warn) ${w.family} ${w.metric}=${(w.got * 100).toFixed(0)}% < ${(w.floor * 100).toFixed(0)}%`);
|
||||
}
|
||||
}
|
||||
|
||||
process.exit(gate.pass ? 0 : 1);
|
||||
}
|
||||
@@ -60,6 +60,12 @@ export async function runEvalCommand(engine: BrainEngine, args: string[]): Promi
|
||||
const { runEvalCodeRetrieval } = await import('./eval-code-retrieval.ts');
|
||||
return runEvalCodeRetrieval(engine, args.slice(1));
|
||||
}
|
||||
if (sub === 'retrieval-quality') {
|
||||
// T6 — NamedThingBench. Gold query set vs hybrid retrieval; gates the
|
||||
// families that ARE the retrieval-maxpool incident (title/alias/dilution).
|
||||
const { runEvalRetrievalQuality } = await import('./eval-retrieval-quality.ts');
|
||||
return runEvalRetrievalQuality(engine, args.slice(1));
|
||||
}
|
||||
if (sub === 'brainstorm') {
|
||||
// v0.37.0 (D3 + codex r2 #11) — three-axis evaluation gate for the
|
||||
// brainstorm + LSD wave. Engine connected (calls hybridSearch +
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
/**
|
||||
* `gbrain reindex --aliases [--limit N] [--dry-run] [--json] [--source <id>]`
|
||||
* (T8 — backfill the free-text alias layer).
|
||||
*
|
||||
* Import-time projection (T3) covers NEW + changed pages; this backfills the
|
||||
* EXISTING pages whose frontmatter `aliases:` predate v110 (or predate the
|
||||
* projection landing). Reads each page's frontmatter `aliases:` and writes
|
||||
* page_aliases via engine.setPageAliases.
|
||||
*
|
||||
* Idempotent: setPageAliases replaces a page's alias set, so re-running is
|
||||
* safe and convergent — no op-checkpoint needed (the op is fast, no embedding).
|
||||
* Walks listAllPageRefs (cheap (source_id, slug) enumeration) so it's
|
||||
* cross-source by default; --source narrows it.
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import { normalizeAliasList } from '../core/search/alias-normalize.ts';
|
||||
import { createProgress } from '../core/progress.ts';
|
||||
import { getCliOptions, cliOptsToProgressOptions } from '../core/cli-options.ts';
|
||||
|
||||
export interface ReindexAliasesResult {
|
||||
scanned: number;
|
||||
pages_with_aliases: number;
|
||||
aliases_written: number;
|
||||
dry_run: boolean;
|
||||
}
|
||||
|
||||
export async function runReindexAliases(engine: BrainEngine, args: string[]): Promise<ReindexAliasesResult> {
|
||||
const dryRun = args.includes('--dry-run');
|
||||
const json = args.includes('--json');
|
||||
const limitIdx = args.indexOf('--limit');
|
||||
const limit = limitIdx >= 0 ? parseInt(args[limitIdx + 1] ?? '', 10) : NaN;
|
||||
const sourceIdx = args.indexOf('--source');
|
||||
const sourceFilter = sourceIdx >= 0 ? args[sourceIdx + 1] : undefined;
|
||||
|
||||
let refs = await engine.listAllPageRefs();
|
||||
if (sourceFilter) refs = refs.filter(r => r.source_id === sourceFilter);
|
||||
if (Number.isFinite(limit) && limit > 0) refs = refs.slice(0, limit);
|
||||
|
||||
const reporter = createProgress(cliOptsToProgressOptions(getCliOptions()));
|
||||
reporter.start('reindex.aliases', refs.length);
|
||||
|
||||
let scanned = 0;
|
||||
let pagesWithAliases = 0;
|
||||
let aliasesWritten = 0;
|
||||
|
||||
for (const ref of refs) {
|
||||
scanned++;
|
||||
reporter.tick();
|
||||
let page;
|
||||
try {
|
||||
page = await engine.getPage(ref.slug, { sourceId: ref.source_id });
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (!page) continue;
|
||||
const aliasNorms = normalizeAliasList((page.frontmatter as Record<string, unknown> | undefined)?.aliases);
|
||||
if (aliasNorms.length === 0) continue;
|
||||
pagesWithAliases++;
|
||||
aliasesWritten += aliasNorms.length;
|
||||
if (!dryRun) {
|
||||
try {
|
||||
await engine.setPageAliases(ref.slug, ref.source_id, aliasNorms);
|
||||
} catch (e) {
|
||||
reporter.finish();
|
||||
throw e; // pre-v110 (no table) or a real write error — surface it.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
reporter.finish();
|
||||
const result: ReindexAliasesResult = {
|
||||
scanned,
|
||||
pages_with_aliases: pagesWithAliases,
|
||||
aliases_written: aliasesWritten,
|
||||
dry_run: dryRun,
|
||||
};
|
||||
|
||||
if (json) {
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
} else {
|
||||
const verb = dryRun ? 'would write' : 'wrote';
|
||||
console.log(`reindex --aliases: scanned ${scanned} pages, ${verb} ${aliasesWritten} aliases across ${pagesWithAliases} pages.`);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
/**
|
||||
* `gbrain search diagnose "<query>" --target <slug> [--json] [--source <id>]`
|
||||
* (T0 — Phase-0 retrieval diagnostic).
|
||||
*
|
||||
* Traces WHERE a target page surfaces (or fails to) across the retrieval
|
||||
* pipeline, so an operator can pin which layer is responsible for an incident
|
||||
* like "Greek amphitheater missed the Mingtang page":
|
||||
*
|
||||
* keyword — rank + score of the target in searchKeyword (ts_rank path)
|
||||
* vector — rank + score of the target in searchVector (per-page max-pool);
|
||||
* best-chunk cosine; skipped if no embedding provider
|
||||
* alias — is the normalized query a registered page_aliases alias of the target?
|
||||
* hybrid — final rank + evidence + create_safety + which boosts fired
|
||||
*
|
||||
* The verdict names the layer that DOES surface the target (or "none"), which
|
||||
* tells you whether the fix is max-pool/innerLimit (vector) vs title/alias.
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import { hybridSearch } from '../core/search/hybrid.ts';
|
||||
import { normalizeAlias } from '../core/search/alias-normalize.ts';
|
||||
|
||||
interface LayerProbe {
|
||||
rank: number | null; // 1-based rank of the target, null if absent
|
||||
score: number | null;
|
||||
top_slug: string | null;
|
||||
note?: string;
|
||||
}
|
||||
|
||||
function rankOf(slugs: string[], target: string): number | null {
|
||||
const i = slugs.indexOf(target);
|
||||
return i >= 0 ? i + 1 : null;
|
||||
}
|
||||
|
||||
export async function runSearchDiagnose(engine: BrainEngine, args: string[]): Promise<void> {
|
||||
const json = args.includes('--json');
|
||||
const targetIdx = args.indexOf('--target');
|
||||
const target = targetIdx >= 0 ? args[targetIdx + 1] : undefined;
|
||||
const sourceIdx = args.indexOf('--source');
|
||||
const sourceId = sourceIdx >= 0 ? args[sourceIdx + 1] : undefined;
|
||||
// 'diagnose' is args[0]; the query is the first non-flag token after it.
|
||||
const query = args.slice(1).find((a, i) => !a.startsWith('--') && args.slice(1)[i - 1] !== '--target' && args.slice(1)[i - 1] !== '--source');
|
||||
|
||||
if (!query || !target) {
|
||||
console.error('Usage: gbrain search diagnose "<query>" --target <slug> [--json] [--source <id>]');
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
const scope = sourceId ? { sourceId } : {};
|
||||
const limit = 50;
|
||||
|
||||
// Keyword layer.
|
||||
const kw = await engine.searchKeyword(query, { limit, ...scope });
|
||||
const keyword: LayerProbe = {
|
||||
rank: rankOf(kw.map(r => r.slug), target),
|
||||
score: kw.find(r => r.slug === target)?.score ?? null,
|
||||
top_slug: kw[0]?.slug ?? null,
|
||||
};
|
||||
|
||||
// Vector layer (skip if no embedding provider).
|
||||
let vector: LayerProbe;
|
||||
const { isAvailable, embedQuery } = await import('../core/ai/gateway.ts');
|
||||
if (!isAvailable('embedding')) {
|
||||
vector = { rank: null, score: null, top_slug: null, note: 'skipped — no embedding provider configured' };
|
||||
} else {
|
||||
try {
|
||||
const qvec = await embedQuery(query);
|
||||
const vec = await engine.searchVector(qvec, { limit, ...scope });
|
||||
vector = {
|
||||
rank: rankOf(vec.map(r => r.slug), target),
|
||||
score: vec.find(r => r.slug === target)?.score ?? null,
|
||||
top_slug: vec[0]?.slug ?? null,
|
||||
};
|
||||
} catch (e) {
|
||||
vector = { rank: null, score: null, top_slug: null, note: `vector probe failed: ${e instanceof Error ? e.message : String(e)}` };
|
||||
}
|
||||
}
|
||||
|
||||
// Alias layer.
|
||||
const qNorm = normalizeAlias(query);
|
||||
let aliasMatch = false;
|
||||
try {
|
||||
const m = await engine.resolveAliases([qNorm], scope);
|
||||
aliasMatch = (m.get(qNorm) ?? []).some(r => r.slug === target);
|
||||
} catch { /* pre-v110: no alias layer */ }
|
||||
|
||||
// Hybrid layer (the production path).
|
||||
const hy = await hybridSearch(engine, query, { limit, ...scope });
|
||||
const hyTarget = hy.find(r => r.slug === target);
|
||||
const hybrid = {
|
||||
rank: rankOf(hy.map(r => r.slug), target),
|
||||
score: hyTarget?.score ?? null,
|
||||
base_score: hyTarget?.base_score ?? null,
|
||||
evidence: hyTarget?.evidence ?? null,
|
||||
create_safety: hyTarget?.create_safety ?? null,
|
||||
title_match_boost: hyTarget?.title_match_boost ?? null,
|
||||
alias_hit: hyTarget?.alias_hit ?? false,
|
||||
top_slug: hy[0]?.slug ?? null,
|
||||
};
|
||||
|
||||
// Verdict: which layer surfaces the target at rank 1?
|
||||
const surfacing: string[] = [];
|
||||
if (keyword.rank === 1) surfacing.push('keyword');
|
||||
if (vector.rank === 1) surfacing.push('vector');
|
||||
if (aliasMatch) surfacing.push('alias');
|
||||
if (hybrid.rank === 1) surfacing.push('hybrid(final)');
|
||||
const verdict = hybrid.rank === 1
|
||||
? `target is rank 1 in hybrid (via: ${surfacing.join(', ') || 'unknown'})`
|
||||
: hybrid.rank
|
||||
? `target is rank ${hybrid.rank} in hybrid — NOT rank 1. Strongest layer: ${surfacing.join(', ') || 'none'}`
|
||||
: `target ABSENT from hybrid top-${limit}. keyword rank=${keyword.rank ?? 'absent'}, vector rank=${vector.rank ?? 'absent'}, alias=${aliasMatch}`;
|
||||
|
||||
const report = { query, target, source_id: sourceId ?? 'default', keyword, vector, alias_match: aliasMatch, hybrid, verdict };
|
||||
|
||||
if (json) {
|
||||
console.log(JSON.stringify(report, null, 2));
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`Diagnose: "${query}" → target ${target}\n`);
|
||||
console.log(` keyword: rank=${keyword.rank ?? 'absent'} score=${fmt(keyword.score)} (top: ${keyword.top_slug ?? '—'})`);
|
||||
console.log(` vector: rank=${vector.rank ?? 'absent'} score=${fmt(vector.score)} (top: ${vector.top_slug ?? '—'})${vector.note ? ` [${vector.note}]` : ''}`);
|
||||
console.log(` alias: ${aliasMatch ? 'query IS a registered alias of the target' : 'no alias match'}`);
|
||||
console.log(` hybrid: rank=${hybrid.rank ?? 'absent'} score=${fmt(hybrid.score)} base=${fmt(hybrid.base_score)} evidence=${hybrid.evidence ?? '—'} create_safety=${hybrid.create_safety ?? '—'}`);
|
||||
console.log(` boosts: title=${hybrid.title_match_boost ?? '—'} alias_hit=${hybrid.alias_hit}`);
|
||||
console.log(`\n VERDICT: ${verdict}`);
|
||||
}
|
||||
|
||||
function fmt(n: number | null): string {
|
||||
return n === null ? '—' : n.toFixed(4);
|
||||
}
|
||||
@@ -54,6 +54,7 @@ const KNOB_DESCRIPTIONS: Record<keyof ModeBundle, string> = {
|
||||
reranker_top_n_out: 'Cap on reranked output (null = no truncate)',
|
||||
reranker_timeout_ms: 'HTTP timeout for the reranker call',
|
||||
floor_ratio: 'Floor-ratio gate for metadata boosts (0..1, undefined = off)',
|
||||
title_boost: 'Title-phrase boost multiplier (query is a title token-run; 1.0 = off)',
|
||||
// v0.36 cross-modal knobs (D3 registry)
|
||||
cross_modal_both_text_weight: "D6 'both'-mode RRF weight for text branch (0.6 default)",
|
||||
cross_modal_both_image_weight: "D6 'both'-mode RRF weight for image branch (0.4 default)",
|
||||
@@ -253,6 +254,12 @@ async function runStatsSubcommand(engine: BrainEngine, args: string[]): Promise<
|
||||
console.log(` Avg results returned: ${stats.avg_results.toFixed(1)}`);
|
||||
console.log(` Avg tokens delivered: ${stats.avg_tokens.toFixed(0)} (char/4 heuristic)`);
|
||||
console.log(` Budget drops total: ${stats.total_budget_dropped}`);
|
||||
// T7 — rank-1 match-quality drift signal. Watch for avg drifting DOWN.
|
||||
if (stats.avg_rank1_score !== null && stats.rank1_count > 0) {
|
||||
const d = stats.rank1_distribution;
|
||||
console.log(` Avg rank-1 score: ${stats.avg_rank1_score.toFixed(3)} (${stats.rank1_count} samples; <0.6:${d.lt_solid} 0.6-0.85:${d.solid} >=0.85:${d.high})`);
|
||||
console.log(` (top-result match quality; a downward drift = retrieval regressing)`);
|
||||
}
|
||||
console.log('');
|
||||
console.log(' Mode distribution:');
|
||||
for (const [m, c] of Object.entries(stats.mode_distribution).sort((a, b) => b[1] - a[1])) {
|
||||
|
||||
@@ -1723,6 +1723,38 @@ export interface BrainEngine {
|
||||
sourceOrSources: string | readonly string[],
|
||||
): Promise<string>;
|
||||
|
||||
/**
|
||||
* T3 retrieval-cathedral — free-text alias resolution for SEARCH.
|
||||
* Given a set of NORMALIZED alias strings (normalizeAlias() output),
|
||||
* return a map alias_norm -> canonical slugs from page_aliases. An alias
|
||||
* may resolve to MORE THAN ONE slug (two pages claimed the same name) —
|
||||
* the caller reports the collision and resolves deterministically.
|
||||
*
|
||||
* Source-scoped: pass scalar `sourceId` (single-source) OR `sourceIds`
|
||||
* (federated read) — array wins when both set, matching sourceScopeOpts.
|
||||
* Empty input → empty map (no query). Throws "relation page_aliases does
|
||||
* not exist" on pre-v110 brains; the alias-hop caller wraps in try/catch
|
||||
* (isUndefinedColumnError) and degrades to no-injection (D9 fail-open).
|
||||
*/
|
||||
resolveAliases(
|
||||
aliasNorms: string[],
|
||||
opts?: { sourceId?: string; sourceIds?: string[] },
|
||||
): Promise<Map<string, Array<{ slug: string; source_id: string }>>>;
|
||||
|
||||
/**
|
||||
* T3 retrieval-cathedral — WRITE side of the alias layer. Replace the full
|
||||
* alias set for one (slug, source) with `aliasNorms` (already normalized):
|
||||
* delete the page's existing page_aliases rows, insert the new set. Empty
|
||||
* `aliasNorms` just clears them. Idempotent (the unique triple). Called by
|
||||
* the ingest projection in importFromContent and the `reindex --aliases`
|
||||
* backfill so a page's declared aliases stay in lockstep with its frontmatter.
|
||||
*/
|
||||
setPageAliases(
|
||||
slug: string,
|
||||
sourceId: string,
|
||||
aliasNorms: string[],
|
||||
): Promise<void>;
|
||||
|
||||
/**
|
||||
* v0.35.5 — narrow UPDATE of `pages.compiled_truth`, `pages.timeline`, and
|
||||
* `pages.content_hash` for a single slug+source. NO chunking, NO embedding,
|
||||
|
||||
@@ -53,6 +53,30 @@ export const METRIC_GLOSSARY: Readonly<Record<string, Readonly<MetricGlossEntry>
|
||||
range: '0..1, higher is better. nDCG@10 above 0.65 is the common "ship it" threshold for hybrid retrieval on technical corpora.',
|
||||
}),
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
// NamedThingBench (retrieval-quality) + the agent-facing evidence contract
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
'hit@1': Object.freeze({
|
||||
industry_term: 'Hit rate at 1 (Hit@1)',
|
||||
eli10: 'Fraction of queries where the right page is the very first result. NamedThingBench hard-gates title-substring Hit@1 >= 0.95 and alias Hit@1 >= 0.98 — a query that is a page\'s name or title phrase should land it at rank 1, not "somewhere in the top 10".',
|
||||
range: '0..1, higher is better.',
|
||||
}),
|
||||
'hit@3': Object.freeze({
|
||||
industry_term: 'Hit rate at 3 (Hit@3)',
|
||||
eli10: 'Fraction of queries where the right page is in the top 3 results. NamedThingBench requires the multi-chunk-dilution family to hit 1.0 — a page with one strong chunk among many weak ones must never be buried.',
|
||||
range: '0..1, higher is better.',
|
||||
}),
|
||||
'avg_rank1_score': Object.freeze({
|
||||
industry_term: 'Average rank-1 match score',
|
||||
eli10: 'The mean base (pre-boost) retrieval score of the TOP result across recent searches, from `gbrain search stats`. It is NOT a labeled accuracy number — it is a drift signal: if this trends DOWN over time, retrieval quality is regressing (the early warning that would have caught the duplicate-page incident before a human did).',
|
||||
range: '0..1. Watch the trend, not the absolute value; pair with the <0.6 / 0.6-0.85 / >=0.85 bucket counts for shape.',
|
||||
}),
|
||||
'create_safety': Object.freeze({
|
||||
industry_term: 'Create-safety hint (evidence contract)',
|
||||
eli10: 'A result\'s answer to "is this page already in the brain — safe to NOT write a new one?" Derived from the strongest evidence, NOT a raw score: exists (alias_hit / exact_title_match / high_vector_match — do not duplicate), probable (solid keyword match — prefer updating), unknown (weak match — look closer). An agent keys its don\'t-duplicate decision off this, which is what prevents the incident\'s duplicate-stub class.',
|
||||
range: 'enum: exists | probable | unknown',
|
||||
}),
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
// Set-similarity / stability metrics (replay + regression checks)
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
@@ -182,6 +206,7 @@ export function renderMetricGlossaryMarkdown(): string {
|
||||
|
||||
const groups: Array<[string, string[]]> = [
|
||||
['Retrieval Metrics', ['precision@k', 'recall@k', 'mrr', 'ndcg@k']],
|
||||
['Retrieval-Quality / Evidence Metrics (NamedThingBench)', ['hit@1', 'hit@3', 'avg_rank1_score', 'create_safety']],
|
||||
['Set-Similarity / Stability Metrics', ['jaccard@k', 'top1_stability']],
|
||||
['Statistical-Significance Metrics', ['p_value', 'confidence_interval']],
|
||||
['Operational / Cost Metrics', ['cache_hit_rate', 'avg_results', 'avg_tokens', 'cost_per_query_usd', 'p99_latency_ms']],
|
||||
|
||||
@@ -28,6 +28,8 @@ import {
|
||||
wrapChunkForEmbedding,
|
||||
} from './embedding-context.ts';
|
||||
import { loadSearchModeConfig, resolveSearchMode } from './search/mode.ts';
|
||||
import { normalizeAliasList } from './search/alias-normalize.ts';
|
||||
import { isUndefinedTableError, warnOncePerProcess } from './utils.ts';
|
||||
import { computeCorpusGeneration } from './contextual-retrieval-service.ts';
|
||||
|
||||
/**
|
||||
@@ -708,6 +710,25 @@ export async function importFromContent(
|
||||
}
|
||||
});
|
||||
|
||||
// T3 — project frontmatter `aliases:` into page_aliases (free-text alias
|
||||
// resolution for search). Runs AFTER the page write commits so the slug
|
||||
// exists. Fail-soft: a pre-v110 brain has no page_aliases table yet (the
|
||||
// migration may not have run); an alias-write failure must NOT fail the
|
||||
// import. Always called (even with []) so REMOVING an alias from frontmatter
|
||||
// clears its row — the content_hash includes non-timestamp frontmatter, so
|
||||
// an alias edit changes the hash and reaches this path (not the skip branch).
|
||||
try {
|
||||
const aliasNorms = normalizeAliasList((parsed.frontmatter as Record<string, unknown>).aliases);
|
||||
await engine.setPageAliases(slug, sourceId ?? 'default', aliasNorms);
|
||||
} catch (e) {
|
||||
if (!isUndefinedTableError(e)) {
|
||||
warnOncePerProcess(
|
||||
'setPageAliases:failed',
|
||||
`[import] page_aliases projection failed (non-fatal): ${e instanceof Error ? e.message : String(e)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return { slug, status: 'imported', chunks: chunks.length, parsedPage };
|
||||
}
|
||||
|
||||
|
||||
@@ -4962,6 +4962,67 @@ export const MIGRATIONS: Migration[] = [
|
||||
idempotent: true,
|
||||
sql: `ALTER TABLE sources ADD COLUMN IF NOT EXISTS newest_content_at TIMESTAMPTZ`,
|
||||
},
|
||||
{
|
||||
version: 110,
|
||||
name: 'page_aliases',
|
||||
// T3 of the retrieval-cathedral wave (retrieval-maxpool incident).
|
||||
//
|
||||
// Free-text alias resolution: a query like "Hall of Light" or "明堂"
|
||||
// should surface the page titled "Mingtang". gbrain stored that mapping
|
||||
// in pages.frontmatter `aliases:` JSONB but it was invisible to search.
|
||||
//
|
||||
// DELIBERATELY SEPARATE from slug_aliases (v105). They answer different
|
||||
// questions and overloading one for both would muddy the semantics:
|
||||
// - slug_aliases: old-slug -> canonical-slug (wikilink/get_page redirect)
|
||||
// - page_aliases: normalized free-text name -> canonical slug (search hop)
|
||||
//
|
||||
// alias_norm is the output of normalizeAlias() (NFKC + lowercase + ws
|
||||
// collapse) so the WRITE side (ingest projection) and READ side (search)
|
||||
// match on the same key. Btree on (source_id, alias_norm) so the hop is an
|
||||
// indexed equality lookup, not ILIKE.
|
||||
//
|
||||
// NOT a UNIQUE(source_id, alias_norm) — real brains may legitimately have
|
||||
// two pages claiming the same alias; we report the collision and resolve
|
||||
// deterministically at query time rather than failing the ingest (Codex#8).
|
||||
// The (source_id, alias_norm, slug) triple is unique so re-ingest is
|
||||
// idempotent without blocking a second page's claim on the same alias.
|
||||
//
|
||||
// Mirror in src/core/pglite-schema.ts (fresh install); forward-reference
|
||||
// bootstrap probe on both engines so pre-v110 brains pick it up cleanly.
|
||||
idempotent: true,
|
||||
sql: `
|
||||
CREATE TABLE IF NOT EXISTS page_aliases (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
source_id TEXT NOT NULL,
|
||||
alias_norm TEXT NOT NULL,
|
||||
slug TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT page_aliases_uniq UNIQUE (source_id, alias_norm, slug)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS page_aliases_lookup_idx
|
||||
ON page_aliases (source_id, alias_norm);
|
||||
CREATE INDEX IF NOT EXISTS page_aliases_slug_idx
|
||||
ON page_aliases (source_id, slug);
|
||||
`,
|
||||
},
|
||||
{
|
||||
version: 111,
|
||||
name: 'search_telemetry_rank1_columns',
|
||||
// T7 of the retrieval-cathedral wave — rank-1 base_score drift signal.
|
||||
// Aggregate columns (NOT per-query rows, D10) so a downward drift in the
|
||||
// median rank-1 match score is computable from the existing day/mode/intent
|
||||
// rollup with bounded growth. search_telemetry lives only in migration v57
|
||||
// (not the schema blobs), so these are ADD COLUMN IF NOT EXISTS on both
|
||||
// engines; fresh installs pick them up right after v57 runs.
|
||||
idempotent: true,
|
||||
sql: `
|
||||
ALTER TABLE search_telemetry ADD COLUMN IF NOT EXISTS sum_rank1_score DOUBLE PRECISION NOT NULL DEFAULT 0;
|
||||
ALTER TABLE search_telemetry ADD COLUMN IF NOT EXISTS count_rank1 INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE search_telemetry ADD COLUMN IF NOT EXISTS rank1_lt_solid INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE search_telemetry ADD COLUMN IF NOT EXISTS rank1_solid INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE search_telemetry ADD COLUMN IF NOT EXISTS rank1_high INTEGER NOT NULL DEFAULT 0;
|
||||
`,
|
||||
},
|
||||
];
|
||||
|
||||
export const LATEST_VERSION = MIGRATIONS.length > 0
|
||||
|
||||
+93
-34
@@ -21,6 +21,9 @@ import { isFactsBackstopEligible } from './facts/eligibility.ts';
|
||||
import { stripTakesFence } from './takes-fence.ts';
|
||||
import { stripFactsFence } from './facts-fence.ts';
|
||||
import { bumpLastRetrievedAt } from './last-retrieved.ts';
|
||||
import { isSearchMode } from './search/mode.ts';
|
||||
import { stampEvidence } from './search/evidence.ts';
|
||||
import type { SearchResult } from './types.ts';
|
||||
import { CJK_SLUG_CHARS } from './cjk.ts';
|
||||
import * as db from './db.ts';
|
||||
import { VERSION } from '../version.ts';
|
||||
@@ -419,6 +422,59 @@ export function sourceScopeOpts(ctx: OperationContext): { sourceId?: string; sou
|
||||
return {};
|
||||
}
|
||||
|
||||
/**
|
||||
* T4/D5 — resolve a per-call search-mode override. Honored ONLY for trusted/
|
||||
* local callers (ctx.remote === false) so a remote OAuth client can't escalate
|
||||
* to the costly tokenmax bundle. Local + unknown mode → loud reject; remote +
|
||||
* mode → silently ignored (server-configured mode wins). Returns undefined to
|
||||
* mean "use the configured mode".
|
||||
*/
|
||||
export function resolvePerCallMode(ctx: OperationContext, raw: unknown): string | undefined {
|
||||
if (typeof raw !== 'string' || raw.length === 0) return undefined;
|
||||
if (ctx.remote !== false) return undefined; // remote can't select mode
|
||||
if (!isSearchMode(raw)) {
|
||||
throw new OperationError(
|
||||
'invalid_params',
|
||||
`Unknown search mode '${raw}'. Valid: conservative, balanced, tokenmax.`,
|
||||
`gbrain search "<query>" --mode balanced`,
|
||||
);
|
||||
}
|
||||
return raw;
|
||||
}
|
||||
|
||||
/** T4 — stamp evidence/create_safety on a result set, fail-soft. */
|
||||
function stampEvidenceSafe(results: SearchResult[]): void {
|
||||
try { stampEvidence(results); } catch { /* non-fatal */ }
|
||||
}
|
||||
|
||||
/** T4 — shared eval-capture for the `search` op (keyword-only + cheap-hybrid paths). */
|
||||
function maybeCaptureSearch(
|
||||
ctx: OperationContext,
|
||||
queryText: string,
|
||||
results: SearchResult[],
|
||||
latency_ms: number,
|
||||
vectorEnabled: boolean,
|
||||
meta?: HybridSearchMeta | null,
|
||||
): void {
|
||||
if (!isEvalCaptureEnabled(ctx.config)) return;
|
||||
void captureEvalCandidate(
|
||||
ctx.engine,
|
||||
{
|
||||
tool_name: 'search',
|
||||
query: queryText,
|
||||
results,
|
||||
meta: meta ?? { vector_enabled: vectorEnabled, detail_resolved: null, expansion_applied: false },
|
||||
latency_ms,
|
||||
remote: ctx.remote ?? false,
|
||||
expand_enabled: false,
|
||||
detail: null,
|
||||
job_id: ctx.jobId ?? null,
|
||||
subagent_id: ctx.subagentId ?? null,
|
||||
},
|
||||
{ scrub_pii: isEvalScrubEnabled(ctx.config) },
|
||||
);
|
||||
}
|
||||
|
||||
export interface Operation {
|
||||
name: string;
|
||||
description: string;
|
||||
@@ -1194,48 +1250,48 @@ const search: Operation = {
|
||||
query: { type: 'string', required: true },
|
||||
limit: { type: 'number', description: 'Max results (default 20)' },
|
||||
offset: { type: 'number', description: 'Skip first N results (for pagination)' },
|
||||
mode: { type: 'string', description: 'Search mode (conservative|balanced|tokenmax). Local callers only.' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
const startedAt = Date.now();
|
||||
const queryText = p.query as string;
|
||||
// v0.34.1 (#861 — P0 leak seal): thread caller's source scope into
|
||||
// searchKeyword. Pre-fix this op silently returned cross-source hits
|
||||
// for any auth'd OAuth client.
|
||||
const raw = await ctx.engine.searchKeyword(queryText, {
|
||||
limit: (p.limit as number) || 20,
|
||||
offset: (p.offset as number) || 0,
|
||||
...sourceScopeOpts(ctx),
|
||||
});
|
||||
const results = dedupResults(raw);
|
||||
const latency_ms = Date.now() - startedAt;
|
||||
const limit = (p.limit as number) || 20;
|
||||
const offset = (p.offset as number) || 0;
|
||||
const scope = sourceScopeOpts(ctx);
|
||||
|
||||
// v0.37.0 (D11): op-layer last_retrieved_at write-back. Fire-and-forget;
|
||||
// results already returned by engine, this just marks them as user-surfaced
|
||||
// for LSD's stale-page signal. 5-min throttle inside bumpLastRetrievedAt.
|
||||
bumpLastRetrievedAt(ctx.engine, results.map((r) => r.page_id));
|
||||
// T4/D5 — per-call mode honored ONLY for trusted/local callers so a remote
|
||||
// OAuth client can't escalate to the costly tokenmax bundle. Local + unknown
|
||||
// mode → loud reject; remote + mode set → silently ignored (uses config).
|
||||
const perCallMode = resolvePerCallMode(ctx, p.mode);
|
||||
|
||||
// Op-layer capture (v0.25.0). Fire-and-forget — no await on the
|
||||
// capture call so MCP response latency is unaffected. search has
|
||||
// no expand/detail/vector semantics so meta fields are fixed.
|
||||
if (isEvalCaptureEnabled(ctx.config)) {
|
||||
void captureEvalCandidate(
|
||||
ctx.engine,
|
||||
{
|
||||
tool_name: 'search',
|
||||
query: queryText,
|
||||
results,
|
||||
meta: { vector_enabled: false, detail_resolved: null, expansion_applied: false },
|
||||
latency_ms,
|
||||
remote: ctx.remote ?? false,
|
||||
expand_enabled: null,
|
||||
detail: null,
|
||||
job_id: ctx.jobId ?? null,
|
||||
subagent_id: ctx.subagentId ?? null,
|
||||
},
|
||||
{ scrub_pii: isEvalScrubEnabled(ctx.config) },
|
||||
);
|
||||
// T4/D17 — escape hatch: keyword-only when the operator opts out of the
|
||||
// hybrid `search` contract (privacy/cost: no query text to an embedding
|
||||
// provider). Defaults to cheap-hybrid (D4/D15).
|
||||
const keywordOnly = (await ctx.engine.getConfig('search.mcp_keyword_only')) === 'true';
|
||||
|
||||
if (keywordOnly) {
|
||||
const raw = await ctx.engine.searchKeyword(queryText, { limit, offset, ...scope });
|
||||
const results = dedupResults(raw);
|
||||
stampEvidenceSafe(results);
|
||||
bumpLastRetrievedAt(ctx.engine, results.map((r) => r.page_id));
|
||||
maybeCaptureSearch(ctx, queryText, results, Date.now() - startedAt, false);
|
||||
return results;
|
||||
}
|
||||
|
||||
// Cheap-hybrid (D4/D15): full vector+keyword+RRF+pool+title+alias, but
|
||||
// expansion OFF (no per-call LLM cost). `query` op is the full-control variant.
|
||||
let capturedMeta: HybridSearchMeta | null = null;
|
||||
const results = await hybridSearchCached(ctx.engine, queryText, {
|
||||
limit,
|
||||
offset,
|
||||
expansion: false,
|
||||
...scope,
|
||||
...(perCallMode ? { mode: perCallMode } : {}),
|
||||
onMeta: (m) => { capturedMeta = m; },
|
||||
});
|
||||
const latency_ms = Date.now() - startedAt;
|
||||
bumpLastRetrievedAt(ctx.engine, results.map((r) => r.page_id));
|
||||
maybeCaptureSearch(ctx, queryText, results, latency_ms, true, capturedMeta);
|
||||
return results;
|
||||
},
|
||||
scope: 'read',
|
||||
@@ -1261,6 +1317,7 @@ const query: Operation = {
|
||||
offset: { type: 'number', description: 'Skip first N results (for pagination)' },
|
||||
expand: { type: 'boolean', description: 'Enable multi-query expansion (default: true)' },
|
||||
detail: { type: 'string', description: 'Result detail level: low (compiled truth only), medium (default, all with dedup), high (all chunks)' },
|
||||
mode: { type: 'string', description: 'Search mode (conservative|balanced|tokenmax). Local callers only; remote uses configured mode.' },
|
||||
// v0.20.0 Cathedral II Layer 10 C1/C2: language + symbol-kind filters.
|
||||
lang: { type: 'string', description: 'Filter to chunks where content_chunks.language matches (e.g., typescript, python, ruby)' },
|
||||
symbol_kind: { type: 'string', description: 'Filter to chunks where content_chunks.symbol_type matches (e.g., function, class, method, type, interface)' },
|
||||
@@ -1391,6 +1448,8 @@ const query: Operation = {
|
||||
offset: (p.offset as number) || 0,
|
||||
expansion: expand,
|
||||
expandFn: expand ? expandQuery : undefined,
|
||||
// T4/D5 — per-call mode (local/trusted only; remote ignored).
|
||||
...((): { mode?: string } => { const m = resolvePerCallMode(ctx, p.mode); return m ? { mode: m } : {}; })(),
|
||||
detail,
|
||||
language: (p.lang as string) || undefined,
|
||||
symbolKind: (p.symbol_kind as string) || undefined,
|
||||
|
||||
+70
-23
@@ -47,7 +47,7 @@ import { normalizeWeightForStorage } from './takes-fence.ts';
|
||||
import { GBrainError, PAGE_SORT_SQL } from './types.ts';
|
||||
import { computeAnomaliesFromBuckets } from './cycle/anomaly.ts';
|
||||
import { resolveBoostMap, resolveHardExcludes } from './search/source-boost.ts';
|
||||
import { buildSourceFactorCase, buildHardExcludeClause, buildVisibilityClause, buildRecencyComponentSql } from './search/sql-ranking.ts';
|
||||
import { buildSourceFactorCase, buildHardExcludeClause, buildVisibilityClause, buildRecencyComponentSql, buildBestPerPagePoolCte } from './search/sql-ranking.ts';
|
||||
import {
|
||||
normalizeEngineColumn,
|
||||
buildVectorCastFragment,
|
||||
@@ -1497,13 +1497,9 @@ export class PGLiteEngine implements BrainEngine {
|
||||
ORDER BY score DESC
|
||||
LIMIT $2
|
||||
),
|
||||
best_per_page AS (
|
||||
SELECT DISTINCT ON (slug) *
|
||||
FROM ranked
|
||||
ORDER BY slug, score DESC
|
||||
)
|
||||
${buildBestPerPagePoolCte('ranked')}
|
||||
SELECT * FROM best_per_page
|
||||
ORDER BY score DESC
|
||||
ORDER BY score DESC, page_id ASC, chunk_id ASC
|
||||
LIMIT $3 OFFSET $4`,
|
||||
params
|
||||
);
|
||||
@@ -1612,13 +1608,9 @@ export class PGLiteEngine implements BrainEngine {
|
||||
ORDER BY score DESC
|
||||
LIMIT $3
|
||||
),
|
||||
best_per_page AS (
|
||||
SELECT DISTINCT ON (slug) *
|
||||
FROM ranked
|
||||
ORDER BY slug, score DESC
|
||||
)
|
||||
${buildBestPerPagePoolCte('ranked')}
|
||||
SELECT * FROM best_per_page
|
||||
ORDER BY score DESC
|
||||
ORDER BY score DESC, page_id ASC, chunk_id ASC
|
||||
LIMIT $4 OFFSET $5`,
|
||||
params,
|
||||
);
|
||||
@@ -1759,7 +1751,10 @@ export class PGLiteEngine implements BrainEngine {
|
||||
// subquery's WHERE would lexically resolve back to `te.page_id` itself
|
||||
// and degrade to `te.page_id = te.page_id` (always true), making every
|
||||
// result stale=true. Codex caught this in adversarial review.
|
||||
const sourceFactorCaseOnSlug = buildSourceFactorCase('hc.slug', boostMap, opts?.detail);
|
||||
// Built on the bare `slug` output column: applied inside the `scored` CTE
|
||||
// whose FROM is the single relation `hnsw_candidates`, so unqualified
|
||||
// `slug` resolves cleanly (T1 per-page pool restructure).
|
||||
const sourceFactorCaseOnSlug = buildSourceFactorCase('slug', boostMap, opts?.detail);
|
||||
const hardExcludePrefixes = resolveHardExcludes(opts?.exclude_slug_prefixes, opts?.include_slug_prefixes);
|
||||
const hardExcludeClause = buildHardExcludeClause('p.slug', hardExcludePrefixes);
|
||||
const innerLimit = offset + Math.max(limit * 5, 100);
|
||||
@@ -1838,23 +1833,33 @@ export class PGLiteEngine implements BrainEngine {
|
||||
WHERE cc.${col} IS NOT NULL ${modalityFilter} ${detailFilter}${extraFilter} ${hardExcludeClause} ${visibilityClause}
|
||||
ORDER BY cc.${col} <=> ${castSql}
|
||||
LIMIT $2
|
||||
)
|
||||
),
|
||||
-- score as a select-list expr; inner ORDER BY stays pure-distance so
|
||||
-- the HNSW index is usable.
|
||||
scored AS (
|
||||
SELECT *, raw_score * ${sourceFactorCaseOnSlug} AS score
|
||||
FROM hnsw_candidates
|
||||
),
|
||||
-- T1 (retrieval-maxpool incident): collapse to the best chunk PER PAGE
|
||||
-- over the full candidate set before the user LIMIT. Shared builder with
|
||||
-- postgres-engine + the keyword path so they cannot drift.
|
||||
${buildBestPerPagePoolCte('scored')}
|
||||
SELECT
|
||||
hc.slug, hc.page_id, hc.title, hc.type, hc.source_id,
|
||||
hc.effective_date, hc.effective_date_source,
|
||||
hc.chunk_id, hc.chunk_index, hc.chunk_text, hc.chunk_source,
|
||||
hc.raw_score * ${sourceFactorCaseOnSlug} AS score,
|
||||
CASE WHEN hc.updated_at < (
|
||||
SELECT MAX(te.created_at) FROM timeline_entries te WHERE te.page_id = hc.page_id
|
||||
bpp.slug, bpp.page_id, bpp.title, bpp.type, bpp.source_id,
|
||||
bpp.effective_date, bpp.effective_date_source,
|
||||
bpp.chunk_id, bpp.chunk_index, bpp.chunk_text, bpp.chunk_source,
|
||||
bpp.score,
|
||||
CASE WHEN bpp.updated_at < (
|
||||
SELECT MAX(te.created_at) FROM timeline_entries te WHERE te.page_id = bpp.page_id
|
||||
) THEN true ELSE false END AS stale
|
||||
FROM hnsw_candidates hc
|
||||
FROM best_per_page bpp
|
||||
-- v0.41.13: stable tiebreaker. When two chunks share a score (same
|
||||
-- source-prefix boost + same cosine distance, the basis-vector + same-
|
||||
-- source-prefix case in eval fixtures), older page_id wins. Without
|
||||
-- this, planner choice + index presence can flip ordering between
|
||||
-- master and feature branches that add unrelated indexes — see the
|
||||
-- pages_dedup_idx (v95) regression that motivated this.
|
||||
ORDER BY score DESC, hc.page_id ASC, hc.chunk_id ASC
|
||||
ORDER BY bpp.score DESC, bpp.page_id ASC, bpp.chunk_id ASC
|
||||
LIMIT $3
|
||||
OFFSET $4`,
|
||||
params
|
||||
@@ -4515,6 +4520,48 @@ export class PGLiteEngine implements BrainEngine {
|
||||
}
|
||||
}
|
||||
|
||||
async resolveAliases(
|
||||
aliasNorms: string[],
|
||||
opts?: { sourceId?: string; sourceIds?: string[] },
|
||||
): Promise<Map<string, Array<{ slug: string; source_id: string }>>> {
|
||||
const out = new Map<string, Array<{ slug: string; source_id: string }>>();
|
||||
if (!aliasNorms || aliasNorms.length === 0) return out;
|
||||
const sources =
|
||||
opts?.sourceIds && opts.sourceIds.length > 0
|
||||
? opts.sourceIds
|
||||
: opts?.sourceId
|
||||
? [opts.sourceId]
|
||||
: null;
|
||||
let q = `SELECT alias_norm, slug, source_id FROM page_aliases WHERE alias_norm = ANY($1::text[])`;
|
||||
const params: unknown[] = [aliasNorms];
|
||||
if (sources) {
|
||||
params.push(sources);
|
||||
q += ` AND source_id = ANY($2::text[])`;
|
||||
}
|
||||
q += ` ORDER BY alias_norm, source_id, slug`;
|
||||
const { rows } = await this.db.query(q, params);
|
||||
for (const r of rows as Array<{ alias_norm: string; slug: string; source_id: string }>) {
|
||||
const list = out.get(r.alias_norm) ?? [];
|
||||
if (!list.some(x => x.slug === r.slug && x.source_id === r.source_id)) {
|
||||
list.push({ slug: r.slug, source_id: r.source_id });
|
||||
}
|
||||
out.set(r.alias_norm, list);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
async setPageAliases(slug: string, sourceId: string, aliasNorms: string[]): Promise<void> {
|
||||
const uniq = Array.from(new Set(aliasNorms.filter(a => a.length > 0)));
|
||||
await this.db.query(`DELETE FROM page_aliases WHERE source_id = $1 AND slug = $2`, [sourceId, slug]);
|
||||
if (uniq.length === 0) return;
|
||||
await this.db.query(
|
||||
`INSERT INTO page_aliases (source_id, alias_norm, slug)
|
||||
SELECT $1, a, $2 FROM unnest($3::text[]) AS a
|
||||
ON CONFLICT (source_id, alias_norm, slug) DO NOTHING`,
|
||||
[sourceId, slug, uniq],
|
||||
);
|
||||
}
|
||||
|
||||
// Config
|
||||
async getConfig(key: string): Promise<string | null> {
|
||||
const { rows } = await this.db.query('SELECT value FROM config WHERE key = $1', [key]);
|
||||
|
||||
@@ -999,6 +999,26 @@ CREATE TABLE IF NOT EXISTS slug_aliases (
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS slug_aliases_canonical_idx
|
||||
ON slug_aliases (source_id, canonical_slug);
|
||||
|
||||
-- T3 retrieval-cathedral (retrieval-maxpool incident): free-text alias
|
||||
-- resolution for SEARCH. Distinct from slug_aliases (slug->slug wikilink
|
||||
-- redirect): page_aliases maps a normalized free-text name ("hall of light",
|
||||
-- "明堂") to a canonical slug so a query that is a chosen name surfaces the
|
||||
-- page. alias_norm is normalizeAlias() output; the (source_id, alias_norm,
|
||||
-- slug) triple is unique so re-ingest is idempotent without blocking a second
|
||||
-- page claiming the same alias (collisions reported + resolved at query time).
|
||||
CREATE TABLE IF NOT EXISTS page_aliases (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
source_id TEXT NOT NULL,
|
||||
alias_norm TEXT NOT NULL,
|
||||
slug TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT page_aliases_uniq UNIQUE (source_id, alias_norm, slug)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS page_aliases_lookup_idx
|
||||
ON page_aliases (source_id, alias_norm);
|
||||
CREATE INDEX IF NOT EXISTS page_aliases_slug_idx
|
||||
ON page_aliases (source_id, slug);
|
||||
`;
|
||||
|
||||
/**
|
||||
|
||||
@@ -55,7 +55,7 @@ import { ConnectionManager } from './connection-manager.ts';
|
||||
import { logConnectionEvent } from './connection-audit.ts';
|
||||
import { validateSlug, contentHash, rowToPage, rowToChunk, rowToSearchResult, parseEmbedding, tryParseEmbedding, takeRowToTake, isUndefinedTableError, warnOncePerProcess } from './utils.ts';
|
||||
import { resolveBoostMap, resolveHardExcludes } from './search/source-boost.ts';
|
||||
import { buildSourceFactorCase, buildHardExcludeClause, buildVisibilityClause, buildRecencyComponentSql } from './search/sql-ranking.ts';
|
||||
import { buildSourceFactorCase, buildHardExcludeClause, buildVisibilityClause, buildRecencyComponentSql, buildBestPerPagePoolCte } from './search/sql-ranking.ts';
|
||||
import { DEFAULT_EMBEDDING_MODEL, DEFAULT_EMBEDDING_DIMENSIONS } from './ai/defaults.ts';
|
||||
import { DELETE_BATCH_SIZE } from './engine-constants.ts';
|
||||
|
||||
@@ -1551,11 +1551,7 @@ export class PostgresEngine implements BrainEngine {
|
||||
ORDER BY score DESC
|
||||
LIMIT ${innerLimitParam}
|
||||
),
|
||||
best_per_page AS (
|
||||
SELECT DISTINCT ON (slug) *
|
||||
FROM ranked_chunks
|
||||
ORDER BY slug, score DESC
|
||||
)
|
||||
${buildBestPerPagePoolCte('ranked_chunks')}
|
||||
SELECT slug, page_id, title, type, source_id,
|
||||
effective_date, effective_date_source,
|
||||
chunk_id, chunk_index, chunk_text, chunk_source, score,
|
||||
@@ -1838,14 +1834,25 @@ export class PostgresEngine implements BrainEngine {
|
||||
${visibilityClause}
|
||||
ORDER BY cc.${col} <=> ${castSql}
|
||||
LIMIT ${innerLimitParam}
|
||||
)
|
||||
),
|
||||
-- score computed as a select-list expr (NOT in the inner ORDER BY, which
|
||||
-- must stay pure-distance so the HNSW index is usable).
|
||||
scored AS (
|
||||
SELECT *, raw_score * ${sourceFactorCaseOnSlug} AS score
|
||||
FROM hnsw_candidates
|
||||
),
|
||||
-- T1 (retrieval-maxpool incident): collapse to the best chunk PER PAGE
|
||||
-- over the full candidate set before the user LIMIT, so a page's strong
|
||||
-- chunk can't be crowded out of the result by weaker chunks of other
|
||||
-- pages. Shared builder keeps keyword + vector × postgres + pglite in lockstep.
|
||||
${buildBestPerPagePoolCte('scored')}
|
||||
SELECT
|
||||
slug, page_id, title, type, source_id,
|
||||
effective_date, effective_date_source,
|
||||
chunk_id, chunk_index, chunk_text, chunk_source,
|
||||
raw_score * ${sourceFactorCaseOnSlug} AS score,
|
||||
score,
|
||||
false AS stale
|
||||
FROM hnsw_candidates
|
||||
FROM best_per_page
|
||||
-- v0.41.13: stable tiebreaker for tied scores. See pglite-engine for
|
||||
-- rationale (basis-vector test fixtures, planner-dependent ordering).
|
||||
ORDER BY score DESC, page_id ASC, chunk_id ASC
|
||||
@@ -4534,6 +4541,54 @@ export class PostgresEngine implements BrainEngine {
|
||||
}
|
||||
}
|
||||
|
||||
async resolveAliases(
|
||||
aliasNorms: string[],
|
||||
opts?: { sourceId?: string; sourceIds?: string[] },
|
||||
): Promise<Map<string, Array<{ slug: string; source_id: string }>>> {
|
||||
const out = new Map<string, Array<{ slug: string; source_id: string }>>();
|
||||
if (!aliasNorms || aliasNorms.length === 0) return out;
|
||||
const sql = this.sql;
|
||||
const sources =
|
||||
opts?.sourceIds && opts.sourceIds.length > 0
|
||||
? opts.sourceIds
|
||||
: opts?.sourceId
|
||||
? [opts.sourceId]
|
||||
: null;
|
||||
const rows = sources
|
||||
? await sql`
|
||||
SELECT alias_norm, slug, source_id
|
||||
FROM page_aliases
|
||||
WHERE alias_norm = ANY(${aliasNorms}::text[])
|
||||
AND source_id = ANY(${sources}::text[])
|
||||
ORDER BY alias_norm, source_id, slug`
|
||||
: await sql`
|
||||
SELECT alias_norm, slug, source_id
|
||||
FROM page_aliases
|
||||
WHERE alias_norm = ANY(${aliasNorms}::text[])
|
||||
ORDER BY alias_norm, source_id, slug`;
|
||||
for (const r of rows) {
|
||||
const a = r.alias_norm as string;
|
||||
const list = out.get(a) ?? [];
|
||||
const ref = { slug: r.slug as string, source_id: r.source_id as string };
|
||||
if (!list.some(x => x.slug === ref.slug && x.source_id === ref.source_id)) list.push(ref);
|
||||
out.set(a, list);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
async setPageAliases(slug: string, sourceId: string, aliasNorms: string[]): Promise<void> {
|
||||
const sql = this.sql;
|
||||
const uniq = Array.from(new Set(aliasNorms.filter(a => a.length > 0)));
|
||||
await sql.begin(async tx => {
|
||||
await tx`DELETE FROM page_aliases WHERE source_id = ${sourceId} AND slug = ${slug}`;
|
||||
if (uniq.length === 0) return;
|
||||
await tx`
|
||||
INSERT INTO page_aliases (source_id, alias_norm, slug)
|
||||
SELECT ${sourceId}, a, ${slug} FROM unnest(${uniq}::text[]) AS a
|
||||
ON CONFLICT (source_id, alias_norm, slug) DO NOTHING`;
|
||||
});
|
||||
}
|
||||
|
||||
// Config
|
||||
async getConfig(key: string): Promise<string | null> {
|
||||
const sql = this.sql;
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
/**
|
||||
* T3 — alias normalization (retrieval-maxpool incident, alias layer).
|
||||
*
|
||||
* ONE normalizer shared by the WRITE path (ingest projects frontmatter
|
||||
* `aliases:` into page_aliases) and the READ path (search matches the query
|
||||
* against page_aliases). If the two sides normalized differently — one
|
||||
* lowercases, the other also collapses whitespace — stored aliases would
|
||||
* silently never match queries, and there'd be no error to notice. Same
|
||||
* single-source-of-truth posture as cjk.ts / escapeLikePattern.
|
||||
*
|
||||
* Normalization (deliberately aggressive + deterministic):
|
||||
* - Unicode NFKC (so 明堂 and full/half-width variants converge)
|
||||
* - lowercase
|
||||
* - strip leading/trailing whitespace
|
||||
* - collapse internal whitespace runs to a single space
|
||||
* - drop surrounding quotes/brackets the YAML parser may leave
|
||||
*
|
||||
* Returns '' for input that normalizes to empty — callers MUST skip empty
|
||||
* aliases (an empty alias would match empty/whitespace queries).
|
||||
*/
|
||||
|
||||
export function normalizeAlias(raw: string): string {
|
||||
if (typeof raw !== 'string') return '';
|
||||
return raw
|
||||
.normalize('NFKC')
|
||||
.toLowerCase()
|
||||
.replace(/[\s ]+/g, ' ')
|
||||
.trim()
|
||||
// strip a single layer of wrapping quotes/brackets left by loose YAML
|
||||
.replace(/^["'`\[(]+/, '')
|
||||
.replace(/["'`\])]+$/, '')
|
||||
.trim();
|
||||
}
|
||||
|
||||
/**
|
||||
* Coerce a frontmatter `aliases:` value (which may be a scalar string, an
|
||||
* array, or absent/garbage) into a deduped list of normalized, non-empty
|
||||
* aliases. Used by the ingest projection AND the backfill walker so both
|
||||
* derive the same alias set from the same JSONB.
|
||||
*/
|
||||
export function normalizeAliasList(value: unknown): string[] {
|
||||
const out = new Set<string>();
|
||||
const push = (v: unknown) => {
|
||||
if (typeof v !== 'string') return;
|
||||
const n = normalizeAlias(v);
|
||||
if (n.length > 0) out.add(n);
|
||||
};
|
||||
if (Array.isArray(value)) {
|
||||
for (const v of value) push(v);
|
||||
} else if (typeof value === 'string') {
|
||||
// A scalar `aliases: Hall of Light` OR a comma-list `aliases: a, b`.
|
||||
if (value.includes(',')) value.split(',').forEach(push);
|
||||
else push(value);
|
||||
}
|
||||
return Array.from(out);
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
/**
|
||||
* T4 — evidence + create_safety contract (retrieval-maxpool incident).
|
||||
*
|
||||
* The incident's ROOT behavior: the agent read a single blended score (0.64)
|
||||
* and decided "no strong match, safe to write a new page" — then wrote a
|
||||
* duplicate on top of a fully-developed concept page. A blended RRF/cosine
|
||||
* score is not a calibrated probability (Codex#5), so keying the
|
||||
* don't-duplicate decision off a raw threshold is fragile.
|
||||
*
|
||||
* The fix (Codex#6): tell the agent WHY a result matched, not just a number.
|
||||
* `evidence` names the strongest signal that surfaced the page; `create_safety`
|
||||
* is the derived "is this page already here?" hint the agent keys off instead
|
||||
* of a raw score. Pure + deterministic; stamped on every result so MCP callers
|
||||
* and `--explain` read the same contract.
|
||||
*
|
||||
* Evidence precedence (strongest signal wins):
|
||||
* alias_hit — query exactly matched the page's declared chosen name
|
||||
* exact_title_match — query is a phrase in the page title (title boost fired)
|
||||
* high_vector_match — base (pre-boost) score >= HIGH_MATCH_FLOOR
|
||||
* keyword_exact — surfaced with a solid score but no title/alias/vector tag
|
||||
* weak_semantic — everything else (low-confidence tail)
|
||||
*
|
||||
* create_safety:
|
||||
* exists — strong evidence this IS the page; do NOT create a duplicate
|
||||
* probable — likely the page; prefer updating over creating
|
||||
* unknown — weak signal; the agent should look closer before deciding
|
||||
*/
|
||||
|
||||
import type { SearchResult } from '../types.ts';
|
||||
|
||||
export type Evidence =
|
||||
| 'alias_hit'
|
||||
| 'exact_title_match'
|
||||
| 'high_vector_match'
|
||||
| 'keyword_exact'
|
||||
| 'weak_semantic';
|
||||
|
||||
export type CreateSafety = 'exists' | 'probable' | 'unknown';
|
||||
|
||||
/** base_score (pre-boost) at/above this is a confident vector/keyword match. */
|
||||
export const HIGH_MATCH_FLOOR = 0.85;
|
||||
/** base_score at/above this is a solid (not weak) match. */
|
||||
export const SOLID_MATCH_FLOOR = 0.6;
|
||||
|
||||
export function classifyEvidence(r: SearchResult): Evidence {
|
||||
if (r.alias_hit) return 'alias_hit';
|
||||
if (r.title_match_boost && r.title_match_boost > 1.0) return 'exact_title_match';
|
||||
const base = typeof r.base_score === 'number' ? r.base_score : r.score;
|
||||
if (Number.isFinite(base) && base >= HIGH_MATCH_FLOOR) return 'high_vector_match';
|
||||
if (Number.isFinite(base) && base >= SOLID_MATCH_FLOOR) return 'keyword_exact';
|
||||
return 'weak_semantic';
|
||||
}
|
||||
|
||||
export function createSafetyFor(evidence: Evidence): CreateSafety {
|
||||
switch (evidence) {
|
||||
case 'alias_hit':
|
||||
case 'exact_title_match':
|
||||
case 'high_vector_match':
|
||||
return 'exists';
|
||||
case 'keyword_exact':
|
||||
return 'probable';
|
||||
case 'weak_semantic':
|
||||
return 'unknown';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Stamp `evidence` + `create_safety` on every result in place. Called once at
|
||||
* the end of the hybrid pipeline (after the alias hop, before slice) so the
|
||||
* agent-facing result carries the contract. Idempotent.
|
||||
*/
|
||||
export function stampEvidence(results: SearchResult[]): void {
|
||||
for (const r of results) {
|
||||
const e = classifyEvidence(r);
|
||||
r.evidence = e;
|
||||
r.create_safety = createSafetyFor(e);
|
||||
}
|
||||
}
|
||||
+210
-7
@@ -25,6 +25,9 @@ import { loadConfigWithEngine } from '../config.ts';
|
||||
import { dedupResults } from './dedup.ts';
|
||||
import { applyReranker } from './rerank.ts';
|
||||
import { autoDetectDetail, classifyQuery, isAmbiguousModalityQuery } from './query-intent.ts';
|
||||
import { isTitlePhraseMatch } from './title-match.ts';
|
||||
import { normalizeAlias } from './alias-normalize.ts';
|
||||
import { stampEvidence } from './evidence.ts';
|
||||
import { expandAnchors, hydrateChunks } from './two-pass.ts';
|
||||
import { enforceTokenBudget } from './token-budget.ts';
|
||||
import { recordSearchTelemetry } from './telemetry.ts';
|
||||
@@ -228,6 +231,43 @@ export function applyRecencyBoost(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* T2 (retrieval-maxpool incident) — apply the title-phrase boost.
|
||||
*
|
||||
* Fires when the normalized query is a contiguous token-run inside a result's
|
||||
* page title (or an exact full-title match), per `isTitlePhraseMatch`. Mutate-
|
||||
* in-place; caller re-sorts. Mirrors applyBacklinkBoost's floor-gate + stamp.
|
||||
*
|
||||
* Bounded by construction: a single fixed multiplier (`factor`, default 1.25),
|
||||
* floor-ratio-gated so a title hit on a weak-overlap page can't leapfrog a
|
||||
* strong primary hit. `base_score` (stamped at runPostFusionStages entry) is
|
||||
* NOT touched, so the agent's dedup gate still reads true match confidence.
|
||||
*
|
||||
* Why page.title and not "first compiled_truth chunk" (Codex#11): the title is
|
||||
* a stable column; the first chunk is a chunking accident that import changes
|
||||
* could shift. The signal is "the query is the name of this thing."
|
||||
*/
|
||||
export function applyTitleBoost(
|
||||
results: SearchResult[],
|
||||
query: string,
|
||||
factor: number,
|
||||
floorThreshold?: number,
|
||||
): void {
|
||||
if (!query || !Number.isFinite(factor) || factor <= 1.0) return;
|
||||
for (const r of results) {
|
||||
if (!Number.isFinite(r.score)) continue;
|
||||
if (floorThreshold !== undefined && r.score < floorThreshold) continue;
|
||||
if (!r.title) continue;
|
||||
if (isTitlePhraseMatch(query, r.title)) {
|
||||
r.score *= factor;
|
||||
r.title_match_boost = factor; // attribution stamp (v0.40.4 convention)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Default title-phrase boost multiplier (mode-overridable via `title_boost`). */
|
||||
export const DEFAULT_TITLE_BOOST = 1.25;
|
||||
|
||||
/**
|
||||
* v0.29.1 — runPostFusionStages: wrap backlink + salience + recency in a
|
||||
* single stage that fires from EVERY hybridSearch return path (codex
|
||||
@@ -284,6 +324,17 @@ export interface PostFusionOpts {
|
||||
* wave via search-stats.
|
||||
*/
|
||||
onScoreDistribution?: (dist: import('./graph-signals.ts').ScoreDistribution) => void;
|
||||
/**
|
||||
* T2 — the raw query string, needed by the title-phrase boost stage.
|
||||
* Undefined disables the stage (e.g. image-only queries).
|
||||
*/
|
||||
query?: string;
|
||||
/**
|
||||
* T2 — title-phrase boost multiplier (mode-resolved from `title_boost`).
|
||||
* <= 1.0 or undefined disables the stage. Floor-ratio-gated like the
|
||||
* metadata stages so a title hit can't bury a strong semantic match.
|
||||
*/
|
||||
titleBoost?: number;
|
||||
}
|
||||
|
||||
export async function runPostFusionStages(
|
||||
@@ -357,6 +408,18 @@ export async function runPostFusionStages(
|
||||
}
|
||||
}
|
||||
|
||||
// T2 — title-phrase boost. Runs after the metadata stages, before graph
|
||||
// signals. Shares the single floor-threshold so a title hit on a weak page
|
||||
// can't leapfrog a strong primary hit (Codex#10). Fail-soft: pure + in-memory,
|
||||
// but guarded so a bad query/title can't throw the whole pipeline.
|
||||
if (opts.query && opts.titleBoost && opts.titleBoost > 1.0) {
|
||||
try {
|
||||
applyTitleBoost(results, opts.query, opts.titleBoost, floorThreshold);
|
||||
} catch {
|
||||
// Non-fatal; preserves the per-stage contract.
|
||||
}
|
||||
}
|
||||
|
||||
// v0.40.4 — graph-signals stage (4th post-fusion stage). Runs AFTER
|
||||
// backlink/salience/recency so it stacks on top of metadata boosts;
|
||||
// shares the same floor-threshold so a weak hub gets the same
|
||||
@@ -447,8 +510,103 @@ async function applyAliasResolvedBoost(
|
||||
}
|
||||
}
|
||||
|
||||
// T3 — free-text alias hop tuning.
|
||||
const ALIAS_HOP_PRESENT_BOOST = 1.10; // bounded boost when canonical already in results
|
||||
const MAX_ALIAS_QUERY_TOKENS = 6; // skip long queries (clearly not a chosen name)
|
||||
const MAX_ALIAS_INJECT = 3; // cap injected pages per query (collision safety)
|
||||
|
||||
/**
|
||||
* T3 — free-text alias hop (retrieval-maxpool incident, the named-thing fix).
|
||||
*
|
||||
* When the normalized query EXACTLY matches a page's declared alias
|
||||
* ("Hall of Light" / "明堂" -> the Mingtang page), make sure that page is in
|
||||
* the result set: boost it if already present, inject it at top-of-organic +
|
||||
* epsilon if absent. This is the only layer that bridges true synonyms with
|
||||
* zero surface overlap — neither max-pool nor title-boost can.
|
||||
*
|
||||
* Precision guards (Codex#7/#10):
|
||||
* - FULL normalized-query exact match only (not substring / not n-grams) —
|
||||
* "light" won't fire unless the whole query normalizes to a stored alias.
|
||||
* - skip queries longer than MAX_ALIAS_QUERY_TOKENS (clearly prose, not a name).
|
||||
* - bounded: present-boost is 1.10x; inject score is top-of-organic + ε,
|
||||
* never an absolute 1.0 (D3 — aliases are not a ranking sledgehammer).
|
||||
* - collisions (two pages claim one alias): deterministic alpha order, capped.
|
||||
*
|
||||
* Fail-open: pre-v110 brains (no page_aliases table) and any lookup error
|
||||
* degrade to the input unchanged (D9). Returns a NEW array; caller re-slices.
|
||||
*/
|
||||
export async function applyAliasHop(
|
||||
engine: import('../engine.ts').BrainEngine,
|
||||
results: SearchResult[],
|
||||
query: string,
|
||||
opts: { sourceId?: string; sourceIds?: string[] },
|
||||
): Promise<SearchResult[]> {
|
||||
if (!query) return results;
|
||||
const qNorm = normalizeAlias(query);
|
||||
if (!qNorm || qNorm.split(' ').length > MAX_ALIAS_QUERY_TOKENS) return results;
|
||||
|
||||
let aliasMap: Map<string, Array<{ slug: string; source_id: string }>>;
|
||||
try {
|
||||
aliasMap = await engine.resolveAliases([qNorm], { sourceId: opts.sourceId, sourceIds: opts.sourceIds });
|
||||
} catch {
|
||||
return results; // pre-v110 table-missing OR transient error -> fail-open
|
||||
}
|
||||
const refs = aliasMap.get(qNorm);
|
||||
if (!refs || refs.length === 0) return results;
|
||||
|
||||
// Deterministic + capped. Source-scoped: each canonical is a (source_id, slug)
|
||||
// pair so a federated caller boosts/injects the RIGHT source's page, never
|
||||
// collapsing or cross-injecting (P0 source-isolation contract).
|
||||
const ordered = [...refs]
|
||||
.sort((a, b) => (a.source_id === b.source_id ? a.slug.localeCompare(b.slug) : a.source_id.localeCompare(b.source_id)))
|
||||
.slice(0, MAX_ALIAS_INJECT);
|
||||
const out = [...results];
|
||||
const topScore = out.reduce((m, r) => (Number.isFinite(r.score) && r.score > m ? r.score : m), 0);
|
||||
let injectScore = topScore > 0 ? topScore : 1.0;
|
||||
|
||||
for (const ref of ordered) {
|
||||
const idx = out.findIndex(r => r.slug === ref.slug && (r.source_id ?? 'default') === ref.source_id);
|
||||
if (idx >= 0) {
|
||||
if (Number.isFinite(out[idx].score)) out[idx].score *= ALIAS_HOP_PRESENT_BOOST;
|
||||
out[idx].alias_hit = true;
|
||||
continue;
|
||||
}
|
||||
// Absent canonical: fetch (in its OWN source) + inject at top-of-organic + epsilon.
|
||||
let page;
|
||||
try {
|
||||
page = await engine.getPage(ref.slug, { sourceId: ref.source_id });
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (!page) continue;
|
||||
injectScore += 1e-6;
|
||||
out.push({
|
||||
slug: page.slug,
|
||||
title: page.title,
|
||||
type: page.type,
|
||||
source_id: page.source_id ?? ref.source_id,
|
||||
chunk_text: (page.compiled_truth ?? '').slice(0, 200),
|
||||
chunk_index: 0,
|
||||
chunk_id: 0,
|
||||
score: injectScore,
|
||||
base_score: injectScore,
|
||||
alias_hit: true,
|
||||
} as SearchResult);
|
||||
}
|
||||
out.sort((a, b) => b.score - a.score);
|
||||
return out;
|
||||
}
|
||||
|
||||
export interface HybridSearchOpts extends SearchOpts {
|
||||
expansion?: boolean;
|
||||
/**
|
||||
* T4/D5 — per-call search-mode selector (one of SEARCH_MODES). Selects the
|
||||
* whole mode bundle for this call, overriding the server-configured mode.
|
||||
* The op layer passes this ONLY for trusted/local callers (ctx.remote ===
|
||||
* false); remote callers leave it undefined so they can't escalate to the
|
||||
* costly tokenmax bundle. Unknown values fall back to the default bundle.
|
||||
*/
|
||||
mode?: string;
|
||||
expandFn?: (query: string) => Promise<string[]>;
|
||||
/** Override default RRF K constant (default: 60). Lower values boost top-ranked results more. */
|
||||
rrfK?: number;
|
||||
@@ -486,7 +644,11 @@ export async function hybridSearch(
|
||||
const { loadSearchModeConfig, resolveSearchMode } = await import('./mode.ts');
|
||||
const modeInput = await loadSearchModeConfig(engine);
|
||||
const resolvedMode = resolveSearchMode({
|
||||
mode: modeInput.mode,
|
||||
// T4/D5 — per-call mode selector (e.g. `--mode tokenmax`). The op layer
|
||||
// only passes this for trusted/local callers; remote callers leave it
|
||||
// undefined and fall through to the server-configured mode (no cost
|
||||
// escalation). Unknown values fall back to the default in resolveSearchMode.
|
||||
mode: opts?.mode ?? modeInput.mode,
|
||||
overrides: modeInput.overrides,
|
||||
perCall: {
|
||||
intentWeighting: opts?.intentWeighting,
|
||||
@@ -581,6 +743,9 @@ export async function hybridSearch(
|
||||
// flush is fire-and-forget on 60s / 100-call thresholds. The hot path
|
||||
// never waits.
|
||||
let lastResultsCount = 0;
|
||||
// T7 — rank-1 base_score for the telemetry drift signal. Set alongside
|
||||
// lastResultsCount at each return path; undefined when there are no results.
|
||||
let lastRank1Score: number | undefined;
|
||||
const emitMeta = (meta: HybridSearchMeta): void => {
|
||||
try {
|
||||
opts?.onMeta?.(meta);
|
||||
@@ -588,7 +753,7 @@ export async function hybridSearch(
|
||||
// swallow — capture telemetry is best-effort
|
||||
}
|
||||
try {
|
||||
recordSearchTelemetry(engine, meta, { results_count: lastResultsCount });
|
||||
recordSearchTelemetry(engine, meta, { results_count: lastResultsCount, rank1_score: lastRank1Score });
|
||||
} catch {
|
||||
// swallow — telemetry must never break the search hot path.
|
||||
}
|
||||
@@ -658,6 +823,10 @@ export async function hybridSearch(
|
||||
// Without this thread, the entire graph-signals wave is dead code —
|
||||
// codex outside-voice caught the missing wire pre-merge.
|
||||
graphSignalsEnabled: resolvedMode.graph_signals,
|
||||
// T2 — title-phrase boost threaded from resolved mode (`title_boost`).
|
||||
// The raw query drives the matcher; default factor when the knob is unset.
|
||||
query,
|
||||
titleBoost: resolvedMode.title_boost,
|
||||
};
|
||||
|
||||
// Skip vector search entirely if the gateway has no embedding provider configured (Codex C3).
|
||||
@@ -672,10 +841,18 @@ export async function hybridSearch(
|
||||
await runPostFusionStages(engine, keywordResults, postFusionOpts);
|
||||
keywordResults.sort((a, b) => b.score - a.score);
|
||||
}
|
||||
const noEmbedSliced = dedupResults(keywordResults).slice(offset, offset + limit);
|
||||
// T3/T4 — alias hop + evidence stamp even without an embedding provider
|
||||
// (the named-thing fix is most valuable exactly when vector is unavailable).
|
||||
const noEmbedHopped = await applyAliasHop(engine, dedupResults(keywordResults), query, {
|
||||
sourceId: opts?.sourceId,
|
||||
sourceIds: opts?.sourceIds,
|
||||
});
|
||||
stampEvidence(noEmbedHopped);
|
||||
const noEmbedSliced = noEmbedHopped.slice(offset, offset + limit);
|
||||
// v0.32.3 search-lite: budget enforcement on the no-embedding-provider path.
|
||||
const { results: noEmbedBudgeted, meta: noEmbedBudgetMeta } = enforceTokenBudget(noEmbedSliced, resolvedMode.tokenBudget);
|
||||
lastResultsCount = noEmbedBudgeted.length;
|
||||
lastRank1Score = noEmbedBudgeted[0] ? (noEmbedBudgeted[0].base_score ?? noEmbedBudgeted[0].score) : undefined;
|
||||
emitMeta({
|
||||
vector_enabled: false,
|
||||
detail_resolved: detailResolved,
|
||||
@@ -882,10 +1059,16 @@ export async function hybridSearch(
|
||||
await runPostFusionStages(engine, keywordResults, postFusionOpts);
|
||||
keywordResults.sort((a, b) => b.score - a.score);
|
||||
}
|
||||
const kwSliced = dedupResults(keywordResults).slice(offset, offset + limit);
|
||||
const kwHopped = await applyAliasHop(engine, dedupResults(keywordResults), query, {
|
||||
sourceId: opts?.sourceId,
|
||||
sourceIds: opts?.sourceIds,
|
||||
});
|
||||
stampEvidence(kwHopped);
|
||||
const kwSliced = kwHopped.slice(offset, offset + limit);
|
||||
// v0.32.3 search-lite: budget enforcement on the keyword-fallback path too.
|
||||
const { results: kwBudgeted, meta: kwBudgetMeta } = enforceTokenBudget(kwSliced, resolvedMode.tokenBudget);
|
||||
lastResultsCount = kwBudgeted.length;
|
||||
lastRank1Score = kwBudgeted[0] ? (kwBudgeted[0].base_score ?? kwBudgeted[0].score) : undefined;
|
||||
emitMeta({
|
||||
vector_enabled: false,
|
||||
detail_resolved: detailResolved,
|
||||
@@ -1035,18 +1218,34 @@ export async function hybridSearch(
|
||||
? await applyReranker(query, deduped, rerankerOpts as any)
|
||||
: deduped;
|
||||
|
||||
// T3 — free-text alias hop. Runs AFTER rerank so a query that is a page's
|
||||
// declared chosen name reliably surfaces that page regardless of how the
|
||||
// reranker scored body chunks. Fail-open on pre-v110 brains.
|
||||
const aliasHopped = await applyAliasHop(engine, reranked, query, {
|
||||
sourceId: opts?.sourceId,
|
||||
sourceIds: opts?.sourceIds,
|
||||
});
|
||||
|
||||
// T4 — stamp evidence + create_safety so the agent's don't-duplicate
|
||||
// decision keys off WHY a page matched, not a raw blended score. Stamp on
|
||||
// the full alias-hopped set before any adaptive trim so the kept results
|
||||
// carry evidence regardless of where the cap lands.
|
||||
stampEvidence(aliasHopped);
|
||||
|
||||
// v0.42 — intent-aware adaptive return-sizing (opt-in, default off). Trim
|
||||
// the ranked candidate set to an intent-driven cap BEFORE the limit slice,
|
||||
// and only on the first page (offset===0) — paginating a confidence-gated
|
||||
// set is incoherent, so paginated calls fall through to the fixed limit.
|
||||
// Runs on the alias-hopped set so an alias-injected page (top-of-organic)
|
||||
// survives the trim.
|
||||
const adaptiveCfg = resolveAdaptiveReturn(
|
||||
opts?.adaptiveReturn,
|
||||
adaptiveReturnFromConfig(cfgForColumn as Record<string, unknown> | null),
|
||||
);
|
||||
let returnPool = reranked;
|
||||
let returnPool = aliasHopped;
|
||||
let adaptiveDecision: AdaptiveReturnDecision | undefined;
|
||||
if (adaptiveCfg.enabled && offset === 0) {
|
||||
const r = applyAdaptiveReturn(reranked, suggestions.intent, adaptiveCfg);
|
||||
const r = applyAdaptiveReturn(aliasHopped, suggestions.intent, adaptiveCfg);
|
||||
returnPool = r.kept;
|
||||
adaptiveDecision = r.decision;
|
||||
}
|
||||
@@ -1058,6 +1257,7 @@ export async function hybridSearch(
|
||||
// the same budget behavior as the production query op.
|
||||
const { results: budgeted, meta: budgetMeta } = enforceTokenBudget(sliced, resolvedMode.tokenBudget);
|
||||
lastResultsCount = budgeted.length;
|
||||
lastRank1Score = budgeted[0] ? (budgeted[0].base_score ?? budgeted[0].score) : undefined;
|
||||
emitMeta({
|
||||
vector_enabled: true,
|
||||
detail_resolved: detailResolved,
|
||||
@@ -1106,7 +1306,10 @@ export async function hybridSearchCached(
|
||||
const { loadSearchModeConfig, resolveSearchMode, knobsHash } = await import('./mode.ts');
|
||||
const modeInputForCache = await loadSearchModeConfig(engine);
|
||||
const resolvedForCache = resolveSearchMode({
|
||||
mode: modeInputForCache.mode,
|
||||
// T4/D5 — per-call mode folds into the cache key (resolved_mode is part
|
||||
// of knobsHash) so a per-call `--mode tokenmax` read can't be served a
|
||||
// server-default-mode cache row.
|
||||
mode: opts?.mode ?? modeInputForCache.mode,
|
||||
overrides: modeInputForCache.overrides,
|
||||
perCall: {
|
||||
cache_enabled: opts?.useCache,
|
||||
|
||||
+38
-1
@@ -140,6 +140,16 @@ export interface ModeBundle {
|
||||
*/
|
||||
floor_ratio: number | undefined;
|
||||
|
||||
/**
|
||||
* T2 (retrieval-maxpool incident) — title-phrase boost multiplier. When a
|
||||
* query is a contiguous token-run inside a page's title (or an exact full-
|
||||
* title match), multiply that result's score by this factor. <= 1.0 or
|
||||
* undefined disables. Floor-ratio-gated so a title hit can't bury a strong
|
||||
* semantic match. Correctness fix (cheap, in-memory) — ON in all bundles.
|
||||
* Override: per-call SearchOpts → `search.title_boost` config → bundle.
|
||||
*/
|
||||
title_boost: number | undefined;
|
||||
|
||||
// v0.36 cross-modal wave knobs (D2 + D3 + D6 + D8 + D13 + LLM-intent).
|
||||
// All three mode bundles default these to the same values — cross-modal
|
||||
// is opt-in per-call (D6 weighting), opt-in per-brain (D8 unified flags),
|
||||
@@ -260,6 +270,8 @@ export const MODE_BUNDLES: Readonly<Record<SearchMode, Readonly<ModeBundle>>> =
|
||||
// v0.35.6.0 — undefined for all three bundles; the per-corpus ablation
|
||||
// (TODOS.md) gates any default flip.
|
||||
floor_ratio: undefined,
|
||||
// T2 — title-phrase boost ON by default (correctness fix, cheap + gated).
|
||||
title_boost: 1.25,
|
||||
// v0.36 cross-modal defaults (same across all modes — opt-in)
|
||||
cross_modal_both_text_weight: 0.6,
|
||||
cross_modal_both_image_weight: 0.4,
|
||||
@@ -300,6 +312,8 @@ export const MODE_BUNDLES: Readonly<Record<SearchMode, Readonly<ModeBundle>>> =
|
||||
// v0.35.6.0 — undefined for all three bundles; the per-corpus ablation
|
||||
// (TODOS.md) gates any default flip.
|
||||
floor_ratio: undefined,
|
||||
// T2 — title-phrase boost ON by default (correctness fix, cheap + gated).
|
||||
title_boost: 1.25,
|
||||
// v0.36 cross-modal defaults (same across all modes — opt-in)
|
||||
cross_modal_both_text_weight: 0.6,
|
||||
cross_modal_both_image_weight: 0.4,
|
||||
@@ -342,6 +356,8 @@ export const MODE_BUNDLES: Readonly<Record<SearchMode, Readonly<ModeBundle>>> =
|
||||
// v0.35.6.0 — undefined for all three bundles; the per-corpus ablation
|
||||
// (TODOS.md) gates any default flip.
|
||||
floor_ratio: undefined,
|
||||
// T2 — title-phrase boost ON by default (correctness fix, cheap + gated).
|
||||
title_boost: 1.25,
|
||||
// v0.36 cross-modal defaults (same across all modes — opt-in)
|
||||
cross_modal_both_text_weight: 0.6,
|
||||
cross_modal_both_image_weight: 0.4,
|
||||
@@ -392,6 +408,8 @@ export interface SearchKeyOverrides {
|
||||
reranker_timeout_ms?: number;
|
||||
// v0.35.6.0 — floor-ratio gate override.
|
||||
floor_ratio?: number;
|
||||
// T2 — title-phrase boost override.
|
||||
title_boost?: number;
|
||||
// v0.36 cross-modal overrides
|
||||
cross_modal_both_text_weight?: number;
|
||||
cross_modal_both_image_weight?: number;
|
||||
@@ -430,6 +448,8 @@ export interface SearchPerCallOpts {
|
||||
reranker_timeout_ms?: number;
|
||||
// v0.35.6.0 — floor-ratio per-call override.
|
||||
floor_ratio?: number;
|
||||
// T2 — title-phrase boost per-call override.
|
||||
title_boost?: number;
|
||||
// v0.36 cross-modal per-call overrides
|
||||
cross_modal_both_text_weight?: number;
|
||||
cross_modal_both_image_weight?: number;
|
||||
@@ -518,6 +538,7 @@ export function resolveSearchMode(input: ResolveSearchModeInput): ResolvedSearch
|
||||
reranker_timeout_ms: pickRerankerTimeoutMs(),
|
||||
// v0.35.6.0 — floor-ratio resolved via the same pick chain.
|
||||
floor_ratio: pick('floor_ratio'),
|
||||
title_boost: pick('title_boost'),
|
||||
// v0.36 cross-modal knobs
|
||||
cross_modal_both_text_weight: pick('cross_modal_both_text_weight'),
|
||||
cross_modal_both_image_weight: pick('cross_modal_both_image_weight'),
|
||||
@@ -616,7 +637,12 @@ export function attributeKnob<K extends keyof ModeBundle>(
|
||||
// 1.05x multiplier. Cached pre-v0.42 entries don't reflect the boost so
|
||||
// must invalidate. Same one-time miss-spike pattern as prior bumps;
|
||||
// fills within cache.ttl_seconds (3600s default).
|
||||
export const KNOBS_HASH_VERSION = 6;
|
||||
//
|
||||
// T2 bump 6→7: title_boost (retrieval-maxpool incident) adds a post-fusion
|
||||
// stage that multiplies title-phrase-matching results. A title-boost-on write
|
||||
// must NOT be served to a title-boost-off lookup (ranking shifts). Same
|
||||
// one-time miss-spike pattern; fills within cache.ttl_seconds.
|
||||
export const KNOBS_HASH_VERSION = 7;
|
||||
|
||||
/**
|
||||
* v0.36 (D8 / CDX-2) — second-arg context for the cache key. The
|
||||
@@ -712,6 +738,8 @@ export function knobsHash(
|
||||
// neutralizes prior cache rows.
|
||||
`cr=${knobs.contextual_retrieval}`,
|
||||
`crd=${knobs.contextual_retrieval_disabled ? 1 : 0}`,
|
||||
// v=7 addition (append-only) — T2 title-phrase boost (retrieval-maxpool).
|
||||
`tib=${knobs.title_boost === undefined ? 'none' : knobs.title_boost.toFixed(4)}`,
|
||||
];
|
||||
const h = createHash('sha256');
|
||||
h.update(parts.join('|'));
|
||||
@@ -809,6 +837,14 @@ export function loadOverridesFromConfig(
|
||||
if (Number.isFinite(n) && n >= 0 && n <= 1) out.floor_ratio = n;
|
||||
}
|
||||
|
||||
// T2 — title-phrase boost factor. >= 1.0 (1.0 disables). Bounded sanity cap
|
||||
// at 5.0 so a fat-fingered config can't make a title hit dominate everything.
|
||||
const tib = get('search.title_boost');
|
||||
if (tib !== undefined) {
|
||||
const n = parseFloat(tib);
|
||||
if (Number.isFinite(n) && n >= 1.0 && n <= 5.0) out.title_boost = n;
|
||||
}
|
||||
|
||||
// v0.36 cross-modal overrides (D3 registry)
|
||||
const cmbt = get('search.cross_modal.both_mode_text_weight');
|
||||
if (cmbt !== undefined) {
|
||||
@@ -878,6 +914,7 @@ export const SEARCH_MODE_CONFIG_KEYS: ReadonlyArray<string> = Object.freeze([
|
||||
'search.reranker.timeout_ms',
|
||||
// v0.35.6.0 — floor-ratio gate
|
||||
'search.floor_ratio',
|
||||
'search.title_boost',
|
||||
// v0.36 cross-modal keys (D3)
|
||||
'search.cross_modal.both_mode_text_weight',
|
||||
'search.cross_modal.both_mode_image_weight',
|
||||
|
||||
@@ -129,6 +129,57 @@ export function buildVisibilityClause(pageAlias: string, sourceAlias: string): s
|
||||
return `AND ${pageAlias}.deleted_at IS NULL AND NOT ${sourceAlias}.archived`;
|
||||
}
|
||||
|
||||
// ============================================================
|
||||
// Per-page max-pool (T1 / D7) — single source of truth
|
||||
// ============================================================
|
||||
|
||||
/**
|
||||
* Build the `best_per_page` pooling CTE: collapse a chunk-grain candidate set
|
||||
* to ONE row per page — the page's highest-scoring chunk.
|
||||
*
|
||||
* This is the per-page max-pool that `searchKeyword` always had and that
|
||||
* `searchVector` was missing (the retrieval-maxpool incident: a page got
|
||||
* represented by whichever chunk survived the candidate cut, not its best
|
||||
* chunk). Both engines (postgres + pglite) AND both retrieval paths
|
||||
* (keyword + vector) consume this one builder so they cannot drift — the
|
||||
* recurring postgres/pglite parity bug class this repo guards against.
|
||||
*
|
||||
* Contract on the candidate CTE (`candidateCte`):
|
||||
* - exposes `source_id` + `slug` columns (the composite per-page collapse key)
|
||||
* - exposes a numeric `score` column (the value pooled on)
|
||||
* - exposes `page_id` and `chunk_id` columns (deterministic tiebreak)
|
||||
*
|
||||
* Collapse key is COMPOSITE `(source_id, slug)`, NOT slug alone — two pages
|
||||
* with the same slug in different sources are distinct pages (the federated
|
||||
* multi-source contract; matches dedup.ts's pageKey and the v0.34.1 source
|
||||
* isolation seal). Pooling on bare slug would collapse them and drop the
|
||||
* neighbor-source page before ranking. `COALESCE(source_id, 'default')` keeps
|
||||
* pre-v0.17 single-source rows (null source_id) collapsing correctly.
|
||||
*
|
||||
* Determinism: `DISTINCT ON` keeps the FIRST row per key under the ORDER BY,
|
||||
* so the tiebreak `… score DESC, page_id ASC, chunk_id ASC` makes the surviving
|
||||
* chunk fully deterministic when two chunks of the same page tie on score
|
||||
* (basis-vector eval fixtures, planner-independent — same rationale as the
|
||||
* v0.41.13 searchVector stable tiebreaker).
|
||||
*
|
||||
* Pooling happens over the FULL candidate set (`innerLimit` rows) BEFORE the
|
||||
* user-facing `LIMIT`, so a page's best chunk can't be truncated out by
|
||||
* weaker chunks of OTHER pages occupying the early `LIMIT` slots — the vector
|
||||
* path now returns N distinct pages (each by best chunk), not N chunks that
|
||||
* collapse to fewer pages downstream.
|
||||
*
|
||||
* @param candidateCte — name of the upstream CTE to pool (e.g. `'hnsw_candidates'`,
|
||||
* `'ranked_chunks'`). Engine-supplied identifier, never user input.
|
||||
* @returns raw SQL fragment: `best_per_page AS ( ... )` (no trailing comma)
|
||||
*/
|
||||
export function buildBestPerPagePoolCte(candidateCte: string): string {
|
||||
return `best_per_page AS (
|
||||
SELECT DISTINCT ON (COALESCE(source_id, 'default'), slug) *
|
||||
FROM ${candidateCte}
|
||||
ORDER BY COALESCE(source_id, 'default'), slug, score DESC, page_id ASC, chunk_id ASC
|
||||
)`;
|
||||
}
|
||||
|
||||
// ============================================================
|
||||
// v0.29.1 — Recency component SQL builder
|
||||
// ============================================================
|
||||
|
||||
@@ -36,8 +36,19 @@ interface Bucket {
|
||||
sum_budget_dropped: number;
|
||||
cache_hit: number;
|
||||
cache_miss: number;
|
||||
// T7 — rank-1 base_score drift signal (aggregate, NOT per-query rows, D10).
|
||||
// sum/count derive the mean; 3 coarse buckets give a distribution shape.
|
||||
sum_rank1_score: number;
|
||||
count_rank1: number;
|
||||
rank1_lt_solid: number; // base_score < 0.6
|
||||
rank1_solid: number; // 0.6 <= base_score < 0.85
|
||||
rank1_high: number; // base_score >= 0.85
|
||||
}
|
||||
|
||||
// T7 — coarse rank-1 score bands (mirror evidence.ts SOLID/HIGH floors).
|
||||
const RANK1_SOLID_FLOOR = 0.6;
|
||||
const RANK1_HIGH_FLOOR = 0.85;
|
||||
|
||||
const FLUSH_INTERVAL_MS = 60_000;
|
||||
const FLUSH_THRESHOLD_CALLS = 100;
|
||||
|
||||
@@ -69,7 +80,7 @@ class TelemetryWriter {
|
||||
* immediately after bumping the in-memory bucket. Flush is async +
|
||||
* fire-and-forget.
|
||||
*/
|
||||
record(meta: HybridSearchMeta, opts: { results_count: number; tokens_estimate?: number } = { results_count: 0 }): void {
|
||||
record(meta: HybridSearchMeta, opts: { results_count: number; tokens_estimate?: number; rank1_score?: number } = { results_count: 0 }): void {
|
||||
const date = nowDate();
|
||||
const mode = meta.mode ?? 'unset';
|
||||
const intent = meta.intent ?? 'unset';
|
||||
@@ -87,6 +98,11 @@ class TelemetryWriter {
|
||||
sum_budget_dropped: 0,
|
||||
cache_hit: 0,
|
||||
cache_miss: 0,
|
||||
sum_rank1_score: 0,
|
||||
count_rank1: 0,
|
||||
rank1_lt_solid: 0,
|
||||
rank1_solid: 0,
|
||||
rank1_high: 0,
|
||||
};
|
||||
this.buckets.set(key, b);
|
||||
}
|
||||
@@ -97,6 +113,16 @@ class TelemetryWriter {
|
||||
b.sum_budget_dropped += Math.max(0, Math.floor(meta.token_budget?.dropped ?? 0));
|
||||
if (meta.cache?.status === 'hit') b.cache_hit += 1;
|
||||
if (meta.cache?.status === 'miss') b.cache_miss += 1;
|
||||
// T7 — rank-1 base_score drift signal. Only counts queries that returned
|
||||
// a result (rank1_score present + finite).
|
||||
if (typeof opts.rank1_score === 'number' && Number.isFinite(opts.rank1_score)) {
|
||||
const s = opts.rank1_score;
|
||||
b.sum_rank1_score += s;
|
||||
b.count_rank1 += 1;
|
||||
if (s < RANK1_SOLID_FLOOR) b.rank1_lt_solid += 1;
|
||||
else if (s < RANK1_HIGH_FLOOR) b.rank1_solid += 1;
|
||||
else b.rank1_high += 1;
|
||||
}
|
||||
|
||||
this.pendingCount += 1;
|
||||
if (this.pendingCount >= FLUSH_THRESHOLD_CALLS) {
|
||||
@@ -129,8 +155,9 @@ class TelemetryWriter {
|
||||
try {
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO search_telemetry
|
||||
(date, mode, intent, count, sum_results, sum_tokens, sum_budget_dropped, cache_hit, cache_miss, first_seen, last_seen)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, now(), now())
|
||||
(date, mode, intent, count, sum_results, sum_tokens, sum_budget_dropped, cache_hit, cache_miss,
|
||||
sum_rank1_score, count_rank1, rank1_lt_solid, rank1_solid, rank1_high, first_seen, last_seen)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, now(), now())
|
||||
ON CONFLICT (date, mode, intent) DO UPDATE SET
|
||||
count = search_telemetry.count + EXCLUDED.count,
|
||||
sum_results = search_telemetry.sum_results + EXCLUDED.sum_results,
|
||||
@@ -138,8 +165,14 @@ class TelemetryWriter {
|
||||
sum_budget_dropped = search_telemetry.sum_budget_dropped + EXCLUDED.sum_budget_dropped,
|
||||
cache_hit = search_telemetry.cache_hit + EXCLUDED.cache_hit,
|
||||
cache_miss = search_telemetry.cache_miss + EXCLUDED.cache_miss,
|
||||
sum_rank1_score = search_telemetry.sum_rank1_score + EXCLUDED.sum_rank1_score,
|
||||
count_rank1 = search_telemetry.count_rank1 + EXCLUDED.count_rank1,
|
||||
rank1_lt_solid = search_telemetry.rank1_lt_solid + EXCLUDED.rank1_lt_solid,
|
||||
rank1_solid = search_telemetry.rank1_solid + EXCLUDED.rank1_solid,
|
||||
rank1_high = search_telemetry.rank1_high + EXCLUDED.rank1_high,
|
||||
last_seen = now()`,
|
||||
[b.date, b.mode, b.intent, b.count, b.sum_results, b.sum_tokens, b.sum_budget_dropped, b.cache_hit, b.cache_miss],
|
||||
[b.date, b.mode, b.intent, b.count, b.sum_results, b.sum_tokens, b.sum_budget_dropped, b.cache_hit, b.cache_miss,
|
||||
b.sum_rank1_score, b.count_rank1, b.rank1_lt_solid, b.rank1_solid, b.rank1_high],
|
||||
);
|
||||
} catch {
|
||||
// swallow — telemetry write must never break the hot path.
|
||||
@@ -230,7 +263,7 @@ export function getTelemetryWriter(): TelemetryWriter {
|
||||
export function recordSearchTelemetry(
|
||||
engine: BrainEngine,
|
||||
meta: HybridSearchMeta,
|
||||
opts: { results_count: number; tokens_estimate?: number } = { results_count: 0 },
|
||||
opts: { results_count: number; tokens_estimate?: number; rank1_score?: number } = { results_count: 0 },
|
||||
): void {
|
||||
try {
|
||||
const w = getTelemetryWriter();
|
||||
@@ -258,6 +291,11 @@ export interface StatsWindow {
|
||||
window_days: number;
|
||||
oldest_seen?: string;
|
||||
newest_seen?: string;
|
||||
// T7 — rank-1 base_score drift signal. avg_rank1_score is the headline the
|
||||
// doctor/operator watches for downward drift; the 3 buckets give shape.
|
||||
avg_rank1_score: number | null; // null when no rank-1 samples
|
||||
rank1_count: number;
|
||||
rank1_distribution: { lt_solid: number; solid: number; high: number };
|
||||
}
|
||||
|
||||
export async function readSearchStats(
|
||||
@@ -277,6 +315,11 @@ export async function readSearchStats(
|
||||
sum_budget_dropped: number;
|
||||
cache_hit: number;
|
||||
cache_miss: number;
|
||||
sum_rank1_score: number;
|
||||
count_rank1: number;
|
||||
rank1_lt_solid: number;
|
||||
rank1_solid: number;
|
||||
rank1_high: number;
|
||||
first_seen: string;
|
||||
last_seen: string;
|
||||
}>(
|
||||
@@ -287,6 +330,11 @@ export async function readSearchStats(
|
||||
SUM(sum_budget_dropped)::int AS sum_budget_dropped,
|
||||
SUM(cache_hit)::int AS cache_hit,
|
||||
SUM(cache_miss)::int AS cache_miss,
|
||||
COALESCE(SUM(sum_rank1_score), 0)::float8 AS sum_rank1_score,
|
||||
COALESCE(SUM(count_rank1), 0)::int AS count_rank1,
|
||||
COALESCE(SUM(rank1_lt_solid), 0)::int AS rank1_lt_solid,
|
||||
COALESCE(SUM(rank1_solid), 0)::int AS rank1_solid,
|
||||
COALESCE(SUM(rank1_high), 0)::int AS rank1_high,
|
||||
MIN(first_seen)::text AS first_seen,
|
||||
MAX(last_seen)::text AS last_seen
|
||||
FROM search_telemetry
|
||||
@@ -305,6 +353,11 @@ export async function readSearchStats(
|
||||
const mode_distribution: Record<string, number> = {};
|
||||
let oldest_seen: string | undefined;
|
||||
let newest_seen: string | undefined;
|
||||
let sum_rank1 = 0;
|
||||
let count_rank1 = 0;
|
||||
let r1_lt = 0;
|
||||
let r1_solid = 0;
|
||||
let r1_high = 0;
|
||||
|
||||
for (const r of rows) {
|
||||
total_calls += r.count;
|
||||
@@ -313,6 +366,11 @@ export async function readSearchStats(
|
||||
total_results += r.sum_results;
|
||||
total_tokens += r.sum_tokens;
|
||||
total_budget_dropped += r.sum_budget_dropped;
|
||||
sum_rank1 += r.sum_rank1_score;
|
||||
count_rank1 += r.count_rank1;
|
||||
r1_lt += r.rank1_lt_solid;
|
||||
r1_solid += r.rank1_solid;
|
||||
r1_high += r.rank1_high;
|
||||
intent_distribution[r.intent] = (intent_distribution[r.intent] ?? 0) + r.count;
|
||||
mode_distribution[r.mode] = (mode_distribution[r.mode] ?? 0) + r.count;
|
||||
if (r.first_seen && (!oldest_seen || r.first_seen < oldest_seen)) oldest_seen = r.first_seen;
|
||||
@@ -333,6 +391,9 @@ export async function readSearchStats(
|
||||
window_days: days,
|
||||
oldest_seen,
|
||||
newest_seen,
|
||||
avg_rank1_score: count_rank1 > 0 ? sum_rank1 / count_rank1 : null,
|
||||
rank1_count: count_rank1,
|
||||
rank1_distribution: { lt_solid: r1_lt, solid: r1_solid, high: r1_high },
|
||||
};
|
||||
} catch {
|
||||
// Table missing or query failed — return empty stats rather than throw.
|
||||
@@ -347,6 +408,9 @@ export async function readSearchStats(
|
||||
intent_distribution: {},
|
||||
mode_distribution: {},
|
||||
window_days: days,
|
||||
avg_rank1_score: null,
|
||||
rank1_count: 0,
|
||||
rank1_distribution: { lt_solid: 0, solid: 0, high: 0 },
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
/**
|
||||
* Title-superstring matching for the T2 title boost (retrieval-maxpool incident).
|
||||
*
|
||||
* The disease: a query that is literally a phrase from a page's title
|
||||
* ("Greek amphitheater" → "The Mingtang — Indoor Greek amphitheater…") matched
|
||||
* a weak body chunk instead of being recognized as a title hit. Names of things
|
||||
* deserve weight. This module decides, deterministically and with zero I/O,
|
||||
* whether a query is a title-phrase match for a page.
|
||||
*
|
||||
* Guard rails (Codex#10 — avoid promoting generic pages on stopword-y queries):
|
||||
* - require >= MIN_CONTENT_TOKENS non-stopword tokens in the query, OR an
|
||||
* exact normalized full-title match (so a deliberate 1-word title still hits);
|
||||
* - match at TOKEN BOUNDARIES (contiguous token run), never raw substring, so
|
||||
* "art" doesn't match "Bartholomew";
|
||||
* - stopwords ("the", "a", "of", …) don't count toward the content-token floor.
|
||||
*
|
||||
* Pure + exported so the NamedThingBench eval and unit tests share one definition
|
||||
* with the production boost (no drift).
|
||||
*/
|
||||
|
||||
const MIN_CONTENT_TOKENS = 2;
|
||||
|
||||
// Small, deliberately conservative English stopword set. CJK has no whitespace
|
||||
// stopword notion here; CJK queries fall through to the exact-match path.
|
||||
const STOPWORDS = new Set([
|
||||
'the', 'a', 'an', 'of', 'and', 'or', 'to', 'in', 'on', 'for', 'with',
|
||||
'at', 'by', 'from', 'as', 'is', 'it', 'this', 'that', 'my', 'your',
|
||||
]);
|
||||
|
||||
/**
|
||||
* Normalize text to a token array: lowercase, split on any run of
|
||||
* non-alphanumeric (Unicode-aware for letters/numbers), drop empties.
|
||||
* CJK characters are letters under \p{L}, so a CJK title collapses to one
|
||||
* token per contiguous run — fine for the exact-match path.
|
||||
*/
|
||||
export function tokenizeTitle(s: string): string[] {
|
||||
if (!s) return [];
|
||||
return s
|
||||
.toLowerCase()
|
||||
.normalize('NFKC')
|
||||
.split(/[^\p{L}\p{N}]+/u)
|
||||
.filter(Boolean);
|
||||
}
|
||||
|
||||
/** Content tokens = tokens that aren't stopwords. */
|
||||
function contentTokens(tokens: string[]): string[] {
|
||||
return tokens.filter(t => !STOPWORDS.has(t));
|
||||
}
|
||||
|
||||
/** True iff `needle` appears as a contiguous token run inside `haystack`. */
|
||||
function containsTokenRun(haystack: string[], needle: string[]): boolean {
|
||||
if (needle.length === 0 || needle.length > haystack.length) return false;
|
||||
for (let i = 0; i + needle.length <= haystack.length; i++) {
|
||||
let ok = true;
|
||||
for (let j = 0; j < needle.length; j++) {
|
||||
if (haystack[i + j] !== needle[j]) { ok = false; break; }
|
||||
}
|
||||
if (ok) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide whether `query` is a title-phrase match for `title`.
|
||||
*
|
||||
* Returns true when EITHER:
|
||||
* (a) the normalized query token sequence is a contiguous run inside the
|
||||
* normalized title AND the query has >= MIN_CONTENT_TOKENS content tokens; OR
|
||||
* (b) the normalized query equals the full normalized title exactly (covers
|
||||
* deliberate single-word titles / chosen names like "Mingtang").
|
||||
*
|
||||
* Order-insensitive matching is intentionally NOT done — a title-phrase signal
|
||||
* should reflect the author's phrasing, and bag-of-words matching is what the
|
||||
* vector/keyword paths already provide.
|
||||
*/
|
||||
export function isTitlePhraseMatch(query: string, title: string): boolean {
|
||||
const qTokens = tokenizeTitle(query);
|
||||
const tTokens = tokenizeTitle(title);
|
||||
if (qTokens.length === 0 || tTokens.length === 0) return false;
|
||||
|
||||
// (b) exact full-title match (covers 1-word chosen names).
|
||||
if (qTokens.length === tTokens.length && qTokens.every((t, i) => t === tTokens[i])) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// (a) contiguous phrase match, gated by content-token floor.
|
||||
const qContent = contentTokens(qTokens);
|
||||
if (qContent.length < MIN_CONTENT_TOKENS) return false;
|
||||
return containsTokenRun(tTokens, qTokens);
|
||||
}
|
||||
|
||||
// Exported for unit tests.
|
||||
export const __test__ = { tokenizeTitle, contentTokens, containsTokenRun, STOPWORDS, MIN_CONTENT_TOKENS };
|
||||
@@ -653,6 +653,38 @@ export interface SearchResult {
|
||||
* as canonical" and lets canonicals outrank fuzzy matches.
|
||||
*/
|
||||
alias_resolved_boost?: number;
|
||||
/**
|
||||
* T2 (retrieval-maxpool incident) — multiplier applied by applyTitleBoost
|
||||
* (1.0 = unchanged; default ~1.25x). Fires when the normalized query is a
|
||||
* contiguous token-run inside the page title (or an exact full-title match).
|
||||
* Floor-ratio-gated + clamped so a title hit reorders without burying a
|
||||
* strong semantic match or lying about base_score (the agent's dedup gate
|
||||
* keys off base_score / evidence, not the boosted score).
|
||||
*/
|
||||
title_match_boost?: number;
|
||||
/**
|
||||
* T3 (retrieval-maxpool incident) — set when this result was surfaced or
|
||||
* boosted by the free-text alias hop (the query exactly matched a page's
|
||||
* declared alias in page_aliases). An injected canonical that wasn't in the
|
||||
* organic candidate set carries alias_hit=true with score = top-of-organic
|
||||
* + epsilon. Drives the evidence=alias_hit signal the agent's don't-duplicate
|
||||
* decision keys off (T4).
|
||||
*/
|
||||
alias_hit?: boolean;
|
||||
/**
|
||||
* T4 — the strongest signal that surfaced this page (alias_hit >
|
||||
* exact_title_match > high_vector_match > keyword_exact > weak_semantic).
|
||||
* Computed by classifyEvidence at the end of the hybrid pipeline.
|
||||
*/
|
||||
evidence?: import('./search/evidence.ts').Evidence;
|
||||
/**
|
||||
* T4 — derived "is this page already in the brain?" hint. The agent's
|
||||
* don't-write-a-duplicate decision keys off THIS, not a raw score:
|
||||
* 'exists' = strong (don't duplicate), 'probable' = prefer update,
|
||||
* 'unknown' = look closer. This is the contract that prevents the
|
||||
* incident's duplicate-stub class.
|
||||
*/
|
||||
create_safety?: import('./search/evidence.ts').CreateSafety;
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -0,0 +1,184 @@
|
||||
/**
|
||||
* NamedThingBench (T6) — the retrieval-quality eval that makes the
|
||||
* retrieval-maxpool incident impossible to reintroduce silently.
|
||||
*
|
||||
* Query families, each a distinct failure class the incident exposed:
|
||||
* title-substring — query is a phrase in the title (the direct regression)
|
||||
* generic-to-named — tourist label -> the named thing
|
||||
* alias-synonym — declared alias / romanization -> canonical
|
||||
* multi-chunk-dilution — one strong chunk among many weak (stresses max-pool)
|
||||
* short-vs-rich — 2-word vs 8-word form of the same intent
|
||||
* graph-relationship — relationship query (guardrail: don't regress)
|
||||
* hard-negative — query that must NOT return a page (precision guard)
|
||||
*
|
||||
* Pure: the caller injects a SearchFn (CLI uses hybridSearch; tests stub it),
|
||||
* so the harness is engine-agnostic and runs identically against postgres +
|
||||
* pglite + a deterministic stub.
|
||||
*/
|
||||
|
||||
export type Family =
|
||||
| 'title-substring'
|
||||
| 'generic-to-named'
|
||||
| 'alias-synonym'
|
||||
| 'multi-chunk-dilution'
|
||||
| 'short-vs-rich'
|
||||
| 'graph-relationship'
|
||||
| 'hard-negative';
|
||||
|
||||
export interface NamedThingQuestion {
|
||||
family: Family;
|
||||
query: string;
|
||||
/** slugs that SHOULD rank for this query (non-hard-negative families). */
|
||||
relevant?: string[];
|
||||
/** slugs that must NOT appear in top-k (hard-negative family). */
|
||||
forbidden?: string[];
|
||||
notes?: string;
|
||||
}
|
||||
|
||||
/** Ranked slugs for a query, best-first. */
|
||||
export type SearchFn = (query: string) => Promise<string[]>;
|
||||
|
||||
export interface QuestionResult {
|
||||
family: Family;
|
||||
query: string;
|
||||
hit_at_1: boolean;
|
||||
hit_at_3: boolean;
|
||||
reciprocal_rank: number; // 0 if no relevant slug in results
|
||||
/** hard-negative only: true when NO forbidden slug appeared in top-3. */
|
||||
negative_clean?: boolean;
|
||||
}
|
||||
|
||||
export interface FamilyReport {
|
||||
family: Family;
|
||||
n: number;
|
||||
hit_at_1: number; // rate
|
||||
hit_at_3: number; // rate
|
||||
mrr: number;
|
||||
}
|
||||
|
||||
export interface RetrievalQualityReport {
|
||||
schema_version: 1;
|
||||
k: number;
|
||||
total: number;
|
||||
families: FamilyReport[];
|
||||
questions: QuestionResult[];
|
||||
}
|
||||
|
||||
const K = 3;
|
||||
|
||||
export function scoreQuestion(q: NamedThingQuestion, ranked: string[]): QuestionResult {
|
||||
if (q.family === 'hard-negative') {
|
||||
const forbidden = new Set(q.forbidden ?? []);
|
||||
const topK = ranked.slice(0, K);
|
||||
const clean = !topK.some(s => forbidden.has(s));
|
||||
return { family: q.family, query: q.query, hit_at_1: clean, hit_at_3: clean, reciprocal_rank: clean ? 1 : 0, negative_clean: clean };
|
||||
}
|
||||
const relevant = new Set(q.relevant ?? []);
|
||||
const firstRelevantIdx = ranked.findIndex(s => relevant.has(s));
|
||||
const hit1 = firstRelevantIdx === 0;
|
||||
const hit3 = firstRelevantIdx >= 0 && firstRelevantIdx < K;
|
||||
const rr = firstRelevantIdx >= 0 ? 1 / (firstRelevantIdx + 1) : 0;
|
||||
return { family: q.family, query: q.query, hit_at_1: hit1, hit_at_3: hit3, reciprocal_rank: rr };
|
||||
}
|
||||
|
||||
export async function runRetrievalQuality(
|
||||
questions: NamedThingQuestion[],
|
||||
searchFn: SearchFn,
|
||||
): Promise<RetrievalQualityReport> {
|
||||
const results: QuestionResult[] = [];
|
||||
for (const q of questions) {
|
||||
let ranked: string[] = [];
|
||||
try { ranked = await searchFn(q.query); } catch { ranked = []; }
|
||||
results.push(scoreQuestion(q, ranked));
|
||||
}
|
||||
const byFamily = new Map<Family, QuestionResult[]>();
|
||||
for (const r of results) {
|
||||
const list = byFamily.get(r.family) ?? [];
|
||||
list.push(r);
|
||||
byFamily.set(r.family, list);
|
||||
}
|
||||
const families: FamilyReport[] = [];
|
||||
for (const [family, list] of byFamily) {
|
||||
const n = list.length;
|
||||
families.push({
|
||||
family,
|
||||
n,
|
||||
hit_at_1: n ? list.filter(r => r.hit_at_1).length / n : 0,
|
||||
hit_at_3: n ? list.filter(r => r.hit_at_3).length / n : 0,
|
||||
mrr: n ? list.reduce((s, r) => s + r.reciprocal_rank, 0) / n : 0,
|
||||
});
|
||||
}
|
||||
families.sort((a, b) => a.family.localeCompare(b.family));
|
||||
return { schema_version: 1, k: K, total: results.length, families, questions: results };
|
||||
}
|
||||
|
||||
// ── Gate ────────────────────────────────────────────────────────────────
|
||||
|
||||
export interface GateOpts {
|
||||
/** Families hard-gated from day one (the bug's families). */
|
||||
hardFamilies: Partial<Record<Family, { hit_at_1?: number; hit_at_3?: number }>>;
|
||||
/** Soft families: reported, breaches are warnings (not failures) for now. */
|
||||
softFamilies: Family[];
|
||||
}
|
||||
|
||||
/**
|
||||
* D12 — hard-gate the two families that ARE this bug from day one
|
||||
* (title-substring Hit@1 >= 0.95, multi-chunk-dilution Hit@3 = 1.0), plus
|
||||
* alias-synonym Hit@1 >= 0.98 (Codex#13 exact numbers). Softer families
|
||||
* (generic-to-named, short-vs-rich, graph-relationship, hard-negative) are
|
||||
* warn-then-enforce until a 3x baseline noise floor is established.
|
||||
* Env-overridable like the existing replay-gate floors.
|
||||
*/
|
||||
export const DEFAULT_GATE: GateOpts = {
|
||||
hardFamilies: {
|
||||
'title-substring': { hit_at_1: floorEnv('GBRAIN_NTB_TITLE_HIT1', 0.95) },
|
||||
'multi-chunk-dilution': { hit_at_3: floorEnv('GBRAIN_NTB_DILUTION_HIT3', 1.0) },
|
||||
'alias-synonym': { hit_at_1: floorEnv('GBRAIN_NTB_ALIAS_HIT1', 0.98) },
|
||||
},
|
||||
softFamilies: ['generic-to-named', 'short-vs-rich', 'graph-relationship', 'hard-negative'],
|
||||
};
|
||||
|
||||
function floorEnv(name: string, dflt: number): number {
|
||||
const v = process.env[name];
|
||||
if (v === undefined) return dflt;
|
||||
const n = parseFloat(v);
|
||||
return Number.isFinite(n) && n >= 0 && n <= 1 ? n : dflt;
|
||||
}
|
||||
|
||||
export interface GateBreach { family: Family; metric: 'hit_at_1' | 'hit_at_3'; got: number; floor: number; }
|
||||
export interface GateResult { pass: boolean; breaches: GateBreach[]; warnings: GateBreach[]; }
|
||||
|
||||
export function evaluateGate(report: RetrievalQualityReport, opts: GateOpts = DEFAULT_GATE): GateResult {
|
||||
const byFamily = new Map(report.families.map(f => [f.family, f]));
|
||||
const breaches: GateBreach[] = [];
|
||||
for (const [family, floors] of Object.entries(opts.hardFamilies) as [Family, { hit_at_1?: number; hit_at_3?: number }][]) {
|
||||
const fr = byFamily.get(family);
|
||||
if (!fr || fr.n === 0) continue; // no questions for this family → nothing to gate
|
||||
if (floors.hit_at_1 !== undefined && fr.hit_at_1 < floors.hit_at_1) {
|
||||
breaches.push({ family, metric: 'hit_at_1', got: fr.hit_at_1, floor: floors.hit_at_1 });
|
||||
}
|
||||
if (floors.hit_at_3 !== undefined && fr.hit_at_3 < floors.hit_at_3) {
|
||||
breaches.push({ family, metric: 'hit_at_3', got: fr.hit_at_3, floor: floors.hit_at_3 });
|
||||
}
|
||||
}
|
||||
// Soft families: surface low Hit@3 as warnings (informational until enforced).
|
||||
const warnings: GateBreach[] = [];
|
||||
for (const family of opts.softFamilies) {
|
||||
const fr = byFamily.get(family);
|
||||
if (fr && fr.n > 0 && fr.hit_at_3 < 0.8) {
|
||||
warnings.push({ family, metric: 'hit_at_3', got: fr.hit_at_3, floor: 0.8 });
|
||||
}
|
||||
}
|
||||
return { pass: breaches.length === 0, breaches, warnings };
|
||||
}
|
||||
|
||||
export function parseQuestionsJsonl(text: string): NamedThingQuestion[] {
|
||||
const out: NamedThingQuestion[] = [];
|
||||
for (const line of text.split('\n')) {
|
||||
const t = line.trim();
|
||||
if (!t || t.startsWith('//') || t.startsWith('#')) continue;
|
||||
const obj = JSON.parse(t) as NamedThingQuestion;
|
||||
out.push(obj);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
/**
|
||||
* T5 — `gbrain search` dispatch reconcile. Subprocess test against the real
|
||||
* cli.ts entrypoint:
|
||||
* - `search modes` -> read-only config dashboard (NOT a search for "modes")
|
||||
* - `search "<q>"` -> cheap-hybrid free-text search (NOT "Unknown subcommand")
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
|
||||
function run(args: string[], home: string) {
|
||||
const r = spawnSync('bun', ['run', 'src/cli.ts', ...args], {
|
||||
cwd: process.cwd(),
|
||||
encoding: 'utf8',
|
||||
env: { ...process.env, GBRAIN_HOME: home, DATABASE_URL: '', GBRAIN_DATABASE_URL: '' },
|
||||
timeout: 45_000,
|
||||
});
|
||||
return { stdout: r.stdout ?? '', stderr: r.stderr ?? '', status: r.status ?? -1 };
|
||||
}
|
||||
|
||||
function withHome(fn: (home: string) => void) {
|
||||
const home = mkdtempSync(join(tmpdir(), 'gbrain-search-dispatch-'));
|
||||
mkdirSync(join(home, '.gbrain'), { recursive: true });
|
||||
writeFileSync(join(home, '.gbrain', 'config.json'), JSON.stringify({ engine: 'pglite' }));
|
||||
try { fn(home); } finally { rmSync(home, { recursive: true, force: true }); }
|
||||
}
|
||||
|
||||
describe('T5 — gbrain search dispatch', () => {
|
||||
test('`search modes --json` routes to the dashboard (active_mode), not a query', () => {
|
||||
withHome((home) => {
|
||||
const { stdout, status } = run(['search', 'modes', '--json'], home);
|
||||
expect(status).toBe(0);
|
||||
expect(stdout).toContain('active_mode');
|
||||
// It must NOT be a search-results payload for the literal word "modes".
|
||||
expect(stdout).not.toContain('Unknown subcommand');
|
||||
});
|
||||
});
|
||||
|
||||
test('`search "<freetext>"` routes to the cheap-hybrid search op (no "Unknown subcommand")', () => {
|
||||
withHome((home) => {
|
||||
const { stdout, stderr, status } = run(['search', 'zzz-no-such-page-xyz'], home);
|
||||
// Empty brain → "No results." (the search op's formatter), exit 0.
|
||||
expect(status).toBe(0);
|
||||
expect(stderr).not.toContain('Unknown subcommand');
|
||||
expect(stdout.toLowerCase()).toContain('no results');
|
||||
});
|
||||
});
|
||||
|
||||
test('`search stats --json` routes to the dashboard', () => {
|
||||
withHome((home) => {
|
||||
const { stdout, status } = run(['search', 'stats', '--json'], home);
|
||||
expect(status).toBe(0);
|
||||
// stats envelope, not a search-results array.
|
||||
expect(stdout).toMatch(/total_calls|cache_hit_rate|window_days/);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -142,7 +142,7 @@ describe('D2 — knobsHash differs across cross-modal knob values', () => {
|
||||
// v0.39 T21 (master) bump to v=4 for graph_signals + schema-pack fields.
|
||||
// v0.40.3.0 D8 bumps to v=5 (sequenced behind salem's v=4 graph-signals).
|
||||
// v0.41.22.0 (type-unification): 5→6 for alias_resolved post-fusion boost.
|
||||
expect(KNOBS_HASH_VERSION).toBe(6);
|
||||
expect(KNOBS_HASH_VERSION).toBe(7);
|
||||
});
|
||||
|
||||
test('flipping unified_multimodal changes the hash', () => {
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
/**
|
||||
* T6 — NamedThingBench hermetic gate. Seeds a synthetic brain matching the
|
||||
* committed fixture (placeholder names) and runs the harness through the real
|
||||
* hybridSearch pipeline. The embed transport is stubbed to throw, forcing the
|
||||
* deterministic keyword + title-boost + alias-hop path (free, no network) — the
|
||||
* vector max-pool guarantee is pinned separately by searchvector-maxpool.test.ts.
|
||||
*
|
||||
* The gate MUST pass for the families that ARE the incident (title-substring,
|
||||
* alias-synonym, multi-chunk-dilution).
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { readFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { hybridSearch } from '../src/core/search/hybrid.ts';
|
||||
import { __setEmbedTransportForTests } from '../src/core/ai/gateway.ts';
|
||||
import { parseQuestionsJsonl, runRetrievalQuality, evaluateGate, type SearchFn } from '../src/eval/retrieval-quality/harness.ts';
|
||||
import type { ChunkInput } from '../src/core/types.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
async function seedPage(slug: string, title: string, type: string, chunks: string[], aliases: string[] = []) {
|
||||
await engine.putPage(slug, { type: type as never, title, compiled_truth: chunks.join('\n') });
|
||||
const ci: ChunkInput[] = chunks.map((text, i) => ({ chunk_index: i, chunk_text: text, chunk_source: 'compiled_truth', token_count: 10 }));
|
||||
await engine.upsertChunks(slug, ci);
|
||||
if (aliases.length) await engine.setPageAliases(slug, 'default', aliases.map(a => a.toLowerCase()));
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
// Force keyword + title + alias path: vector embed throws → hybrid falls open.
|
||||
__setEmbedTransportForTests(() => { throw new Error('stub: no embed in NamedThingBench test'); });
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
|
||||
await seedPage(
|
||||
'projects/example-amphitheater',
|
||||
'The Example Hall — Indoor Greek Amphitheater for Adversarial Debate',
|
||||
'note',
|
||||
[
|
||||
'Indoor greek amphitheater for adversarial debate in the city.',
|
||||
'Ceiling treatment acoustics for the amphitheater dome and seating.',
|
||||
],
|
||||
['Hall of Light'],
|
||||
);
|
||||
await seedPage(
|
||||
'projects/example-civic-platform',
|
||||
'Example Civic Feedback Platform',
|
||||
'note',
|
||||
['A civic feedback platform for the city to gather resident input.'],
|
||||
['the widget tracker'],
|
||||
);
|
||||
await seedPage(
|
||||
'people/alice-example',
|
||||
'Alice Example',
|
||||
'person',
|
||||
['Alice works on the civic feedback platform and gathers resident input.'],
|
||||
);
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
__setEmbedTransportForTests(null);
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
describe('NamedThingBench gate (hermetic)', () => {
|
||||
test('the bug families pass the gate (title-substring, alias-synonym, dilution)', async () => {
|
||||
const questions = parseQuestionsJsonl(
|
||||
readFileSync(join(import.meta.dir, 'fixtures/retrieval-quality/namedthing.jsonl'), 'utf8'),
|
||||
);
|
||||
const searchFn: SearchFn = async (q) => {
|
||||
const rs = await hybridSearch(engine, q, { limit: 10, sourceId: 'default' });
|
||||
return rs.map(r => r.slug);
|
||||
};
|
||||
const report = await runRetrievalQuality(questions, searchFn);
|
||||
const gate = evaluateGate(report);
|
||||
|
||||
// Diagnostic on failure: surface per-family rates.
|
||||
if (!gate.pass) {
|
||||
console.error('NamedThingBench families:', JSON.stringify(report.families, null, 2));
|
||||
console.error('breaches:', JSON.stringify(gate.breaches, null, 2));
|
||||
}
|
||||
expect(gate.pass).toBe(true);
|
||||
|
||||
const byFam = new Map(report.families.map(f => [f.family, f]));
|
||||
expect(byFam.get('title-substring')!.hit_at_1).toBeGreaterThanOrEqual(0.95);
|
||||
expect(byFam.get('alias-synonym')!.hit_at_1).toBeGreaterThanOrEqual(0.98);
|
||||
expect(byFam.get('multi-chunk-dilution')!.hit_at_3).toBe(1.0);
|
||||
});
|
||||
|
||||
test('hard-negative: the named projects do NOT surface for an unrelated query', async () => {
|
||||
const rs = await hybridSearch(engine, 'quarterly tax filing checklist', { limit: 10, sourceId: 'default' });
|
||||
const slugs = rs.map(r => r.slug);
|
||||
expect(slugs).not.toContain('projects/example-amphitheater');
|
||||
expect(slugs).not.toContain('projects/example-civic-platform');
|
||||
});
|
||||
});
|
||||
@@ -4,28 +4,41 @@
|
||||
* Pins both code paths that must respect the v0.23.2 marker:
|
||||
* - extractFactsFromTurn(isDreamGenerated:true) → []
|
||||
* - put_page backstop on dream_generated:true frontmatter → skipped:dream_generated
|
||||
*
|
||||
* The two `put_page` cases drive `dispatchToolCall → put_page`, whose
|
||||
* importFromContent embeds by design (embed failure PROPAGATES — Codex C2).
|
||||
* A sibling shard file that left the gateway configured with a fake/
|
||||
* non-legacy key makes put_page's embed step 401 and throw (isError), which
|
||||
* is what failed these two tests in CI shard 2 while the engine-free
|
||||
* extractFactsFromTurn cases passed. We don't want embedding here at all,
|
||||
* so resetGateway() before each test → empty gateway → embedBatch is a
|
||||
* graceful no-op → put_page succeeds regardless of sibling pollution.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { dispatchToolCall } from '../src/mcp/dispatch.ts';
|
||||
import { extractFactsFromTurn } from '../src/core/facts/extract.ts';
|
||||
import { resetGateway } from '../src/core/ai/gateway.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
// 30s hook timeout — when this file runs deep in a shard process that's
|
||||
// already created ~20 PGLite engines, the WASM cold-start + 95 migrations
|
||||
// on a fresh DB legitimately exceeds bun's 5s hook default. CI shard 4
|
||||
// hit this on v0.41.17.0 (95 migrations × 21 files × 1 bun process).
|
||||
// 60s hook timeout — generous budget for the PGLite WASM cold-start + full
|
||||
// migration chain on a fresh DB, well above bun's 5s hook default.
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
}, 30_000);
|
||||
}, 60_000);
|
||||
|
||||
// Empty gateway → put_page's embed step is a graceful no-op (see header).
|
||||
beforeEach(() => {
|
||||
resetGateway();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
}, 30_000);
|
||||
}, 60_000);
|
||||
|
||||
describe('anti-loop dream_generated marker', () => {
|
||||
test('extractFactsFromTurn skips when isDreamGenerated:true', async () => {
|
||||
|
||||
@@ -8,9 +8,10 @@
|
||||
* responses.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { dispatchToolCall } from '../src/mcp/dispatch.ts';
|
||||
import { resetGateway } from '../src/core/ai/gateway.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
@@ -20,6 +21,18 @@ beforeAll(async () => {
|
||||
await engine.initSchema();
|
||||
});
|
||||
|
||||
// These tests drive put_page, whose importFromContent embeds by design
|
||||
// (embed failure PROPAGATES — a deliberate Codex C2 choice). A sibling
|
||||
// shard file that left the gateway configured with a fake/non-legacy key
|
||||
// would make put_page's embed step 401 and throw (isError). We don't want
|
||||
// embedding here at all — the assertions are about the facts backstop, not
|
||||
// vectors — so reset the gateway before each test. An empty gateway makes
|
||||
// embedBatch a graceful no-op, so put_page succeeds regardless of what a
|
||||
// prior file in the shard process leaked.
|
||||
beforeEach(() => {
|
||||
resetGateway();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
{"family":"title-substring","query":"greek amphitheater","relevant":["projects/example-amphitheater"],"notes":"the direct incident regression: query is a phrase in the title"}
|
||||
{"family":"title-substring","query":"civic feedback platform","relevant":["projects/example-civic-platform"]}
|
||||
{"family":"alias-synonym","query":"hall of light","relevant":["projects/example-amphitheater"],"notes":"declared alias -> canonical"}
|
||||
{"family":"alias-synonym","query":"the widget tracker","relevant":["projects/example-civic-platform"],"notes":"alias romanization/synonym"}
|
||||
{"family":"generic-to-named","query":"indoor amphitheater for debate","relevant":["projects/example-amphitheater"]}
|
||||
{"family":"multi-chunk-dilution","query":"ceiling treatment acoustics amphitheater","relevant":["projects/example-amphitheater"],"notes":"strong chunk among many weak ones"}
|
||||
{"family":"short-vs-rich","query":"amphitheater","relevant":["projects/example-amphitheater"]}
|
||||
{"family":"short-vs-rich","query":"indoor greek amphitheater for adversarial debate in the city","relevant":["projects/example-amphitheater"]}
|
||||
{"family":"graph-relationship","query":"who works on the civic platform","relevant":["people/alice-example"]}
|
||||
{"family":"hard-negative","query":"quarterly tax filing checklist","forbidden":["projects/example-amphitheater","projects/example-civic-platform"],"notes":"must NOT return the named projects"}
|
||||
@@ -196,7 +196,7 @@ describe('op-layer capture — query', () => {
|
||||
describe('op-layer capture — search', () => {
|
||||
const searchOp = operations.find(o => o.name === 'search')!;
|
||||
|
||||
test('captures search call with tool_name="search" and vector_enabled=false', async () => {
|
||||
test('captures search call with tool_name="search" (cheap-hybrid contract, T4/D4)', async () => {
|
||||
const ctx = makeCtx();
|
||||
await searchOp.handler(ctx, { query: 'alice' });
|
||||
await waitForCapture();
|
||||
@@ -205,9 +205,12 @@ describe('op-layer capture — search', () => {
|
||||
expect(rows).toHaveLength(1);
|
||||
const row = rows[0]!;
|
||||
expect(row.tool_name).toBe('search');
|
||||
expect(row.vector_enabled).toBe(false); // search never runs vector
|
||||
expect(row.expand_enabled).toBeNull(); // no expansion concept for search
|
||||
expect(row.detail_resolved).toBeNull();
|
||||
// T4/D4: search is now cheap-hybrid (vector+keyword+RRF). With no embedding
|
||||
// provider in this test it falls open to keyword (vector_enabled=false), but
|
||||
// expansion is structurally OFF for `search` (the cheap-hybrid contract) —
|
||||
// false, not null, because search now HAS an expansion concept.
|
||||
expect(row.vector_enabled).toBe(false); // no embedding provider configured in test
|
||||
expect(row.expand_enabled).toBe(false); // cheap-hybrid: expansion always off
|
||||
});
|
||||
|
||||
test('respects eval.capture=false off-switch', async () => {
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
/**
|
||||
* T6 — NamedThingBench harness unit tests (pure scoring + gate logic).
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import {
|
||||
scoreQuestion,
|
||||
runRetrievalQuality,
|
||||
evaluateGate,
|
||||
parseQuestionsJsonl,
|
||||
type NamedThingQuestion,
|
||||
} from '../src/eval/retrieval-quality/harness.ts';
|
||||
|
||||
describe('scoreQuestion', () => {
|
||||
test('Hit@1 when relevant slug is rank 0', () => {
|
||||
const q: NamedThingQuestion = { family: 'title-substring', query: 'x', relevant: ['a'] };
|
||||
const r = scoreQuestion(q, ['a', 'b', 'c']);
|
||||
expect(r.hit_at_1).toBe(true);
|
||||
expect(r.hit_at_3).toBe(true);
|
||||
expect(r.reciprocal_rank).toBe(1);
|
||||
});
|
||||
test('Hit@3 but not Hit@1 when relevant is rank 2', () => {
|
||||
const q: NamedThingQuestion = { family: 'title-substring', query: 'x', relevant: ['c'] };
|
||||
const r = scoreQuestion(q, ['a', 'b', 'c', 'd']);
|
||||
expect(r.hit_at_1).toBe(false);
|
||||
expect(r.hit_at_3).toBe(true);
|
||||
expect(r.reciprocal_rank).toBeCloseTo(1 / 3, 6);
|
||||
});
|
||||
test('miss when relevant absent from top-3', () => {
|
||||
const q: NamedThingQuestion = { family: 'title-substring', query: 'x', relevant: ['z'] };
|
||||
const r = scoreQuestion(q, ['a', 'b', 'c', 'z']);
|
||||
expect(r.hit_at_3).toBe(false);
|
||||
expect(r.reciprocal_rank).toBeCloseTo(0.25, 6);
|
||||
});
|
||||
test('hard-negative: clean when forbidden slug absent from top-3', () => {
|
||||
const q: NamedThingQuestion = { family: 'hard-negative', query: 'x', forbidden: ['bad'] };
|
||||
expect(scoreQuestion(q, ['a', 'b', 'c']).negative_clean).toBe(true);
|
||||
expect(scoreQuestion(q, ['a', 'bad', 'c']).negative_clean).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('evaluateGate', () => {
|
||||
async function reportFor(qs: NamedThingQuestion[], ranked: Record<string, string[]>) {
|
||||
return runRetrievalQuality(qs, async (query) => ranked[query] ?? []);
|
||||
}
|
||||
|
||||
test('passes when hard families meet floors', async () => {
|
||||
const qs: NamedThingQuestion[] = [
|
||||
{ family: 'title-substring', query: 't', relevant: ['p'] },
|
||||
{ family: 'alias-synonym', query: 'a', relevant: ['p'] },
|
||||
{ family: 'multi-chunk-dilution', query: 'd', relevant: ['p'] },
|
||||
];
|
||||
const report = await reportFor(qs, { t: ['p'], a: ['p'], d: ['p'] });
|
||||
const gate = evaluateGate(report);
|
||||
expect(gate.pass).toBe(true);
|
||||
expect(gate.breaches).toHaveLength(0);
|
||||
});
|
||||
|
||||
test('fails when title-substring Hit@1 below floor', async () => {
|
||||
const qs: NamedThingQuestion[] = [
|
||||
{ family: 'title-substring', query: 't', relevant: ['p'] },
|
||||
];
|
||||
const report = await reportFor(qs, { t: ['other', 'p'] }); // p at rank 1, not 0
|
||||
const gate = evaluateGate(report);
|
||||
expect(gate.pass).toBe(false);
|
||||
expect(gate.breaches[0].family).toBe('title-substring');
|
||||
expect(gate.breaches[0].metric).toBe('hit_at_1');
|
||||
});
|
||||
|
||||
test('soft family low score is a warning, not a breach', async () => {
|
||||
const qs: NamedThingQuestion[] = [
|
||||
{ family: 'graph-relationship', query: 'g', relevant: ['p'] },
|
||||
];
|
||||
const report = await reportFor(qs, { g: ['x', 'y', 'z', 'p'] }); // miss in top-3
|
||||
const gate = evaluateGate(report);
|
||||
expect(gate.pass).toBe(true); // soft → no breach
|
||||
expect(gate.warnings.some(w => w.family === 'graph-relationship')).toBe(true);
|
||||
});
|
||||
|
||||
test('no questions for a hard family → not gated (no false breach)', async () => {
|
||||
const qs: NamedThingQuestion[] = [{ family: 'short-vs-rich', query: 's', relevant: ['p'] }];
|
||||
const report = await reportFor(qs, { s: ['p'] });
|
||||
expect(evaluateGate(report).pass).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('NamedThingBench fixture privacy guard', () => {
|
||||
test('fixture uses placeholder names only (every slug is an *-example)', async () => {
|
||||
const { readFileSync } = await import('fs');
|
||||
const { join } = await import('path');
|
||||
const raw = readFileSync(join(import.meta.dir, 'fixtures/retrieval-quality/namedthing.jsonl'), 'utf8').toLowerCase();
|
||||
// The load-bearing guard: every relevant/forbidden slug must be an
|
||||
// `*-example` placeholder, so no real brain page can be referenced. (We
|
||||
// don't enumerate banned real names here — listing them as string literals
|
||||
// would itself trip scripts/check-test-real-names.sh.)
|
||||
const slugs = [...raw.matchAll(/"(projects|people|notes|companies)\/([a-z0-9-]+)"/g)];
|
||||
expect(slugs.length).toBeGreaterThan(0);
|
||||
for (const m of slugs) {
|
||||
expect(m[2]).toContain('example');
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('parseQuestionsJsonl', () => {
|
||||
test('skips blanks + comments', () => {
|
||||
const qs = parseQuestionsJsonl(`
|
||||
{"family":"title-substring","query":"x","relevant":["a"]}
|
||||
# comment
|
||||
{"family":"hard-negative","query":"y","forbidden":["b"]}
|
||||
`);
|
||||
expect(qs).toHaveLength(2);
|
||||
expect(qs[1].forbidden).toEqual(['b']);
|
||||
});
|
||||
});
|
||||
@@ -659,6 +659,15 @@ test('every CREATE INDEX column in PGLITE_SCHEMA_SQL is covered by CREATE TABLE
|
||||
// ─────────────────────────────────────────────────────────────────
|
||||
|
||||
const COLUMN_EXEMPTIONS = new Set<string>([
|
||||
// T7 — search_telemetry rank-1 drift columns (migration v111). search_telemetry
|
||||
// is created entirely by migration v57 (not in the schema blob), so the v57+v111
|
||||
// chain handles fresh + upgrade; no CREATE INDEX references these columns, so
|
||||
// there's no forward reference for the bootstrap to cover.
|
||||
'search_telemetry.sum_rank1_score',
|
||||
'search_telemetry.count_rank1',
|
||||
'search_telemetry.rank1_lt_solid',
|
||||
'search_telemetry.rank1_solid',
|
||||
'search_telemetry.rank1_high',
|
||||
// Schema-blob-not-yet-refreshed: each of these columns is added by a
|
||||
// migration but NOT (yet) referenced by `PGLITE_SCHEMA_SQL` (neither in a
|
||||
// CREATE TABLE body nor in any CREATE INDEX). Bootstrap doesn't need to
|
||||
|
||||
@@ -90,6 +90,6 @@ describe('alias_resolved boost stage', () => {
|
||||
|
||||
describe('KNOBS_HASH_VERSION', () => {
|
||||
it('bumped to 6 to invalidate caches across v0.42 boost stage addition', () => {
|
||||
expect(KNOBS_HASH_VERSION).toBe(6);
|
||||
expect(KNOBS_HASH_VERSION).toBe(7);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -69,6 +69,7 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => {
|
||||
reranker_top_n_out: null,
|
||||
reranker_timeout_ms: 5000,
|
||||
floor_ratio: undefined,
|
||||
title_boost: 1.25,
|
||||
...CROSS_MODAL_DEFAULTS,
|
||||
graph_signals: false,
|
||||
...CR_DISABLED_DEFAULT,
|
||||
@@ -93,6 +94,7 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => {
|
||||
reranker_top_n_out: null,
|
||||
reranker_timeout_ms: 5000,
|
||||
floor_ratio: undefined,
|
||||
title_boost: 1.25,
|
||||
...CROSS_MODAL_DEFAULTS,
|
||||
graph_signals: true,
|
||||
...CR_DISABLED_DEFAULT,
|
||||
@@ -115,6 +117,7 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => {
|
||||
reranker_top_n_out: null,
|
||||
reranker_timeout_ms: 5000,
|
||||
floor_ratio: undefined,
|
||||
title_boost: 1.25,
|
||||
...CROSS_MODAL_DEFAULTS,
|
||||
graph_signals: true,
|
||||
...CR_DISABLED_DEFAULT,
|
||||
@@ -376,7 +379,7 @@ describe('knobsHash determinism + cross-mode separation (CDX-4)', () => {
|
||||
// post-fusion boost stage. A query against a brain with slug_aliases
|
||||
// populated must not be served from a cache row written before the
|
||||
// boost stage existed.
|
||||
expect(KNOBS_HASH_VERSION).toBe(6);
|
||||
expect(KNOBS_HASH_VERSION).toBe(7);
|
||||
});
|
||||
|
||||
test('T1 (codex): floor_ratio set vs unset produces DIFFERENT hashes (cache contamination prevention)', () => {
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
/**
|
||||
* T3 — alias hop + ingest projection (the named-thing fix).
|
||||
* Hermetic PGLite. applyAliasHop is tested directly (no embedding provider
|
||||
* needed); ingest projection is tested via importFromContent(noEmbed).
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
||||
import { resetPgliteState } from '../helpers/reset-pglite.ts';
|
||||
import { applyAliasHop } from '../../src/core/search/hybrid.ts';
|
||||
import { importFromContent } from '../../src/core/import-file.ts';
|
||||
import type { SearchResult } from '../../src/core/types.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
afterAll(async () => { await engine.disconnect(); });
|
||||
beforeEach(async () => { await resetPgliteState(engine); });
|
||||
|
||||
function res(slug: string, score: number, title = slug): SearchResult {
|
||||
return { slug, title, score, chunk_text: '', type: 'note', source_id: 'default', chunk_index: 0, chunk_id: 1 } as unknown as SearchResult;
|
||||
}
|
||||
|
||||
describe('applyAliasHop', () => {
|
||||
test('injects an absent canonical when the query matches its alias', async () => {
|
||||
await engine.putPage('projects/mingtang', { type: 'note', title: 'The Mingtang', compiled_truth: 'Indoor Greek amphitheater.' });
|
||||
await engine.setPageAliases('projects/mingtang', 'default', ['hall of light', '明堂']);
|
||||
|
||||
const organic = [res('notes/unrelated', 0.5)];
|
||||
const out = await applyAliasHop(engine, organic, 'Hall of Light', { sourceId: 'default' });
|
||||
|
||||
const hit = out.find(r => r.slug === 'projects/mingtang');
|
||||
expect(hit).toBeDefined();
|
||||
expect(hit!.alias_hit).toBe(true);
|
||||
expect(out[0].slug).toBe('projects/mingtang'); // injected at top-of-organic + ε
|
||||
expect(hit!.score).toBeGreaterThan(0.5);
|
||||
});
|
||||
|
||||
test('romanization/CJK alias also resolves', async () => {
|
||||
await engine.putPage('projects/mingtang', { type: 'note', title: 'The Mingtang', compiled_truth: 'x' });
|
||||
await engine.setPageAliases('projects/mingtang', 'default', ['明堂']);
|
||||
const out = await applyAliasHop(engine, [], '明堂', { sourceId: 'default' });
|
||||
expect(out.map(r => r.slug)).toContain('projects/mingtang');
|
||||
});
|
||||
|
||||
test('boosts (does not duplicate) a canonical already in results', async () => {
|
||||
await engine.putPage('projects/mingtang', { type: 'note', title: 'The Mingtang', compiled_truth: 'x' });
|
||||
await engine.setPageAliases('projects/mingtang', 'default', ['hall of light']);
|
||||
const organic = [res('projects/mingtang', 0.4), res('notes/other', 0.5)];
|
||||
const out = await applyAliasHop(engine, organic, 'hall of light', { sourceId: 'default' });
|
||||
expect(out.filter(r => r.slug === 'projects/mingtang').length).toBe(1); // no dup
|
||||
const m = out.find(r => r.slug === 'projects/mingtang')!;
|
||||
expect(m.alias_hit).toBe(true);
|
||||
expect(m.score).toBeCloseTo(0.4 * 1.10, 6); // bounded present-boost
|
||||
});
|
||||
|
||||
test('P0 source-isolation: alias hop boosts only the aliased source, not the same slug in another source', async () => {
|
||||
// The alias belongs to the src-b page only. Two same-slug results, different
|
||||
// sources, both in the organic set. The hop must boost ONLY src-b's row.
|
||||
await engine.setPageAliases('shared/page', 'src-b', ['only in b']);
|
||||
const organic = [
|
||||
{ slug: 'shared/page', source_id: 'src-a', score: 0.5 } as unknown as SearchResult,
|
||||
{ slug: 'shared/page', source_id: 'src-b', score: 0.5 } as unknown as SearchResult,
|
||||
];
|
||||
const out = await applyAliasHop(engine, organic, 'only in b', { sourceIds: ['src-a', 'src-b'] });
|
||||
const a = out.find(r => r.source_id === 'src-a')!;
|
||||
const b = out.find(r => r.source_id === 'src-b')!;
|
||||
expect(b.alias_hit).toBe(true);
|
||||
expect(b.score).toBeCloseTo(0.5 * 1.10, 6);
|
||||
expect(a.alias_hit).toBeUndefined(); // NOT cross-boosted
|
||||
expect(a.score).toBe(0.5);
|
||||
});
|
||||
|
||||
test('no alias match → input unchanged', async () => {
|
||||
const organic = [res('a', 0.9), res('b', 0.8)];
|
||||
const out = await applyAliasHop(engine, organic, 'some unrelated query', { sourceId: 'default' });
|
||||
expect(out.map(r => r.slug)).toEqual(['a', 'b']);
|
||||
});
|
||||
|
||||
test('long query is skipped (clearly prose, not a chosen name)', async () => {
|
||||
await engine.putPage('p/x', { type: 'note', title: 'X', compiled_truth: 'x' });
|
||||
await engine.setPageAliases('p/x', 'default', ['one two three four five six seven']);
|
||||
const out = await applyAliasHop(engine, [], 'one two three four five six seven', { sourceId: 'default' });
|
||||
expect(out.length).toBe(0); // >6 tokens → skipped
|
||||
});
|
||||
|
||||
test('collision: two pages claim one alias → both injected, capped + deterministic', async () => {
|
||||
await engine.putPage('a/hall', { type: 'note', title: 'A Hall', compiled_truth: 'x' });
|
||||
await engine.putPage('b/hall', { type: 'note', title: 'B Hall', compiled_truth: 'x' });
|
||||
await engine.setPageAliases('a/hall', 'default', ['the hall']);
|
||||
await engine.setPageAliases('b/hall', 'default', ['the hall']);
|
||||
const out = await applyAliasHop(engine, [], 'the hall', { sourceId: 'default' });
|
||||
expect(out.map(r => r.slug).sort()).toEqual(['a/hall', 'b/hall']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('ingest projection (importFromContent)', () => {
|
||||
test('frontmatter aliases: land in page_aliases on import', async () => {
|
||||
const md = `---\ntype: note\ntitle: The Mingtang\naliases:\n - Hall of Light\n - 明堂\n---\nIndoor Greek amphitheater.`;
|
||||
await importFromContent(engine, 'projects/mingtang', md, { sourceId: 'default', noEmbed: true });
|
||||
const m = await engine.resolveAliases(['hall of light', '明堂'], { sourceId: 'default' });
|
||||
expect((m.get('hall of light') ?? []).map(r => r.slug)).toEqual(['projects/mingtang']);
|
||||
expect((m.get('明堂') ?? []).map(r => r.slug)).toEqual(['projects/mingtang']);
|
||||
});
|
||||
|
||||
test('removing an alias from frontmatter clears it on re-import', async () => {
|
||||
const v1 = `---\ntype: note\ntitle: X\naliases: [old name, keep name]\n---\nbody one`;
|
||||
await importFromContent(engine, 'p/x', v1, { sourceId: 'default', noEmbed: true });
|
||||
const v2 = `---\ntype: note\ntitle: X\naliases: [keep name]\n---\nbody two changed`;
|
||||
await importFromContent(engine, 'p/x', v2, { sourceId: 'default', noEmbed: true });
|
||||
const m = await engine.resolveAliases(['old name', 'keep name'], { sourceId: 'default' });
|
||||
expect(m.get('old name')).toBeUndefined();
|
||||
expect((m.get('keep name') ?? []).map(r => r.slug)).toEqual(['p/x']);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,50 @@
|
||||
/**
|
||||
* T3 — alias normalization unit tests. The write path (ingest) and read path
|
||||
* (search) MUST normalize identically or stored aliases silently never match.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { normalizeAlias, normalizeAliasList } from '../../src/core/search/alias-normalize.ts';
|
||||
|
||||
describe('normalizeAlias', () => {
|
||||
test('lowercases + trims + collapses whitespace', () => {
|
||||
expect(normalizeAlias(' Hall of Light ')).toBe('hall of light');
|
||||
});
|
||||
test('write and read of the same name converge', () => {
|
||||
expect(normalizeAlias('The Hall Of Light')).toBe(normalizeAlias('the hall of light'));
|
||||
});
|
||||
test('NFKC folds width variants', () => {
|
||||
expect(normalizeAlias('Ｍｉｎｇｔａｎｇ')).toBe('mingtang'); // fullwidth → ascii
|
||||
});
|
||||
test('keeps CJK', () => {
|
||||
expect(normalizeAlias(' 明堂 ')).toBe('明堂');
|
||||
});
|
||||
test('strips a layer of wrapping quotes/brackets from loose YAML', () => {
|
||||
expect(normalizeAlias('"Hall of Light"')).toBe('hall of light');
|
||||
expect(normalizeAlias('[Mingtang]')).toBe('mingtang');
|
||||
});
|
||||
test('empty / non-string → empty', () => {
|
||||
expect(normalizeAlias('')).toBe('');
|
||||
expect(normalizeAlias(' ')).toBe('');
|
||||
expect(normalizeAlias(undefined as unknown as string)).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('normalizeAliasList', () => {
|
||||
test('array → normalized + deduped, empties dropped', () => {
|
||||
expect(normalizeAliasList(['Hall of Light', 'hall of light', '', ' ', 'Mingtang']))
|
||||
.toEqual(['hall of light', 'mingtang']);
|
||||
});
|
||||
test('scalar string → single-element list', () => {
|
||||
expect(normalizeAliasList('Hall of Light')).toEqual(['hall of light']);
|
||||
});
|
||||
test('comma-separated scalar → split list', () => {
|
||||
expect(normalizeAliasList('Hall of Light, Mingtang, 明堂'))
|
||||
.toEqual(['hall of light', 'mingtang', '明堂']);
|
||||
});
|
||||
test('absent / non-string → empty list', () => {
|
||||
expect(normalizeAliasList(undefined)).toEqual([]);
|
||||
expect(normalizeAliasList(null)).toEqual([]);
|
||||
expect(normalizeAliasList(42)).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,75 @@
|
||||
/**
|
||||
* T4 — evidence + create_safety contract. This is the agent-facing signal that
|
||||
* prevents the incident's duplicate-stub class: the don't-write decision keys
|
||||
* off WHY a page matched, not a raw blended score.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { classifyEvidence, createSafetyFor, stampEvidence, HIGH_MATCH_FLOOR, SOLID_MATCH_FLOOR } from '../../src/core/search/evidence.ts';
|
||||
import type { SearchResult } from '../../src/core/types.ts';
|
||||
|
||||
function r(partial: Partial<SearchResult>): SearchResult {
|
||||
return { slug: 's', title: 't', chunk_text: '', type: 'note', source_id: 'default', chunk_index: 0, chunk_id: 1, score: 0.5, ...partial } as SearchResult;
|
||||
}
|
||||
|
||||
describe('classifyEvidence precedence', () => {
|
||||
test('alias_hit wins over everything', () => {
|
||||
expect(classifyEvidence(r({ alias_hit: true, title_match_boost: 1.25, base_score: 0.99 }))).toBe('alias_hit');
|
||||
});
|
||||
test('exact_title_match when title boost fired', () => {
|
||||
expect(classifyEvidence(r({ title_match_boost: 1.25, base_score: 0.2 }))).toBe('exact_title_match');
|
||||
});
|
||||
test('high_vector_match at/above HIGH_MATCH_FLOOR', () => {
|
||||
expect(classifyEvidence(r({ base_score: HIGH_MATCH_FLOOR }))).toBe('high_vector_match');
|
||||
expect(classifyEvidence(r({ base_score: 0.90 }))).toBe('high_vector_match');
|
||||
});
|
||||
test('keyword_exact in the solid band', () => {
|
||||
expect(classifyEvidence(r({ base_score: SOLID_MATCH_FLOOR }))).toBe('keyword_exact');
|
||||
expect(classifyEvidence(r({ base_score: 0.7 }))).toBe('keyword_exact');
|
||||
});
|
||||
test('weak_semantic below the solid floor (the incident: 0.64 body chunk... 0.5 here)', () => {
|
||||
expect(classifyEvidence(r({ base_score: 0.4 }))).toBe('weak_semantic');
|
||||
});
|
||||
test('falls back to score when base_score absent', () => {
|
||||
expect(classifyEvidence(r({ score: 0.95, base_score: undefined }))).toBe('high_vector_match');
|
||||
});
|
||||
});
|
||||
|
||||
describe('createSafetyFor', () => {
|
||||
test('strong evidence → exists (do NOT duplicate)', () => {
|
||||
expect(createSafetyFor('alias_hit')).toBe('exists');
|
||||
expect(createSafetyFor('exact_title_match')).toBe('exists');
|
||||
expect(createSafetyFor('high_vector_match')).toBe('exists');
|
||||
});
|
||||
test('keyword_exact → probable', () => {
|
||||
expect(createSafetyFor('keyword_exact')).toBe('probable');
|
||||
});
|
||||
test('weak_semantic → unknown', () => {
|
||||
expect(createSafetyFor('weak_semantic')).toBe('unknown');
|
||||
});
|
||||
});
|
||||
|
||||
describe('stampEvidence', () => {
|
||||
test('stamps both fields on every result', () => {
|
||||
const rs = [r({ alias_hit: true }), r({ base_score: 0.3 })];
|
||||
stampEvidence(rs);
|
||||
expect(rs[0].evidence).toBe('alias_hit');
|
||||
expect(rs[0].create_safety).toBe('exists');
|
||||
expect(rs[1].evidence).toBe('weak_semantic');
|
||||
expect(rs[1].create_safety).toBe('unknown');
|
||||
});
|
||||
|
||||
test('the incident: a 0.64 body-chunk match reads as weak/unknown (agent should look closer, not blindly duplicate)', () => {
|
||||
const incident = r({ base_score: 0.64, score: 0.64 });
|
||||
stampEvidence([incident]);
|
||||
expect(incident.evidence).toBe('keyword_exact'); // 0.64 is in the solid band
|
||||
expect(incident.create_safety).toBe('probable'); // not 'unknown' — prefer update over create
|
||||
});
|
||||
|
||||
test('after the fix: the same page via alias/title reads as exists (do not duplicate)', () => {
|
||||
const fixed = r({ base_score: 0.64, alias_hit: true });
|
||||
stampEvidence([fixed]);
|
||||
expect(fixed.evidence).toBe('alias_hit');
|
||||
expect(fixed.create_safety).toBe('exists');
|
||||
});
|
||||
});
|
||||
@@ -55,7 +55,7 @@ describe('KNOBS_HASH_VERSION + version invariants', () => {
|
||||
// v0.41.22.0 (type-unification): 5→6 to fold the alias_resolved
|
||||
// post-fusion boost. Cache rows written before the boost stage
|
||||
// cannot leak past the new stage.
|
||||
expect(KNOBS_HASH_VERSION).toBe(6);
|
||||
expect(KNOBS_HASH_VERSION).toBe(7);
|
||||
});
|
||||
|
||||
test('hash is 16 hex chars regardless of reranker config', () => {
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
/**
|
||||
* T3 — page_aliases engine round-trip (resolveAliases + setPageAliases).
|
||||
* Hermetic PGLite. Pins: write→read, collision (two slugs one alias),
|
||||
* source-scoping, replace-on-rewrite, empty-clears, empty-input.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
||||
import { resetPgliteState } from '../helpers/reset-pglite.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetPgliteState(engine);
|
||||
});
|
||||
|
||||
const slugsOf = (m: Map<string, Array<{ slug: string; source_id: string }>>, k: string) =>
|
||||
(m.get(k) ?? []).map(r => r.slug).sort();
|
||||
|
||||
describe('setPageAliases + resolveAliases', () => {
|
||||
test('write then read maps alias_norm → slug', async () => {
|
||||
await engine.setPageAliases('projects/mingtang', 'default', ['hall of light', '明堂']);
|
||||
const m = await engine.resolveAliases(['hall of light'], { sourceId: 'default' });
|
||||
expect(slugsOf(m, 'hall of light')).toEqual(['projects/mingtang']);
|
||||
expect(m.get('hall of light')![0].source_id).toBe('default');
|
||||
});
|
||||
|
||||
test('collision: two pages claim the same alias → both returned', async () => {
|
||||
await engine.setPageAliases('projects/mingtang', 'default', ['the hall']);
|
||||
await engine.setPageAliases('projects/other-hall', 'default', ['the hall']);
|
||||
const m = await engine.resolveAliases(['the hall'], { sourceId: 'default' });
|
||||
expect(slugsOf(m, 'the hall')).toEqual(['projects/mingtang', 'projects/other-hall']);
|
||||
});
|
||||
|
||||
test('source-scoped: alias in source A not returned for source B', async () => {
|
||||
await engine.setPageAliases('a/page', 'src-a', ['shared name']);
|
||||
await engine.setPageAliases('b/page', 'src-b', ['shared name']);
|
||||
const aOnly = await engine.resolveAliases(['shared name'], { sourceId: 'src-a' });
|
||||
expect(slugsOf(aOnly, 'shared name')).toEqual(['a/page']);
|
||||
expect(aOnly.get('shared name')![0].source_id).toBe('src-a');
|
||||
const both = await engine.resolveAliases(['shared name'], { sourceIds: ['src-a', 'src-b'] });
|
||||
expect(slugsOf(both, 'shared name')).toEqual(['a/page', 'b/page']);
|
||||
// same slug across sources stays distinct (the P0 contract)
|
||||
expect((both.get('shared name') ?? []).map(r => r.source_id).sort()).toEqual(['src-a', 'src-b']);
|
||||
});
|
||||
|
||||
test('rewrite replaces the prior alias set (delete + insert)', async () => {
|
||||
await engine.setPageAliases('p/x', 'default', ['old name']);
|
||||
await engine.setPageAliases('p/x', 'default', ['new name']);
|
||||
const oldM = await engine.resolveAliases(['old name'], { sourceId: 'default' });
|
||||
const newM = await engine.resolveAliases(['new name'], { sourceId: 'default' });
|
||||
expect(oldM.get('old name')).toBeUndefined();
|
||||
expect(slugsOf(newM, 'new name')).toEqual(['p/x']);
|
||||
});
|
||||
|
||||
test('empty alias set clears the page', async () => {
|
||||
await engine.setPageAliases('p/x', 'default', ['temp']);
|
||||
await engine.setPageAliases('p/x', 'default', []);
|
||||
const m = await engine.resolveAliases(['temp'], { sourceId: 'default' });
|
||||
expect(m.size).toBe(0);
|
||||
});
|
||||
|
||||
test('empty input → empty map, no query', async () => {
|
||||
const m = await engine.resolveAliases([], { sourceId: 'default' });
|
||||
expect(m.size).toBe(0);
|
||||
});
|
||||
|
||||
test('idempotent re-write does not duplicate (unique triple)', async () => {
|
||||
await engine.setPageAliases('p/x', 'default', ['name', 'name']);
|
||||
const m = await engine.resolveAliases(['name'], { sourceId: 'default' });
|
||||
expect(slugsOf(m, 'name')).toEqual(['p/x']);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,33 @@
|
||||
/**
|
||||
* T4/D5 — per-call mode gate: local/trusted callers may select a mode; remote
|
||||
* callers can't (no tokenmax cost escalation); unknown modes reject loudly.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { resolvePerCallMode, OperationError } from '../../src/core/operations.ts';
|
||||
import type { OperationContext } from '../../src/core/operations.ts';
|
||||
|
||||
function ctx(remote: boolean | undefined): OperationContext {
|
||||
return { remote } as unknown as OperationContext;
|
||||
}
|
||||
|
||||
describe('resolvePerCallMode', () => {
|
||||
test('local caller (remote===false) gets a valid mode', () => {
|
||||
expect(resolvePerCallMode(ctx(false), 'tokenmax')).toBe('tokenmax');
|
||||
expect(resolvePerCallMode(ctx(false), 'conservative')).toBe('conservative');
|
||||
});
|
||||
test('local caller + unknown mode → loud reject', () => {
|
||||
expect(() => resolvePerCallMode(ctx(false), 'thorough')).toThrow(OperationError);
|
||||
expect(() => resolvePerCallMode(ctx(false), 'NUKE')).toThrow(/Unknown search mode/);
|
||||
});
|
||||
test('remote caller (remote===true) → mode ignored (no escalation)', () => {
|
||||
expect(resolvePerCallMode(ctx(true), 'tokenmax')).toBeUndefined();
|
||||
});
|
||||
test('remote caller (remote===undefined, treated as remote) → ignored', () => {
|
||||
expect(resolvePerCallMode(ctx(undefined), 'tokenmax')).toBeUndefined();
|
||||
});
|
||||
test('no mode passed → undefined (use configured)', () => {
|
||||
expect(resolvePerCallMode(ctx(false), undefined)).toBeUndefined();
|
||||
expect(resolvePerCallMode(ctx(false), '')).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,56 @@
|
||||
/**
|
||||
* T9 regression — pre-v110 brains (no page_aliases table) must keep working.
|
||||
* The alias layer is additive: every alias touchpoint fails open so a brain
|
||||
* mid-upgrade (migration not yet run) still ingests and searches normally.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
||||
import { resetPgliteState } from '../helpers/reset-pglite.ts';
|
||||
import { applyAliasHop } from '../../src/core/search/hybrid.ts';
|
||||
import { importFromContent } from '../../src/core/import-file.ts';
|
||||
import type { SearchResult } from '../../src/core/types.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
afterAll(async () => { await engine.disconnect(); });
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetPgliteState(engine);
|
||||
// Simulate a pre-v110 brain: drop the page_aliases table entirely.
|
||||
await engine.executeRaw('DROP TABLE IF EXISTS page_aliases');
|
||||
});
|
||||
|
||||
function r(slug: string, score: number): SearchResult {
|
||||
return { slug, title: slug, score, chunk_text: '', type: 'note', source_id: 'default', chunk_index: 0, chunk_id: 1 } as unknown as SearchResult;
|
||||
}
|
||||
|
||||
describe('pre-migration (no page_aliases table) fail-open', () => {
|
||||
test('applyAliasHop returns input unchanged, does not throw', async () => {
|
||||
const organic = [r('a', 0.9), r('b', 0.8)];
|
||||
const out = await applyAliasHop(engine, organic, 'hall of light', { sourceId: 'default' });
|
||||
expect(out.map(x => x.slug)).toEqual(['a', 'b']);
|
||||
});
|
||||
|
||||
test('importFromContent with frontmatter aliases does not throw (projection swallows table-missing)', async () => {
|
||||
const md = `---\ntype: note\ntitle: X\naliases: [Hall of Light]\n---\nbody`;
|
||||
const res = await importFromContent(engine, 'p/x', md, { sourceId: 'default', noEmbed: true });
|
||||
expect(res.status).toBe('imported'); // page write succeeded despite no alias table
|
||||
});
|
||||
|
||||
test('resolveAliases throws table-missing (caller is responsible for catching)', async () => {
|
||||
// The engine method itself surfaces the error; the alias-hop caller wraps it.
|
||||
let threw = false;
|
||||
try {
|
||||
await engine.resolveAliases(['x'], { sourceId: 'default' });
|
||||
} catch {
|
||||
threw = true;
|
||||
}
|
||||
expect(threw).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,65 @@
|
||||
/**
|
||||
* T8 — `gbrain reindex --aliases` backfill. Seeds pages with frontmatter
|
||||
* aliases (as if imported before the T3 projection landed), runs the backfill,
|
||||
* and verifies page_aliases is populated. Idempotent + --dry-run + --source.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
||||
import { resetPgliteState } from '../helpers/reset-pglite.ts';
|
||||
import { runReindexAliases } from '../../src/commands/reindex-aliases.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
afterAll(async () => { await engine.disconnect(); });
|
||||
beforeEach(async () => { await resetPgliteState(engine); });
|
||||
|
||||
async function seed(slug: string, aliases: unknown) {
|
||||
await engine.putPage(slug, { type: 'note' as never, title: slug, compiled_truth: 'x', frontmatter: { aliases } });
|
||||
}
|
||||
|
||||
describe('runReindexAliases', () => {
|
||||
test('backfills page_aliases from frontmatter for existing pages', async () => {
|
||||
await seed('projects/mingtang', ['Hall of Light', '明堂']);
|
||||
await seed('notes/plain', undefined); // no aliases
|
||||
|
||||
const result = await runReindexAliases(engine, ['--json']);
|
||||
expect(result.pages_with_aliases).toBe(1);
|
||||
expect(result.aliases_written).toBe(2);
|
||||
|
||||
const m = await engine.resolveAliases(['hall of light', '明堂'], { sourceId: 'default' });
|
||||
expect((m.get('hall of light') ?? []).map(r => r.slug)).toEqual(['projects/mingtang']);
|
||||
expect((m.get('明堂') ?? []).map(r => r.slug)).toEqual(['projects/mingtang']);
|
||||
});
|
||||
|
||||
test('--dry-run writes nothing', async () => {
|
||||
await seed('p/x', ['some alias']);
|
||||
const result = await runReindexAliases(engine, ['--dry-run', '--json']);
|
||||
expect(result.dry_run).toBe(true);
|
||||
expect(result.aliases_written).toBe(1); // would-write count
|
||||
const m = await engine.resolveAliases(['some alias'], { sourceId: 'default' });
|
||||
expect(m.size).toBe(0); // nothing actually written
|
||||
});
|
||||
|
||||
test('idempotent: second run converges, no duplicates', async () => {
|
||||
await seed('p/x', ['name one', 'name two']);
|
||||
await runReindexAliases(engine, ['--json']);
|
||||
await runReindexAliases(engine, ['--json']);
|
||||
const m = await engine.resolveAliases(['name one'], { sourceId: 'default' });
|
||||
expect((m.get('name one') ?? []).map(r => r.slug)).toEqual(['p/x']);
|
||||
});
|
||||
|
||||
test('handles comma-scalar frontmatter aliases', async () => {
|
||||
await seed('p/y', 'Alpha, Beta');
|
||||
const result = await runReindexAliases(engine, ['--json']);
|
||||
expect(result.aliases_written).toBe(2);
|
||||
const m = await engine.resolveAliases(['alpha', 'beta'], { sourceId: 'default' });
|
||||
expect((m.get('alpha') ?? []).map(r => r.slug)).toEqual(['p/y']);
|
||||
expect((m.get('beta') ?? []).map(r => r.slug)).toEqual(['p/y']);
|
||||
});
|
||||
});
|
||||
@@ -11,7 +11,7 @@
|
||||
* - rerankerFn test seam used over gateway.rerank
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll } from 'bun:test';
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { applyReranker, type RerankerOpts } from '../../src/core/search/rerank.ts';
|
||||
import { RerankError, type RerankResult } from '../../src/core/ai/gateway.ts';
|
||||
import type { SearchResult } from '../../src/core/types.ts';
|
||||
@@ -33,6 +33,13 @@ function makeResult(slug: string, score: number, chunk: string): SearchResult {
|
||||
|
||||
// Setup: gateway must be configured so the rerank-audit logger doesn't
|
||||
// trip on missing env. We can call configureGateway with a minimal stub.
|
||||
// NOTE: this stub omits embedding_model, so the gateway falls back to the
|
||||
// v0.37 default (zeroentropyai:zembed-1 / 1280-d). Without the afterAll
|
||||
// reset below it would LEAK that default to the next file in the shard
|
||||
// process — a sibling that runs initSchema in beforeAll would build a
|
||||
// vector(1280) column and then mismatch on 1536-d fixtures. resetGateway
|
||||
// in afterAll restores the empty slot so the legacy-embedding preload
|
||||
// re-pins OpenAI/1536 for the next file.
|
||||
beforeAll(async () => {
|
||||
const { configureGateway } = await import('../../src/core/ai/gateway.ts');
|
||||
configureGateway({
|
||||
@@ -40,6 +47,11 @@ beforeAll(async () => {
|
||||
});
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
const { resetGateway } = await import('../../src/core/ai/gateway.ts');
|
||||
resetGateway();
|
||||
});
|
||||
|
||||
describe('applyReranker — happy path', () => {
|
||||
test('reorders top-N by reranker relevance score', async () => {
|
||||
const results = [
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
/**
|
||||
* T0 — `gbrain search diagnose` Phase-0 retrieval diagnostic. Seeds a synthetic
|
||||
* brain, forces the keyword+alias path (embed stubbed to throw → vector skipped),
|
||||
* and asserts the per-layer trace + verdict.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
||||
import { __setEmbedTransportForTests } from '../../src/core/ai/gateway.ts';
|
||||
import { runSearchDiagnose } from '../../src/commands/search-diagnose.ts';
|
||||
import type { ChunkInput } from '../../src/core/types.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
async function captureJson(fn: () => Promise<void>): Promise<any> {
|
||||
const orig = console.log;
|
||||
const lines: string[] = [];
|
||||
console.log = (...a: unknown[]) => { lines.push(a.map(String).join(' ')); };
|
||||
try { await fn(); } finally { console.log = orig; }
|
||||
return JSON.parse(lines.join('\n'));
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
__setEmbedTransportForTests(() => { throw new Error('stub: no embed'); });
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
await engine.putPage('projects/mingtang', {
|
||||
type: 'note' as never,
|
||||
title: 'The Example Hall — Indoor Greek Amphitheater',
|
||||
compiled_truth: 'Indoor greek amphitheater for adversarial debate.',
|
||||
});
|
||||
const ci: ChunkInput[] = [{ chunk_index: 0, chunk_text: 'Indoor greek amphitheater for adversarial debate.', chunk_source: 'compiled_truth', token_count: 10 }];
|
||||
await engine.upsertChunks('projects/mingtang', ci);
|
||||
await engine.setPageAliases('projects/mingtang', 'default', ['hall of light']);
|
||||
});
|
||||
|
||||
afterAll(async () => { __setEmbedTransportForTests(null); await engine.disconnect(); });
|
||||
|
||||
describe('search diagnose', () => {
|
||||
test('alias query: trace shows alias match + hybrid rank 1', async () => {
|
||||
const report = await captureJson(() =>
|
||||
runSearchDiagnose(engine, ['diagnose', 'hall of light', '--target', 'projects/mingtang', '--source', 'default', '--json']),
|
||||
);
|
||||
expect(report.target).toBe('projects/mingtang');
|
||||
expect(report.alias_match).toBe(true);
|
||||
// vector probe degrades gracefully (no provider OR embed call failed) — the
|
||||
// point is it doesn't crash the diagnose. A note is present either way.
|
||||
expect(report.vector.rank).toBeNull();
|
||||
expect(typeof report.vector.note).toBe('string');
|
||||
expect(report.hybrid.rank).toBe(1);
|
||||
expect(report.hybrid.alias_hit).toBe(true);
|
||||
expect(report.verdict).toContain('rank 1');
|
||||
});
|
||||
|
||||
test('title-phrase query: keyword + title boost surface the target', async () => {
|
||||
const report = await captureJson(() =>
|
||||
runSearchDiagnose(engine, ['diagnose', 'greek amphitheater', '--target', 'projects/mingtang', '--source', 'default', '--json']),
|
||||
);
|
||||
expect(report.keyword.rank).toBe(1);
|
||||
expect(report.hybrid.rank).toBe(1);
|
||||
// title boost fired (query is a phrase in the title)
|
||||
expect(report.hybrid.title_match_boost).toBeGreaterThan(1.0);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,130 @@
|
||||
/**
|
||||
* T1 regression — searchVector per-page max-pool (retrieval-maxpool incident).
|
||||
*
|
||||
* The bug: `searchVector` returned chunk-grain top-k with NO `DISTINCT ON (slug)`.
|
||||
* A hub page with several strong chunks could occupy every early candidate slot,
|
||||
* crowding a different page's single strong chunk out of the result entirely, OR
|
||||
* returning the same page multiple times. The keyword path always pooled per page
|
||||
* via `best_per_page`; the vector path did not. This pins the fix.
|
||||
*
|
||||
* Failing-on-old-code construction:
|
||||
* - query embeds at basis direction 0.
|
||||
* - `notes/hub` has THREE chunks at cosine 0.99 / 0.98 / 0.97.
|
||||
* - `notes/needle` has ONE chunk at cosine 0.95.
|
||||
* - two fillers at 0.90.
|
||||
* With limit=3 the OLD chunk-grain path returns [hub@0.99, hub@0.98, hub@0.97]
|
||||
* — needle (rank 4 at chunk grain) is truncated and the page is absent, and hub
|
||||
* appears 3×. The pooled path returns 3 DISTINCT pages [hub, needle, filler],
|
||||
* so needle surfaces and hub appears once.
|
||||
*
|
||||
* `detail: 'high'` neutralizes the source-factor boost (buildSourceFactorCase
|
||||
* returns 1.0) so the assertion keys off pure cosine, not slug-prefix weighting.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
||||
import { configureGateway } from '../../src/core/ai/gateway.ts';
|
||||
import type { ChunkInput } from '../../src/core/types.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
const DIM = 1536;
|
||||
|
||||
/** Unit vector with cosine-similarity `cos` against basis direction 0. */
|
||||
function gradedEmb(cos: number, otherDim: number): Float32Array {
|
||||
const e = new Float32Array(DIM);
|
||||
e[0] = cos;
|
||||
e[otherDim] = Math.sqrt(Math.max(0, 1 - cos * cos));
|
||||
return e;
|
||||
}
|
||||
|
||||
function basisEmbedding(idx: number): Float32Array {
|
||||
const e = new Float32Array(DIM);
|
||||
e[idx % DIM] = 1.0;
|
||||
return e;
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
// Pin the legacy OpenAI/1536 gateway BEFORE initSchema so the
|
||||
// content_chunks.embedding column is vector(1536), matching this file's
|
||||
// hardcoded 1536-d basis vectors. initSchema runs in beforeAll (before
|
||||
// any preload beforeEach can re-pin), so we cannot rely on the legacy
|
||||
// preload default surviving a sibling shard file that reconfigured the
|
||||
// gateway to the v0.37 ZE/1280 default and didn't reset.
|
||||
configureGateway({
|
||||
embedding_model: 'openai:text-embedding-3-large',
|
||||
embedding_dimensions: DIM,
|
||||
env: { ...process.env },
|
||||
});
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({}); // in-memory
|
||||
await engine.initSchema();
|
||||
|
||||
// Seeded first → lowest page_id. Three strong chunks.
|
||||
await engine.putPage('notes/hub', {
|
||||
type: 'note',
|
||||
title: 'Hub Page',
|
||||
compiled_truth: 'hub a. hub b. hub c.',
|
||||
});
|
||||
const hubChunks: ChunkInput[] = [
|
||||
{ chunk_index: 0, chunk_text: 'hub a', chunk_source: 'compiled_truth', embedding: gradedEmb(0.99, 10), token_count: 2 },
|
||||
{ chunk_index: 1, chunk_text: 'hub b', chunk_source: 'compiled_truth', embedding: gradedEmb(0.98, 11), token_count: 2 },
|
||||
{ chunk_index: 2, chunk_text: 'hub c', chunk_source: 'compiled_truth', embedding: gradedEmb(0.97, 12), token_count: 2 },
|
||||
];
|
||||
await engine.upsertChunks('notes/hub', hubChunks);
|
||||
|
||||
// The page that MUST surface — one strong chunk, below hub's chunks but above fillers.
|
||||
await engine.putPage('notes/needle', {
|
||||
type: 'note',
|
||||
title: 'Needle Page',
|
||||
compiled_truth: 'the needle',
|
||||
});
|
||||
await engine.upsertChunks('notes/needle', [
|
||||
{ chunk_index: 0, chunk_text: 'the needle', chunk_source: 'compiled_truth', embedding: gradedEmb(0.95, 13), token_count: 2 },
|
||||
]);
|
||||
|
||||
await engine.putPage('notes/filler-a', { type: 'note', title: 'Filler A', compiled_truth: 'filler a' });
|
||||
await engine.upsertChunks('notes/filler-a', [
|
||||
{ chunk_index: 0, chunk_text: 'filler a', chunk_source: 'compiled_truth', embedding: gradedEmb(0.90, 14), token_count: 2 },
|
||||
]);
|
||||
await engine.putPage('notes/filler-b', { type: 'note', title: 'Filler B', compiled_truth: 'filler b' });
|
||||
await engine.upsertChunks('notes/filler-b', [
|
||||
{ chunk_index: 0, chunk_text: 'filler b', chunk_source: 'compiled_truth', embedding: gradedEmb(0.90, 15), token_count: 2 },
|
||||
]);
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
describe('searchVector per-page max-pool (T1)', () => {
|
||||
test('returns DISTINCT pages — no page appears more than once', async () => {
|
||||
const results = await engine.searchVector(basisEmbedding(0), { limit: 3, detail: 'high' });
|
||||
const slugs = results.map(r => r.slug);
|
||||
const unique = new Set(slugs);
|
||||
expect(unique.size).toBe(slugs.length); // pooling: one row per page
|
||||
});
|
||||
|
||||
test('a strong chunk is NOT crowded out by a hub page (needle surfaces)', async () => {
|
||||
const results = await engine.searchVector(basisEmbedding(0), { limit: 3, detail: 'high' });
|
||||
const slugs = results.map(r => r.slug);
|
||||
// OLD chunk-grain path: top-3 = hub,hub,hub → needle absent. Pooled: needle present.
|
||||
expect(slugs).toContain('notes/needle');
|
||||
expect(slugs.filter(s => s === 'notes/hub').length).toBe(1);
|
||||
});
|
||||
|
||||
test('hub still ranks #1 by its best chunk (max-pool keeps the strongest)', async () => {
|
||||
const results = await engine.searchVector(basisEmbedding(0), { limit: 3, detail: 'high' });
|
||||
expect(results[0].slug).toBe('notes/hub');
|
||||
// hub represented by its 0.99 chunk (chunk_index 0), not a weaker one
|
||||
expect(results[0].chunk_index).toBe(0);
|
||||
});
|
||||
|
||||
test('larger limit returns each page once, ordered by best-chunk score', async () => {
|
||||
const results = await engine.searchVector(basisEmbedding(0), { limit: 10, detail: 'high' });
|
||||
const slugs = results.map(r => r.slug);
|
||||
expect(new Set(slugs).size).toBe(slugs.length);
|
||||
expect(slugs[0]).toBe('notes/hub');
|
||||
expect(slugs[1]).toBe('notes/needle'); // 0.95 beats the 0.90 fillers
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,58 @@
|
||||
/**
|
||||
* T7 — rank-1 score drift telemetry. Records search calls with rank-1 scores,
|
||||
* flushes, and reads back the aggregate (avg + buckets) from search_telemetry.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
||||
import { resetPgliteState } from '../helpers/reset-pglite.ts';
|
||||
import { getTelemetryWriter, readSearchStats, recordSearchTelemetry, _resetTelemetryWriterForTest } from '../../src/core/search/telemetry.ts';
|
||||
import type { HybridSearchMeta } from '../../src/core/types.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
afterAll(async () => { _resetTelemetryWriterForTest(); await engine.disconnect(); });
|
||||
beforeEach(async () => {
|
||||
_resetTelemetryWriterForTest();
|
||||
await engine.executeRaw('DELETE FROM search_telemetry');
|
||||
});
|
||||
|
||||
const meta: HybridSearchMeta = { mode: 'balanced', intent: 'general' } as HybridSearchMeta;
|
||||
|
||||
describe('rank-1 score telemetry', () => {
|
||||
test('records mean + bucket distribution and reads them back', async () => {
|
||||
recordSearchTelemetry(engine, meta, { results_count: 5, rank1_score: 0.95 }); // high
|
||||
recordSearchTelemetry(engine, meta, { results_count: 3, rank1_score: 0.70 }); // solid
|
||||
recordSearchTelemetry(engine, meta, { results_count: 1, rank1_score: 0.40 }); // lt_solid
|
||||
await getTelemetryWriter().flush();
|
||||
|
||||
const stats = await readSearchStats(engine, { days: 1 });
|
||||
expect(stats.rank1_count).toBe(3);
|
||||
expect(stats.avg_rank1_score).toBeCloseTo((0.95 + 0.70 + 0.40) / 3, 5);
|
||||
expect(stats.rank1_distribution).toEqual({ lt_solid: 1, solid: 1, high: 1 });
|
||||
});
|
||||
|
||||
test('queries with no result (no rank1_score) do not pollute the average', async () => {
|
||||
recordSearchTelemetry(engine, meta, { results_count: 0 }); // no rank1
|
||||
recordSearchTelemetry(engine, meta, { results_count: 2, rank1_score: 0.88 });
|
||||
await getTelemetryWriter().flush();
|
||||
|
||||
const stats = await readSearchStats(engine, { days: 1 });
|
||||
expect(stats.total_calls).toBe(2);
|
||||
expect(stats.rank1_count).toBe(1); // only the one with a result
|
||||
expect(stats.avg_rank1_score).toBeCloseTo(0.88, 5);
|
||||
expect(stats.rank1_distribution.high).toBe(1);
|
||||
});
|
||||
|
||||
test('empty window → null avg, zero buckets (no NaN)', async () => {
|
||||
const stats = await readSearchStats(engine, { days: 1 });
|
||||
expect(stats.avg_rank1_score).toBeNull();
|
||||
expect(stats.rank1_count).toBe(0);
|
||||
expect(stats.rank1_distribution).toEqual({ lt_solid: 0, solid: 0, high: 0 });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,56 @@
|
||||
/**
|
||||
* T2 — applyTitleBoost stage unit tests (mutate-in-place post-fusion stage).
|
||||
* Verifies: title-match multiplies + stamps; non-match untouched; floor-gate
|
||||
* skips below-threshold results; idempotent factor stamping.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { applyTitleBoost, DEFAULT_TITLE_BOOST } from '../../src/core/search/hybrid.ts';
|
||||
import type { SearchResult } from '../../src/core/types.ts';
|
||||
|
||||
function mk(slug: string, title: string, score: number): SearchResult {
|
||||
return {
|
||||
slug, title, score,
|
||||
chunk_text: '', type: 'note', source_id: 'default',
|
||||
chunk_index: 0, chunk_id: 1,
|
||||
} as unknown as SearchResult;
|
||||
}
|
||||
|
||||
describe('applyTitleBoost', () => {
|
||||
test('multiplies score and stamps title_match_boost on a title-phrase match', () => {
|
||||
const r = mk('projects/mingtang', 'The Mingtang — Indoor Greek Amphitheater', 0.8);
|
||||
applyTitleBoost([r], 'greek amphitheater', DEFAULT_TITLE_BOOST);
|
||||
expect(r.score).toBeCloseTo(0.8 * 1.25, 6);
|
||||
expect(r.title_match_boost).toBe(1.25);
|
||||
});
|
||||
|
||||
test('leaves non-matching results untouched (no stamp)', () => {
|
||||
const r = mk('notes/other', 'Completely Unrelated Title', 0.8);
|
||||
applyTitleBoost([r], 'greek amphitheater', DEFAULT_TITLE_BOOST);
|
||||
expect(r.score).toBe(0.8);
|
||||
expect(r.title_match_boost).toBeUndefined();
|
||||
});
|
||||
|
||||
test('floor-gate skips a matching result below threshold', () => {
|
||||
const strong = mk('projects/mingtang', 'Indoor Greek Amphitheater', 1.0);
|
||||
const weak = mk('notes/aside', 'A Greek Amphitheater Footnote', 0.50);
|
||||
// threshold = 0.85 * topScore(1.0) = 0.85; weak (0.50) is below.
|
||||
applyTitleBoost([strong, weak], 'greek amphitheater', DEFAULT_TITLE_BOOST, 0.85);
|
||||
expect(strong.title_match_boost).toBe(1.25);
|
||||
expect(weak.title_match_boost).toBeUndefined(); // gated out
|
||||
expect(weak.score).toBe(0.50);
|
||||
});
|
||||
|
||||
test('factor <= 1.0 is a no-op (disabled)', () => {
|
||||
const r = mk('projects/mingtang', 'Greek Amphitheater', 0.8);
|
||||
applyTitleBoost([r], 'greek amphitheater', 1.0);
|
||||
expect(r.score).toBe(0.8);
|
||||
expect(r.title_match_boost).toBeUndefined();
|
||||
});
|
||||
|
||||
test('empty query is a no-op', () => {
|
||||
const r = mk('projects/mingtang', 'Greek Amphitheater', 0.8);
|
||||
applyTitleBoost([r], '', DEFAULT_TITLE_BOOST);
|
||||
expect(r.score).toBe(0.8);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,68 @@
|
||||
/**
|
||||
* T2 — title-phrase matcher unit tests (retrieval-maxpool incident).
|
||||
*
|
||||
* Pins isTitlePhraseMatch: the incident query "Greek amphitheater" must match
|
||||
* the Mingtang title; generic single-word / stopword queries must NOT promote
|
||||
* arbitrary pages (Codex#10 precision guard).
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { isTitlePhraseMatch, tokenizeTitle, __test__ } from '../../src/core/search/title-match.ts';
|
||||
|
||||
describe('isTitlePhraseMatch — positive (the incident)', () => {
|
||||
const MINGTANG = 'The Mingtang (明堂) — Indoor Greek Amphitheater for Adversarial Debate';
|
||||
|
||||
test('the exact incident query matches', () => {
|
||||
expect(isTitlePhraseMatch('Greek amphitheater', MINGTANG)).toBe(true);
|
||||
});
|
||||
test('case + whitespace insensitive', () => {
|
||||
expect(isTitlePhraseMatch(' greek AMPHITHEATER ', MINGTANG)).toBe(true);
|
||||
});
|
||||
test('contiguous multi-token phrase inside a longer title', () => {
|
||||
expect(isTitlePhraseMatch('indoor greek amphitheater', MINGTANG)).toBe(true);
|
||||
});
|
||||
test('exact full-title match of a 1-word chosen name', () => {
|
||||
expect(isTitlePhraseMatch('Mingtang', 'Mingtang')).toBe(true);
|
||||
});
|
||||
test('exact full-title match with punctuation/stopwords normalized', () => {
|
||||
expect(isTitlePhraseMatch('the hall of light', 'The Hall of Light')).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('isTitlePhraseMatch — negative (precision guard)', () => {
|
||||
const MINGTANG = 'The Mingtang — Indoor Greek Amphitheater';
|
||||
test('single generic content word does NOT match (would over-promote)', () => {
|
||||
// "greek" alone is 1 content token → below MIN_CONTENT_TOKENS, no boost.
|
||||
expect(isTitlePhraseMatch('greek', MINGTANG)).toBe(false);
|
||||
});
|
||||
test('single stopword does NOT match', () => {
|
||||
expect(isTitlePhraseMatch('the', MINGTANG)).toBe(false);
|
||||
});
|
||||
test('non-contiguous tokens do NOT match (phrase, not bag-of-words)', () => {
|
||||
// "greek" and "indoor" both present but not contiguous in query order.
|
||||
expect(isTitlePhraseMatch('amphitheater indoor', 'The Indoor X Greek Amphitheater')).toBe(false);
|
||||
});
|
||||
test('substring that is not a token boundary does NOT match', () => {
|
||||
// "art" must not match "Bartholomew".
|
||||
expect(isTitlePhraseMatch('art history', 'Bartholomew History Notes')).toBe(false);
|
||||
});
|
||||
test('empty query / empty title', () => {
|
||||
expect(isTitlePhraseMatch('', MINGTANG)).toBe(false);
|
||||
expect(isTitlePhraseMatch('greek amphitheater', '')).toBe(false);
|
||||
});
|
||||
test('two stopwords only do NOT match', () => {
|
||||
expect(isTitlePhraseMatch('the of', 'The Theory of Everything')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('tokenizeTitle', () => {
|
||||
test('splits on punctuation + whitespace, lowercases', () => {
|
||||
expect(tokenizeTitle('The Mingtang — Indoor Greek!')).toEqual(['the', 'mingtang', 'indoor', 'greek']);
|
||||
});
|
||||
test('keeps CJK runs as tokens', () => {
|
||||
expect(tokenizeTitle('明堂 hall')).toEqual(['明堂', 'hall']);
|
||||
});
|
||||
test('MIN_CONTENT_TOKENS is 2 (the precision floor)', () => {
|
||||
expect(__test__.MIN_CONTENT_TOKENS).toBe(2);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user