Salvage of PR #2875 (which subsumes the base projection from #2873),
rebased onto current master with the release bookkeeping (VERSION /
package.json / CHANGELOG) dropped, plus one hot-path hardening fix.
Salvaged (verified on this head):
- project trusted message_id / thread_id / Message-ID-gated source_subject
through keyword, chunk-keyword, CJK, and vector paths in BOTH engines
- preserve the citation DTO through alias injection, relational
recall/fanout, two-pass hydration, vector fusion/reranking, and
semantic-cache hits
- raw source_subject is never trusted; only allowlisted `subject` may
supply it, and only when a nonblank Message-ID proves the page is an
email; malformed/non-object frontmatter fails closed (no double-decode)
- source visibility / quarantine / deletion rechecked across indirect
retrieval paths (alias hop, relational hydrate, two-pass expansion,
graph walk, cache-hit gate)
- typed JSON cache scope keys (scalar/set/all) — injective encoding, no
forged-key collisions; store-side write gate skips writeback when the
page-generation clock advanced during the producing search
- KNOBS_HASH_VERSION 12 -> 13 so pre-projection cached DTOs miss instead
of replaying the old shape
Fixed on top of the original head (the flagged hot-path defect):
- cacheScopeKey's forged-id rejection was evaluated inline at the cache
lookup/store call sites inside hybridSearchCached, outside any catch —
an invalid scope id broke the whole search instead of skipping the
cache. The key is now computed once, fail-open: invalid scope =>
cache skipped, search unaffected. Pinned by
test/search/hybrid-cache-scope-failopen.serial.test.ts (fails on the
original head, passes here).
Verified on this exact head: typecheck clean; 379 touched unit tests,
3 serial files (one process each), pglite cache-gate/source-isolation
e2e, and real-PostgreSQL engine-parity (26 pass) + source-routing —
all green. jsonb-pattern/params, key-files-current-state,
test-isolation, progress-to-stdout guards clean.
Closes#2962
Co-authored-by: amtagrwl <amtagrwl@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
