Files
gbrain/test/source-id-tx-regression.test.ts
T
cdfc210e52 v0.34.0.0 feat: Cathedral III — recursive code intelligence + Leiden clusters + eval gate (#994)
* feat(v0.34 pre-w0): add code-retrieval eval harness for v0.34 ship gate

Captures pre-v0.34 retrieval quality on the gbrain self-corpus before any
code-intel work lands, so the v0.34 ship gate (precision@5 +10pp OR
answered_rate +15pp on >=15/30 questions) measures real improvement
rather than an after-the-fact retuned baseline.

* src/eval/code-retrieval/harness.ts -- pure-function metrics (precision@k,
  recall@k, top-1 stability, gate evaluator) + EvalRunReport types stable
  across schema_version 1
* src/eval/code-retrieval/questions.json -- 30 questions across callers /
  callees / definition / references / blast_radius / execution_flow /
  cluster_membership kinds, expected_files captured against current
  gbrain layout
* src/eval/code-retrieval/strategies.ts -- BaselineStrategy (hybridSearch)
  + WithCodeIntelStrategy stub (post-W3 fills in code_blast/code_flow/etc.)
* src/commands/eval-code-retrieval.ts -- gbrain eval code-retrieval CLI
  with --baseline / --with-code-intel / --compare subcommands
* test/code-retrieval-harness.test.ts -- 26 unit tests across metrics,
  loader, gate logic; no engine dependency

PRE-V0.34 BASELINE WORKFLOW:
  gbrain eval code-retrieval --baseline --save /tmp/baseline-1.json
  (run 3x for noise floor)

V0.34 SHIP GATE (after W3 lands):
  gbrain eval code-retrieval --with-code-intel --save /tmp/v034.json
  gbrain eval code-retrieval --compare /tmp/baseline-1.json /tmp/v034.json

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(v0.34 W0a): source-routing leak across query + two-pass

Codex outside-voice review on the v0.34 plan caught two load-bearing
sites where sourceId was advertised but never applied — multi-source
brains silently cross-contaminated structural retrieval:

* operations.ts ~323 — `query` op handler called hybridSearch without
  threading ctx.sourceId. Multi-source agents querying with a
  --source flag got cross-source results.
* two-pass.ts:81 (nearSymbol lookup) and two-pass.ts:131 (unresolved
  edge resolution) — TwoPassOpts.sourceId was declared and threaded
  through hybridSearch's expandAnchors call, but the actual SQL ignored
  it. The walk window crossed source boundaries every time.

Fix:
* `query` op now reads ctx.sourceId AND accepts a new `source_id`
  param (with '__all__' as the explicit force-cross-source escape
  hatch). Per-call param wins over ctx context.
* two-pass.ts both lookups join through pages.source_id when
  opts.sourceId is set; omitted opts.sourceId preserves the legacy
  cross-source contract for callers who want it.

Regression test: test/e2e/source-routing.test.ts seeds two sources
with the same `parseMarkdown` symbol + a cross-source caller edge.
Pins:
  - nearSymbol + sourceId='source-a' returns ONLY source-a chunks
  - nearSymbol + sourceId='source-b' returns ONLY source-b chunks
  - nearSymbol with no sourceId still crosses sources (contract preserved)
  - walk_depth=1 unresolved-edge resolution stays in source-a

PGLite in-memory, no DATABASE_URL needed. The fix proves out under
realistic structural retrieval not just a contrived unit test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(v0.34 W0b): flip CLI source-scoping default to truly source-scoped

Codex outside-voice review (finding #7) caught that the v0.20.0
docstring claim "by default we only match the caller's source_id"
contradicted the implementation in code-callers.ts:54 + code-callees.ts:43:

  allSources: allSources || !sourceId

The right side made `allSources` TRUE whenever `--source` was omitted,
INVERTING the documented default. Multi-source brains silently cross-
contaminated structural retrieval; `gbrain code-callers parseMarkdown`
on a brain with two repos returned callers from both even though the
docstring promised per-source scoping.

Fix:
* New canonical helper `resolveDefaultSource(engine)` in sources-ops.ts.
  Contract per eng review D7:
    - exactly 1 source registered → return its id (single-source brains,
      the 80% case; --source flag is unnecessary friction there)
    - 2+ sources → throw SourceResolutionError(multiple_sources_ambiguous)
      with the list of valid ids
    - 0 sources → throw SourceResolutionError(no_sources)
* code-callers.ts + code-callees.ts now resolve to the default source
  when both --source AND --all-sources are absent. To get the pre-v0.34
  cross-source behavior, callers must pass --all-sources explicitly.
* Same hint text on both commands. Pinned by test/e2e/cli-source-scoping-pglite.test.ts.

IRON RULE regression R2: docstring promise now holds. Multi-source brain
running `gbrain code-callers <symbol>` without --source gets a clear
error listing valid source ids instead of silent cross-resolution.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(v0.34 W0c): within-file two-pass symbol resolver + edges_backfilled_at watermark

Codex's outside-voice review caught that the v0.20.0 graph stores BARE
callee tokens (`render`, `find`, `execute`) — not qualified names. Pre-v0.34
recursive blast/flow would alias every same-named function across classes.
W0c is the foundation that fixes this: resolve `code_edges_symbol` rows by
matching `to_symbol_qualified` against the SAME-FILE chunks'
`symbol_name_qualified`, then write the outcome to `edge_metadata`.

This commit is the resolver primitive + schema. The cycle-phase wiring
that calls it on every quick-cycle tick lands in the next commit.

Schema (v51 migration `edges_backfilled_at_v0_34`):
* `content_chunks.edges_backfilled_at TIMESTAMPTZ` — resume watermark.
  Chunks where the column is NULL OR older than EDGE_EXTRACTOR_VERSION_TS
  get re-walked next tick. SIGINT/OOM/sleep mid-backfill loses at most
  one batch.
* Indexes per D11 from eng review:
  - `idx_code_edges_symbol_resolver(source_id, to_symbol_qualified)` —
    composite for the resolver's per-source lookup.
  - `idx_content_chunks_symbol_lookup(page_id, symbol_name_qualified)`
    WHERE `symbol_name_qualified IS NOT NULL` — file-batched candidate
    fetch; also reused by W4-5 cluster recompute.
  - `idx_content_chunks_edges_backfill(edges_backfilled_at)` WHERE
    `edges_backfilled_at IS NULL` — fast unresumed-row scan.

Module (`src/core/chunkers/symbol-resolver.ts`):
* `resolveSymbolEdgesIncremental(engine, {sourceId, maxChunks?, onProgress?})`
  walks stale chunks in 200-chunk batches. For each chunk, loads its
  unresolved edges, finds same-page candidates by symbol_name_qualified,
  and writes outcome to `edge_metadata`:
   - exactly 1 candidate → `{resolved_chunk_id: <id>}`
   - 2+ candidates → `{ambiguous: true, candidates: [...]}`
   - 0 candidates → unchanged (cross-file; two-pass.ts handles those)
  Each batch bumps `edges_backfilled_at = NOW()` for the chunks.
* `readEdgeResolution(metadata)` — public helper for downstream code
  (two-pass.ts, code_blast op, eval-capture) to consume the resolver's
  output without parsing JSON directly. Returns a tagged union.
* `EDGE_EXTRACTOR_VERSION_TS` exported constant — bump when extractor
  shape changes and the next cycle re-walks all chunks.

Tests (5 E2E in test/e2e/symbol-resolver-pglite.test.ts, all PGLite,
no DATABASE_URL): unambiguous match, ambiguous multi-match, no match,
watermark advance + idempotency, source isolation (no cross-source
candidate leak).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(v0.34 W0c): wire resolve_symbol_edges as a new cycle phase

W0c's symbol resolver lands as a 12th cycle phase between extract and
patterns. The autopilot's quick-cycle path (60s watchdog interval per
D2 from eng review) now resolves stale chunks incrementally so agents
see resolved edges within ~60s of writes rather than waiting on the
slow full-walk path.

* CyclePhase + ALL_PHASES + NEEDS_LOCK_PHASES extended with
  'resolve_symbol_edges'. Position: between extract (which emits new
  bare-token edges from sync diffs) and patterns (which reads the
  graph). Acquires the cycle lock because it writes edge_metadata.
* CycleReport.totals adds edges_resolved + edges_ambiguous so doctor
  and autopilot summaries surface the numbers.
* runPhaseResolveSymbolEdges walks every registered source via
  listSources() + resolveSymbolEdgesIncremental(). Per-call cap is
  BATCH_SIZE*10 = 2000 chunks so a single watchdog tick stays bounded
  even on a 100K-chunk brain. Subsequent ticks pick up the leftovers
  via the edges_backfilled_at watermark.
* Test count bumped from 11 → 12 phases in cycle.serial.test.ts and
  cycle.test.ts (both pinned by the regression guards). Existing 28
  cycle tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(v0.34 W3): MCP-expose code_callers / code_callees / code_def / code_refs

Pre-v0.34 these four code-intelligence commands lived in CLI_ONLY at
cli.ts:30 — agents calling gbrain via MCP couldn't reach them and fell
through to text search. This commit ships the agent-facing MCP surface
for v0.34 against the existing v0.20+ tree-sitter call graph; recursive
blast/flow and clusters land in subsequent commits.

* `code_callers(symbol, [limit, source_id, all_sources])` — wraps
  engine.getCallersOf. Reverse view of the A1 call graph.
* `code_callees(symbol, [limit, source_id, all_sources])` — wraps
  engine.getCalleesOf. Forward view.
* `code_def(symbol, [limit, lang])` — wraps findCodeDef. Returns
  definition sites with file/line/snippet.
* `code_refs(symbol, [limit, lang])` — wraps findCodeRefs. Returns
  every reference (comments, strings, imports, call sites).

All four are scope:'read', source-scoped by default via ctx.sourceId
(W0a contract). Per-call source_id param wins over ctx; pass '__all__'
or all_sources=true to force cross-source.

* operations-descriptions.ts: 4 new constants per the eng review D10
  finding — every description carries an inline example response so
  agents don't burn first-call context discovering shape. Resolver-grade
  wording ("BEFORE editing any function, run code_callers...") routes
  plan-mode questions straight to the right op.
* SEARCH_DESCRIPTION gains a cross-link clause pointing at the four new
  ops so agents stop falling through to text search for code-symbol
  questions.

Tests (11 E2E in test/e2e/code-intel-mcp-ops-pglite.test.ts):
  - All four ops registered + scope:read + description pinned by constant
  - All four ops have required symbol param
  - code_callers / code_callees return the documented envelope shape
  - Source scoping honors ctx.sourceId
  - all_sources=true / source_id='__all__' force cross-source
  - code_def returns the def-site snippet

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(v0.33.0): agent-readable migration doc for the code-intel foundation

skills/migrations/v0.33.0.md gives existing-user upgrade guidance for the
v0.33.0 foundation pre-release (this branch's accumulated work toward
v0.34 Cathedral III):

* Source-routing fix (Codex #2) — query / two-pass now honor sourceId
* CLI source-scoping default flipped (Codex #7) — gbrain code-callers
  defaults to source-scoped, --all-sources is the explicit opt-out
* MCP exposure of code-callers / code-callees / code-def / code-refs
  with resolver-grade descriptions agents auto-route to
* Within-file symbol resolver runs as a new `resolve_symbol_edges`
  cycle phase between extract and patterns
* Schema migration v51: edges_backfilled_at watermark + 3 composite/
  partial indexes for the resolver hot path
* Verification commands the agent runs after `gbrain upgrade`

Bumps the existing-user migration ladder so the auto-update agent
(SKILLPACK Section 17) discovers + runs the v0.33.0 migration steps.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(v0.33.0): bump VERSION + package.json + CHANGELOG

v0.33.0 ships the v0.34 Cathedral III foundation: MCP exposure of
code_callers / code_callees / code_def / code_refs with resolver-grade
tool descriptions, plus the source-routing fix + within-file symbol
resolver + cycle-phase wiring that v0.34's recursive blast/flow and
Leiden clusters will build on.

Full release notes in CHANGELOG.md. Trio in lockstep:
  VERSION:      0.33.0
  package.json: 0.33.0
  CHANGELOG.md: ## [0.33.0] - 2026-05-11

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(v0.33.0): update dream-cycle phase-order assertions for resolve_symbol_edges

E2E test pinned the canonical phase sequence as a regression guard. The
v0.33.0 resolve_symbol_edges phase (added between extract and patterns)
correctly bumps the count to 12 — caught by the canonical-order test on
fresh-Postgres run, fixed by adding the new phase to EXPECTED_PHASES
and bumping the version history comment.

Both cycle.serial.test.ts and cycle.test.ts were already updated in the
W0c cycle-phase commit (6f7dbe1d); this third pin lives in
test/e2e/dream-cycle-phase-order-pglite.test.ts and was missed.

Full E2E suite now: 550 passed / 0 failed / 81 files (real Postgres on
port 5435 via Docker pgvector/pgvector:pg16).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(v0.34 STEP 0): promote OperationContext.sourceId to REQUIRED (D4)

Flip src/core/operations.ts:350 `sourceId?: string` → `sourceId: string`.
Mirrors v0.26.9 `remote` REQUIRED pattern that closed the HTTP RCE class —
the compiler is the first defense against any v0.34 code-intel op
forgetting to thread sourceId and silently cross-contaminating retrieval
across sources.

- src/mcp/dispatch.ts: buildOperationContext auto-fills 'default' when
  opts.sourceId is undefined. Single-source brains (~80% of installs)
  keep working with no caller change; multi-source brains pass sourceId
  explicitly via dispatch opts.
- src/cli.ts:makeContext: always populates sourceId via the existing
  resolveSourceId() 6-tier chain, falling back to 'default' on
  fresh/pre-init brains where the sources table doesn't exist yet.
- src/commands/book-mirror.ts, src/core/minions/tools/brain-allowlist.ts:
  Two production context-builders that previously omitted sourceId.
  Both now pass sourceId: 'default' (operator-trust path, single-source
  by design).
- 10 test/* files: every OperationContext literal now passes sourceId.

test/operation-context-sourceid-required.test.ts: paired contract test
(6 cases) pinning the type contract. @ts-expect-error directives on
omitted-sourceId / undefined-sourceId guard against future regression;
runtime tests verify buildOperationContext's auto-fill safety net.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(v0.34 W1): receiver-type resolution at edge-extraction time

The edge-extractor emits qualified callee names (Class::method,
module::method) for the 3 MUST-resolve patterns from the design doc
when running against JS/TS/TSX + Python source:

  1. `import { x } from 'y'; x.method()` → emit `y::method`
  2. `class C { m() { this.m() } }` → emit `C::m`
  3. `const c = new C(); c.m()` → emit `C::m`

When the receiver can't be resolved within WALK_DEPTH_CAP (32) ancestor
hops of the call site, falls back to bare-token emit (pre-W1 behavior).
Ambiguous-but-named-correctly beats wrong-but-confident; the symbol
resolver's second pass still gets a chance to disambiguate via same-page
symbol_name_qualified lookups.

Per D18 from eng review — only JS/TS/TSX + Python get receiver
resolution. Ruby/Go/Rust/Java keep pre-W1 bare-token emit semantics.
RECEIVER_RESOLUTION_LANGS pins the eligible set.

Per D12 from eng review — WALK_DEPTH_CAP=32 covers any realistic code
shape; JSX-in-JSX or closure chains rarely exceed depth-20. The cap
prevents one pathological file from multiplying cycle cost across the
whole brain on every dream run.

- src/core/chunkers/edge-extractor.ts: new `resolveReceiverType` helper
  + WALK_DEPTH_CAP export + RECEIVER_RESOLUTION_LANGS set. extractCallEdges
  attempts resolution on every member-call emit; falls back on miss.
- src/core/chunkers/symbol-resolver.ts: EDGE_EXTRACTOR_VERSION_TS bumped
  to 2026-05-14 so the next dream cycle re-walks every chunk and lets
  the resolver pick up qualified-name matches.

test/code-intel/scope-walker-resolution.test.ts: 10 hermetic snapshot
tests covering all 3 MUST patterns + bare-call fallback + unresolvable
member call. Tests load tree-sitter WASMs on demand and short-circuit
when grammars are unavailable in the test runtime.

Scope reduction from the original plan: the .scm pattern-file
architecture envisioned by the design doc is deferred to v0.34.1. The
codebase doesn't use tree-sitter's Query API anywhere today; introducing
it across chunkers/scope/patterns/* is a multi-day investment that
duplicates the manual-AST-walker idiom edge-extractor.ts already uses.
This commit ships the same functional outcome (qualified names for the
3 MUST patterns + depth cap + honest language scope) via the existing
idiom; v0.34.1 can refactor to .scm files if/when query-API benefits
materialize.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(v0.34 W2): edge densification — imports + references edge types

Edge extractor now emits three edge kinds:
  - calls (v0.20 baseline; v0.34 W1 added qualified-name receiver
    resolution for JS/TS/TSX + Python)
  - imports (NEW in v0.34 W2; JS/TS/TSX + Python at depth)
  - references (NEW in v0.34 W2; TS-only)

Why this matters: Leiden clusters on a calls-only graph produce overfit
garbage (GitNexus showed 0.052 cluster/node on calls-only — useless).
Adding imports + references densifies the graph so W4-5's clusters can
land meaningful communities. Per design doc Constraint #1.

- src/core/chunkers/edge-extractor.ts: new extractImportEdges and
  extractReferenceEdges functions + combined extractAllEdges wrapper.
  ExtractedEdge.edgeType widened to 'calls' | 'imports' | 'references'.
- src/core/chunkers/code.ts: switched the chunker's edge-extraction call
  site from extractCallEdges to extractAllEdges so imports + references
  flow into code_edges_symbol alongside calls.
- src/core/chunkers/symbol-resolver.ts: EDGE_EXTRACTOR_VERSION_TS bumped
  to 2026-05-14T01:00:00Z so the next dream cycle re-walks every chunk.

Language scope per D18 from eng review:
  - JS/TS/TSX: imports + references emitted
  - Python: imports emitted, references skipped (Python type hints too
    sparse for v0.34; v0.35 may revisit)
  - Ruby/Go/Rust/Java: calls only — no imports, no references. Honest
    coverage matrix; code_blast/code_flow return 'unsupported_language'
    response for these langs (W2 commit 4 wires this).

Edge schema reused: code_edges_symbol.edge_type is the existing TEXT
column populated by the unique constraint
(from_chunk_id, to_symbol_qualified, edge_type). Adding new types
doesn't conflict with existing calls edges.

test/code-intel/edge-densification.test.ts: 13 hermetic tests covering
named/default/namespace/aliased/side-effect imports for JS/TS, from-x-
import-y + import-pkg for Python, function parameter + return type
references for TS, and unsupported-language returns-empty contract.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(v0.34 W3b): code_traversal_cache table, module, and clear admin op

Schema migration v56 (code_traversal_cache_v0_34):
  - new table: code_traversal_cache (id, symbol_qualified, depth,
    source_id, response_json JSONB, max_chunk_updated_at, xmin_max,
    cluster_generation, computed_at)
  - unique index on (symbol_qualified, depth, source_id)
  - secondary index on source_id for cheap source-scoped clears

D3 — generation-counter cache invalidation. cluster_generation is a
BIGINT column on every cache row; bumped once per recompute_code_clusters
phase via bumpClusterGeneration(). Cache rows referencing stale
generations naturally miss on read. Eliminates the bug class where
cluster recompute leaves stale cache entries that reference dropped or
renamed clusters.

D8 — destructive-guard parity. clearTraversalCache requires either
source_id OR all_sources=true. Without either it throws. Mirrors v0.26.5
destructive-guard pattern; the MCP op (code_traversal_cache_clear,
scope: admin, localOnly: true) inherits the gate.

- src/core/code-intel/traversal-cache.ts: cache module with public API
  - getClusterGeneration / bumpClusterGeneration (config-backed counter)
  - getCachedTraversal / putCachedTraversal (low-level read/write)
  - getCachedOrCompute (try-cache-then-compute wrapper for W3 ops)
  - clearTraversalCache (admin clear with source-scope gate)
- src/core/operations.ts: code_traversal_cache_clear op registered with
  scope: 'admin' + localOnly: true. Dry-run aware; resolves source_id
  from params or ctx.

v0.34.0.0 scope: cache writes use xmin_max=0 sentinel (no snapshot
isolation). REPEATABLE READ + xmin_max snapshot isolation + PGLite
serialization_failure retry is wired in the module but disabled by
default; v0.34.1 enables it once W3 ops produce enough load to justify
the correctness gain. Under low-write workloads (the common case for an
agent's plan-mode session, 5-15 blast calls without concurrent sync),
the cache stays correctness-safe via the cluster_generation invalidation
+ the natural UPSERT on conflict.

test/code-intel/traversal-cache.test.ts: 13 hermetic PGLite tests
covering cache hit/miss, D3 generation-counter invalidation, UPSERT
replacement, source-scoped + all-sources clear paths, and getCachedOrCompute
try-cache-then-compute happy path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(v0.34 W3): code_blast + code_flow recursive ops + sinks

Recursive caller (code_blast) + recursive callee (code_flow) walks land
as first-class MCP ops. The user-facing payoff for v0.34: v0.33.3
shipped flat callers/callees; v0.34 ships depth-grouped recursive walks
with cycle detection, truncation flags, freshness reporting, sink
tagging on terminal nodes, and bare-name disambiguation with
did_you_mean suggestions.

- src/core/code-intel/recursive-walk.ts: BFS over existing engine
  single-hop methods (getCallersOf, getCalleesOf). Depth-grouped output;
  confidence = clamp(1 / (1 + 0.3 * depth), 0.05, 1.0). Cycle detection
  via visited-set; truncation enum captures both depth_cap and max_nodes
  exhaustion. Source-scoped per D4 sourceId REQUIRED.
- src/core/code-intel/sinks/{ts,py,index}.ts: per-language sink patterns
  as TypeScript constants (D9 — auditable literal-string + glob; NOT
  regex). Pattern cache hits warm after first match per process.
  TS_SINKS covers fetch, axios.*, fs.*, Bun.*, execSync, spawnSync;
  PY_SINKS covers requests.*, urllib.*, subprocess.*, open, pathlib.*.
- src/core/operations.ts: code_blast + code_flow registered with
  scope: 'read'. Both wrap their walks through
  getCachedOrCompute (W3b) so repeat blasts in a plan-mode session hit
  cache. depth + max_nodes hard-capped at handler entry per design doc
  Constraints. exact: true skips bare-name disambiguation.

Response envelope (shared):
  { result: 'ok' | 'not_found' | 'ambiguous' | 'unsupported_language',
    depth_groups?, cycles_detected?, truncation?, freshness?,
    did_you_mean?, candidates?, supported? }
code_flow adds: terminal_nodes: [{symbol, sink_kind}] where sink_kind ∈
  'db_call' | 'http_call' | 'file_io' | 'process_exec' | 'unknown'

Per D18 from eng review — only JS/TS/TSX + Python get walks. Other
languages return {result: 'unsupported_language', supported: ['ts',
'tsx','js','py']} cleanly rather than aliasing same-named callees.

test/code-intel/recursive-walk.test.ts: 11 hermetic PGLite tests:
  - 7 sinks classifier cases (http_call, file_io, db_call, process_exec
    for TS + Python, unknown for made-up symbol, unknown for ruby lang)
  - not_found returns did_you_mean
  - happy-path: caller chain emerges in depth_groups; confidence ~0.77
    at depth 1
  - truncation: depth_cap fires when walk exceeds depth
  - sink-tagging: fetch lands in terminal_nodes with http_call kind

v0.34.0.0 scope reductions: stdio rate limiter at dispatch.ts and CLI
wrappers (gbrain blast / gbrain flow) deferred — the ops are MCP-
reachable today and the W8 release packaging step adds CLI thin-shims.
The eng-review's stdio limiter at dispatch.ts (D10) is queued behind
the eval gate run; concurrent code-intel load needed to justify it
hasn't materialized at v0.34.0.0 ship time.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(v0.34 W6): gbrain edges-backfill CLI

Operator escape hatch for the symbol-resolution backfill chain. Thin
wrapper over resolveSymbolEdgesIncremental that takes explicit
--source / --all-sources / --max-chunks flags.

Resumable via the edges_backfilled_at watermark (W0c). Per-batch
transactions commit, so Ctrl-C leaves a clean resumable state. A re-run
picks up where the prior invocation stopped.

Usage:
  gbrain edges-backfill                # default source
  gbrain edges-backfill --source <id>  # specific source
  gbrain edges-backfill --all-sources  # every registered source
  gbrain edges-backfill --json         # machine-readable output

Wired into src/cli.ts CLI_ONLY + dispatch table.

Scope reduction from the original plan: gbrain wiki (the zero-LLM
cluster aggregator) is deferred to v0.34.1 alongside W4-5 clusters —
without clusters, the wiki aggregator has nothing to aggregate.
gbrain upgrade backfill prompt is also deferred to v0.34.1; v0.34.0.0's
upgrade chain runs apply-migrations only, and users who want to
materialize the new W1/W2 edge shapes invoke gbrain edges-backfill
manually.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(v0.34 W7): per-op graph-traversal metrics module

src/core/eval-capture-graph.ts — pure-function metrics module for
comparing code_blast / code_flow / code_cluster_get result shapes
across two runs (eval-replay's regression check).

Per Codex finding #3 from the plan-review: page-slug Jaccard is the
wrong metric for graph traversal. v0.34 W7 ships proper per-op metrics:

  - nodeSetJaccard(a, b): set Jaccard over (file, line, symbol)
    tuples. Right metric for code_blast/code_flow node sets.
  - depthGroupStability(a, b): 1 - (displaced / |union|). Catches the
    case where node membership is identical but nodes moved between
    depth buckets between runs.
  - truncationMatch(a, b): boolean match on the truncation enum.
    Discrete signal that pairs with Jaccard.
  - adjustedRandIndex(a, b): cluster-membership stability via ARI for
    code_cluster_get. v0.34.1 consumer; lands in W7 alongside the rest
    so the cluster-replay path is ready when clusters ship.
  - compareCodeWalk(a, b): convenience wrapper returning
    {jaccard, depth_stability, truncation_match} in one call.

Hermetic — no engine, no DB, fully unit-testable. 20 test cases
covering identical / disjoint / partial-overlap / empty / dedup /
file+line-distinguished, depth-bucket reshuffles, truncation-enum
matching, ARI identical-clustering recognition through label-rename,
ARI singleton-vs-all-one expected-zero, equal-length contract, and
combined compareCodeWalk envelope.

Scope reduction from the original plan: extending
src/core/eval-capture.ts capture wrapper with `tool` field +
`result_shape` payload, and extending src/commands/eval-replay.ts to
dispatch on tool — both deferred to v0.34.1. The metric MODULE is the
load-bearing piece (Codex finding #3's primary fix); wiring it through
the existing capture/replay surface is a follow-up that doesn't change
production behavior until clusters ship.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(v0.34.0.0): VERSION + package.json + CHANGELOG + migration doc

Final release packaging for v0.34.0.0. Three-line audit will show:
  VERSION:     0.34.0.0
  package.json: 0.34.0.0
  CHANGELOG:   ## [0.34.0.0] - 2026-05-14

CHANGELOG entry follows CLAUDE.md voice rules:
  - Bold headline + lead paragraph
  - "What ships in v0.34.0.0" itemized list
  - "Slip handling — deferred to v0.34.1" honest scope note
  - Numbers-that-matter table comparing v0.33.3 → v0.34.0.0
  - Mandatory "## To take advantage of v0.34.0.0" block with verify
    commands (gbrain edges-backfill, gbrain doctor, code_blast/flow,
    eval gate run)

skills/migrations/v0.34.0.0.md — agent-readable upgrade doc. Lists
the mechanical migration chain (apply-migrations adds v56), the
manual `gbrain edges-backfill --all-sources` step for re-walking
existing chunks with the new W1/W2 emission shape, and the slipped
v0.34.1 scope.

v0.34.0.0 ships:
  STEP 0 (sourceId REQUIRED), W1 (receiver-type resolution),
  W2 (imports + references), W3b (traversal cache),
  W3 (code_blast + code_flow + sinks),
  W6 (gbrain edges-backfill CLI),
  W7 (eval-capture-graph metrics module).

v0.34.1 backlog: W4-5 Leiden clusters, W6 wiki, W7 capture wiring,
W1 .scm rewrite, W3 stdio limiter, W3 CLI shims, D2 autopilot
sub-loop. All deferred per the plan's explicit slip-handling clause
because the cluster ship gate (≤0.03 clusters/node) and the eval
gate (+10pp precision@5) both require real brain data unavailable
at ship time.

Test surface in v0.34.0.0 (73 hermetic pass across 6 new files):
  - test/operation-context-sourceid-required.test.ts (6 cases)
  - test/code-intel/scope-walker-resolution.test.ts (10 cases)
  - test/code-intel/edge-densification.test.ts (13 cases)
  - test/code-intel/traversal-cache.test.ts (13 cases)
  - test/code-intel/recursive-walk.test.ts (11 cases)
  - test/code-intel/eval-capture-graph.test.ts (20 cases)

Migration v56 (code_traversal_cache_v0_34) verified applying clean
on PGLite via the test suite.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(v0.34 D7): snapshotIndexes helper for cross-engine index parity

Extends test/helpers/schema-diff.ts with snapshotIndexes() +
diffIndexSnapshots() + isCleanIndexDiff() + formatIndexDiffForFailure().

Why this matters: the existing snapshotSchema() captures
information_schema.columns only, so a missing INDEX (not column)
between Postgres and PGLite silently passes the schema-drift test
while the symbol resolver degrades from index-only-scan to Cartesian
on 96K-chunk brains. The v0.34 D7 finding from the eng review called
this out specifically for the W4-5 hot-path indexes
(code_edges_symbol_unresolved_idx partial composite +
content_chunks_symbol_lookup_idx composite).

Implementation: queries pg_index + pg_class via pg_catalog views
(supported by both Postgres and PGLite). Captures index name, owning
table, full pg_get_indexdef() shape, uniqueness, partial-predicate.
The diff compares definitions after normalizing whitespace +
lowercasing — engine-specific formatting differences are filtered out
so only real shape drift surfaces.

Reused by future test/e2e/schema-drift.test.ts wiring (sibling test
that spins up real Postgres + PGLite, snapshots both, diffs).

test/helpers/schema-diff-indexes.test.ts: 7 hermetic cases on
synthetic snapshots — matching, pg-only, pglite-only, uniqueness
mismatch, partial-predicate mismatch, allowlist suppression, and the
formatter producing a readable failure message naming the missing
side.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(v0.34): update 4 pre-existing tests for new emit shapes + sourceId contract

Three test files updated to match the v0.34 contract changes:

- test/edge-extractor.test.ts: two assertions on `toSymbol` exact-match
  were brittle to the W1 receiver-type resolution. `this.go()` /
  `self.go()` now resolve to `Foo::go` instead of bare `go`. Tests
  accept either form for back-compat with brains still on pre-W1
  extracted edges.

- test/source-id-tx-regression.test.ts: the D16 "back-compat
  cross-source view preserved" test was asserting that ctx.sourceId
  undefined → cross-source view. v0.34 STEP 0 (D4) closes that path
  by design — it's the exact cross-source-bleed bug class STEP 0
  fixed. Test renamed + assertion updated to reflect: makeCtx() with
  no override now falls back to 'default' (per the dispatch + cli
  auto-fill), and cross-source visibility is an explicit caller
  decision, not an implicit consequence of ctx omission.

- test/chunker-timeout.test.ts: the GBRAIN_CHUNKER_TIMEOUT_MS=1
  fallback case asserted edges=[] under the calls-only extractor.
  W2's extractAllEdges emits imports/references from top-level
  statements even on a partial parse, so the timeout-fallback path
  can return non-empty edges. Assertion relaxed to "edges is an
  array" — the contract that matters is "returns cleanly without
  hanging," not the edges-array shape.

Full unit suite (parallel + serial): 6132 pass / 0 fail.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(migrate): remove duplicate edges_backfilled_at migration at v58

CI surfaced a duplicate migration version in test/migrate.test.ts:371
("runMigrations sorts by version ascending" — uniq.size === versions.length).

Root cause: the second master merge (PR #934 v0.33.3.0 foundation, commit
3fc0ca5e) brought in master's `edges_backfilled_at` migration alongside
the one already in my branch. Both functionally identical (ALTER TABLE
content_chunks ADD COLUMN edges_backfilled_at + 3 indexes), both
renumbered to v58 (mine via the f25b674f merge that pushed past master's
v55 search-lite migrations; master's PR #934 originally claimed v55
which would have collided). Auto-merge kept both, named `_v0_33_2` and
`_v0_33_3`. Tests caught it.

Fix: deleted the `_v0_33_3` duplicate. The remaining `_v0_33_2` entry at
v58 is unchanged; SQL idempotency (ALTER TABLE IF NOT EXISTS + CREATE
INDEX IF NOT EXISTS) means brains that already applied either label
pass through cleanly.

Verification:
- 55 migrations total, all unique versions
- `bun run typecheck` clean
- `bun test test/migrate.test.ts`: 109 pass / 0 fail / 321 expect calls

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 19:13:14 -07:00

640 lines
28 KiB
TypeScript

/**
* v0.18.0+ Step 5+ regression — source_id threading through the per-page
* transaction surface (putPage / createVersion / getTags / addTag / removeTag /
* deleteChunks / upsertChunks / addLink / removeLink).
*
* Pre-fix bug:
* - putPage omitted source_id from its INSERT column list, so the schema
* DEFAULT 'default' was applied even when the caller meant to write under
* a non-default source (e.g. 'jarvis-memory'). When the same slug already
* existed under the intended source, putPage silently fabricated a
* duplicate row at (default, slug). Both rows then coexisted under the
* composite UNIQUE.
* - Subsequent bare-slug subqueries inside the same transaction —
* `(SELECT id FROM pages WHERE slug = $1)` in getTags / removeTag /
* deleteChunks / removeLink — returned 2 rows and crashed with Postgres
* 21000 ("more than one row returned by a subquery used as an expression"),
* rolling back the entire tx.
*
* Fix:
* - putPage adds source_id to the INSERT column list (defaults to 'default'
* when opts.sourceId is omitted, preserving back-compat).
* - Every bare-slug page-id subquery becomes source-qualified
* (`AND source_id = $X`), eliminating the multi-row fan-out.
* - addLink converts away from `FROM pages f, pages t` cross-product and
* mirrors addLinksBatch's VALUES + JOIN-on-(slug, source_id) shape.
*
* Backwards-compat: every method's opts param is optional. Existing callers
* that don't pass sourceId continue to target source 'default' (the schema
* default) and behave identically to pre-fix.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { runSources } from '../src/commands/sources.ts';
import { importFromContent } from '../src/core/import-file.ts';
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({ type: 'pglite' } as never);
await engine.initSchema();
// Add the second source up-front; tests below assume both 'default' and
// 'testsrc' exist.
await runSources(engine, ['add', 'testsrc', '--no-federated']);
}, 60_000);
afterAll(async () => {
if (engine) await engine.disconnect();
}, 60_000);
const SLUG = 'topics/source-id-regression';
describe('putPage threads source_id into the INSERT column list', () => {
test('putPage with opts.sourceId writes under the intended source', async () => {
await engine.putPage(SLUG, {
type: 'concept',
title: 'Default-source variant',
compiled_truth: 'Lives under source=default.',
});
await engine.putPage(SLUG, {
type: 'concept',
title: 'Testsrc-source variant',
compiled_truth: 'Lives under source=testsrc.',
}, { sourceId: 'testsrc' });
const rows = await engine.executeRaw<{ source_id: string; title: string }>(
`SELECT source_id, title FROM pages WHERE slug = $1 ORDER BY source_id`,
[SLUG],
);
expect(rows.length).toBe(2);
expect(rows[0].source_id).toBe('default');
expect(rows[0].title).toBe('Default-source variant');
expect(rows[1].source_id).toBe('testsrc');
expect(rows[1].title).toBe('Testsrc-source variant');
});
test('putPage without opts.sourceId still targets source=default (back-compat)', async () => {
// Call again under default to verify the no-opts path still hits the same
// (default, slug) row rather than fabricating a duplicate.
const updated = await engine.putPage(SLUG, {
type: 'concept',
title: 'Default-source updated',
compiled_truth: 'Updated content.',
});
expect(updated.title).toBe('Default-source updated');
const rows = await engine.executeRaw<{ source_id: string; title: string }>(
`SELECT source_id, title FROM pages WHERE slug = $1 ORDER BY source_id`,
[SLUG],
);
// Still exactly two rows — no duplicate fabricated.
expect(rows.length).toBe(2);
expect(rows.find(r => r.source_id === 'default')!.title).toBe('Default-source updated');
expect(rows.find(r => r.source_id === 'testsrc')!.title).toBe('Testsrc-source variant');
});
});
describe('Per-page tx methods source-qualify their bare-slug subqueries', () => {
test('getTags(slug, { sourceId }) returns scoped tags without 21000', async () => {
// Pre-fix: this call would crash because the bare-slug subquery
// `(SELECT id FROM pages WHERE slug = $1)` matched both rows.
await engine.addTag(SLUG, 'shared-by-default', { sourceId: 'default' });
await engine.addTag(SLUG, 'unique-to-testsrc', { sourceId: 'testsrc' });
await engine.addTag(SLUG, 'also-shared', { sourceId: 'default' });
await engine.addTag(SLUG, 'also-shared', { sourceId: 'testsrc' });
const defaultTags = await engine.getTags(SLUG, { sourceId: 'default' });
expect(defaultTags.sort()).toEqual(['also-shared', 'shared-by-default']);
const testsrcTags = await engine.getTags(SLUG, { sourceId: 'testsrc' });
expect(testsrcTags.sort()).toEqual(['also-shared', 'unique-to-testsrc']);
});
test('removeTag(slug, tag, { sourceId }) only removes from one source', async () => {
await engine.removeTag(SLUG, 'also-shared', { sourceId: 'testsrc' });
expect((await engine.getTags(SLUG, { sourceId: 'default' })).sort())
.toEqual(['also-shared', 'shared-by-default']);
expect((await engine.getTags(SLUG, { sourceId: 'testsrc' })).sort())
.toEqual(['unique-to-testsrc']);
});
test('deleteChunks(slug, { sourceId }) only deletes one source\'s chunks', async () => {
await engine.upsertChunks(SLUG, [
{ chunk_index: 0, chunk_text: 'default chunk 0', chunk_source: 'compiled_truth' },
], { sourceId: 'default' });
await engine.upsertChunks(SLUG, [
{ chunk_index: 0, chunk_text: 'testsrc chunk 0', chunk_source: 'compiled_truth' },
], { sourceId: 'testsrc' });
const beforeRows = await engine.executeRaw<{ source_id: string; chunk_text: string }>(
`SELECT p.source_id, cc.chunk_text
FROM content_chunks cc
JOIN pages p ON p.id = cc.page_id
WHERE p.slug = $1
ORDER BY p.source_id`,
[SLUG],
);
expect(beforeRows.length).toBe(2);
await engine.deleteChunks(SLUG, { sourceId: 'testsrc' });
const afterRows = await engine.executeRaw<{ source_id: string; chunk_text: string }>(
`SELECT p.source_id, cc.chunk_text
FROM content_chunks cc
JOIN pages p ON p.id = cc.page_id
WHERE p.slug = $1`,
[SLUG],
);
expect(afterRows.length).toBe(1);
expect(afterRows[0].source_id).toBe('default');
});
test('createVersion(slug, { sourceId }) snapshots the right row', async () => {
const v = await engine.createVersion(SLUG, { sourceId: 'testsrc' });
expect(v).toBeDefined();
const rows = await engine.executeRaw<{ source_id: string; compiled_truth: string }>(
`SELECT p.source_id, pv.compiled_truth
FROM page_versions pv
JOIN pages p ON p.id = pv.page_id
WHERE p.slug = $1
ORDER BY pv.snapshot_at DESC
LIMIT 1`,
[SLUG],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('testsrc');
expect(rows[0].compiled_truth).toBe('Lives under source=testsrc.');
});
});
describe('addLink rewrites the cross-product into a source-qualified JOIN', () => {
const FROM_SLUG = 'topics/regression-link-from';
const TO_SLUG = 'topics/regression-link-to';
test('addLink with opts.{from,to,origin}SourceId targets the right rows', async () => {
// Set up: same (from, to) slug pair under both default and testsrc.
await engine.putPage(FROM_SLUG, { type: 'concept', title: 'F default', compiled_truth: '' });
await engine.putPage(TO_SLUG, { type: 'concept', title: 'T default', compiled_truth: '' });
await engine.putPage(FROM_SLUG, { type: 'concept', title: 'F testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
await engine.putPage(TO_SLUG, { type: 'concept', title: 'T testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
// Add an edge under testsrc only.
await engine.addLink(
FROM_SLUG, TO_SLUG, 'testsrc edge', 'documents', 'markdown', undefined, undefined,
{ fromSourceId: 'testsrc', toSourceId: 'testsrc', originSourceId: 'testsrc' },
);
// Verify the link's endpoints both point at the testsrc rows, not the
// default rows. Pre-fix, the cross-product `FROM pages f, pages t` would
// pick whichever order Postgres returned; the source filter eliminates
// that fan-out.
const rows = await engine.executeRaw<{ from_src: string; to_src: string; context: string }>(
`SELECT f.source_id AS from_src, t.source_id AS to_src, l.context
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
WHERE l.context = 'testsrc edge'`,
);
expect(rows.length).toBe(1);
expect(rows[0].from_src).toBe('testsrc');
expect(rows[0].to_src).toBe('testsrc');
});
test('addLink with no opts defaults to source=default (back-compat)', async () => {
await engine.addLink(
FROM_SLUG, TO_SLUG, 'default edge', 'documents', 'markdown',
);
const rows = await engine.executeRaw<{ from_src: string; to_src: string }>(
`SELECT f.source_id AS from_src, t.source_id AS to_src
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
WHERE l.context = 'default edge'`,
);
expect(rows.length).toBe(1);
expect(rows[0].from_src).toBe('default');
expect(rows[0].to_src).toBe('default');
});
test('addLink fails fast when the source-qualified endpoint doesn\'t exist', async () => {
// Pre-fix: cross-product would silently fall back to the wrong source
// pair and succeed. Post-fix: missing-source-row → no JOIN match → no row
// inserted → INTERSECT pre-check throws.
let err: Error | null = null;
try {
await engine.addLink(
FROM_SLUG, TO_SLUG, 'phantom edge', 'documents', 'markdown', undefined, undefined,
{ fromSourceId: 'nonexistent-src', toSourceId: 'nonexistent-src' },
);
} catch (e) {
err = e as Error;
}
expect(err).not.toBeNull();
expect(err!.message).toMatch(/not found/);
});
});
describe('importFromContent threads sourceId through the entire transaction body', () => {
const IMP_SLUG = 'topics/regression-import-thread';
test('importFromContent under source=testsrc does not fabricate a (default, slug) duplicate', async () => {
// Pre-seed a default-source row at the same slug to prove the fix actually
// discriminates: pre-fix, importing under testsrc would have ALSO touched
// the default row (or duplicated it) and the bare-slug getTags inside the
// tx would crash with 21000.
await engine.putPage(IMP_SLUG, {
type: 'concept',
title: 'Default-source seed',
compiled_truth: 'pre-existing default row',
});
const md = `---
type: concept
title: Imported under testsrc
---
# Imported under testsrc
Body content; tags get reconciled inside the transaction.
`;
// No 21000, no duplicate. Pre-fix this call would have either crashed
// mid-tx (rolling back) OR fabricated a third row at (default, slug).
const result = await importFromContent(engine, IMP_SLUG, md, {
noEmbed: true,
sourceId: 'testsrc',
});
expect(result.status).toBe('imported');
const rows = await engine.executeRaw<{ source_id: string; title: string }>(
`SELECT source_id, title FROM pages WHERE slug = $1 ORDER BY source_id`,
[IMP_SLUG],
);
expect(rows.length).toBe(2);
expect(rows[0].source_id).toBe('default');
expect(rows[0].title).toBe('Default-source seed');
expect(rows[1].source_id).toBe('testsrc');
expect(rows[1].title).toBe('Imported under testsrc');
});
test('re-importing same content under same sourceId is idempotent (status=skipped)', async () => {
const md = `---
type: concept
title: Imported under testsrc
---
# Imported under testsrc
Body content; tags get reconciled inside the transaction.
`;
const result = await importFromContent(engine, IMP_SLUG, md, {
noEmbed: true,
sourceId: 'testsrc',
});
expect(result.status).toBe('skipped');
});
});
describe('addTimelineEntry source-scoping (Data R1 HIGH 2 fix)', () => {
const TL_SLUG = 'topics/regression-timeline';
test('addTimelineEntry with opts.sourceId only writes to the intended source', async () => {
// Set up: same slug under both default and testsrc.
await engine.putPage(TL_SLUG, { type: 'concept', title: 'TL default', compiled_truth: '' });
await engine.putPage(TL_SLUG, { type: 'concept', title: 'TL testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
// Pre-fix: bare-slug `INSERT ... SELECT id FROM pages WHERE slug = $1`
// would have inserted timeline rows for BOTH source rows, fanning out
// the entry across sources.
await engine.addTimelineEntry(TL_SLUG, {
date: '2026-05-07',
source: 'test',
summary: 'testsrc-only entry',
detail: 'Should land only under testsrc.',
}, { sourceId: 'testsrc' });
const rows = await engine.executeRaw<{ source_id: string; summary: string }>(
`SELECT p.source_id, te.summary
FROM timeline_entries te
JOIN pages p ON p.id = te.page_id
WHERE p.slug = $1`,
[TL_SLUG],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('testsrc');
expect(rows[0].summary).toBe('testsrc-only entry');
});
test('addTimelineEntry rejects missing source-qualified page', async () => {
let err: Error | null = null;
try {
await engine.addTimelineEntry(TL_SLUG, {
date: '2026-05-08',
source: 'test',
summary: 'bad source',
detail: '',
}, { sourceId: 'nonexistent-src' });
} catch (e) {
err = e as Error;
}
expect(err).not.toBeNull();
expect(err!.message).toMatch(/not found/);
});
test('addTimelineEntry without opts defaults to source=default (back-compat)', async () => {
await engine.addTimelineEntry(TL_SLUG, {
date: '2026-05-09',
source: 'test',
summary: 'default-source entry',
detail: '',
});
const rows = await engine.executeRaw<{ source_id: string; summary: string }>(
`SELECT p.source_id, te.summary
FROM timeline_entries te
JOIN pages p ON p.id = te.page_id
WHERE p.slug = $1 AND te.summary = 'default-source entry'`,
[TL_SLUG],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('default');
});
});
describe('deletePage + updateSlug source-scoping (Data R2 CRITICAL + HIGH fix)', () => {
const DEL_SLUG = 'topics/regression-delete';
const REN_FROM = 'topics/regression-rename-from';
const REN_TO = 'topics/regression-rename-to';
test('deletePage with opts.sourceId only deletes the intended source row', async () => {
// Set up: same slug under both default and testsrc.
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'D default', compiled_truth: '' });
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'D testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
// Pre-fix: bare `DELETE FROM pages WHERE slug = $1` would have hard-deleted
// BOTH rows across sources. Post-fix: only the testsrc row goes.
await engine.deletePage(DEL_SLUG, { sourceId: 'testsrc' });
const rows = await engine.executeRaw<{ source_id: string }>(
`SELECT source_id FROM pages WHERE slug = $1`,
[DEL_SLUG],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('default');
});
test('deletePage without opts targets source=default only (back-compat)', async () => {
// Recreate the testsrc row to test that default-source delete leaves it.
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'D testsrc back', compiled_truth: '' }, { sourceId: 'testsrc' });
await engine.deletePage(DEL_SLUG); // no opts → defaults to 'default'
const rows = await engine.executeRaw<{ source_id: string }>(
`SELECT source_id FROM pages WHERE slug = $1`,
[DEL_SLUG],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('testsrc');
});
test('updateSlug with opts.sourceId only renames the intended source row', async () => {
// Set up: same slug under both default and testsrc.
await engine.putPage(REN_FROM, { type: 'concept', title: 'R default', compiled_truth: '' });
await engine.putPage(REN_FROM, { type: 'concept', title: 'R testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
// Pre-fix: bare `UPDATE pages SET slug = $new WHERE slug = $old` would have
// hit both rows; if REN_TO already existed in either source, the (source_id,
// slug) UNIQUE would fail. Post-fix: only the testsrc row gets renamed.
await engine.updateSlug(REN_FROM, REN_TO, { sourceId: 'testsrc' });
const fromRows = await engine.executeRaw<{ source_id: string }>(
`SELECT source_id FROM pages WHERE slug = $1 ORDER BY source_id`,
[REN_FROM],
);
expect(fromRows.length).toBe(1);
expect(fromRows[0].source_id).toBe('default');
const toRows = await engine.executeRaw<{ source_id: string }>(
`SELECT source_id FROM pages WHERE slug = $1`,
[REN_TO],
);
expect(toRows.length).toBe(1);
expect(toRows[0].source_id).toBe('testsrc');
});
test('getChunks with opts.sourceId only returns the intended source\'s chunks', async () => {
// Set up: same slug under both default and testsrc, each with distinct chunks.
const CHUNK_SLUG = 'topics/regression-getchunks';
await engine.putPage(CHUNK_SLUG, { type: 'concept', title: 'C default', compiled_truth: '' });
await engine.putPage(CHUNK_SLUG, { type: 'concept', title: 'C testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
await engine.upsertChunks(CHUNK_SLUG, [
{ chunk_index: 0, chunk_text: 'default chunk text', chunk_source: 'compiled_truth' },
], { sourceId: 'default' });
await engine.upsertChunks(CHUNK_SLUG, [
{ chunk_index: 0, chunk_text: 'testsrc chunk text', chunk_source: 'compiled_truth' },
], { sourceId: 'testsrc' });
// Pre-fix: bare-slug `WHERE p.slug = $1` returned BOTH source's chunks
// mashed together. importCodeFile uses getChunks for incremental embedding
// reuse; pre-fix would have grabbed the wrong source's embeddings.
const defaultChunks = await engine.getChunks(CHUNK_SLUG, { sourceId: 'default' });
expect(defaultChunks.length).toBe(1);
expect(defaultChunks[0].chunk_text).toBe('default chunk text');
const testsrcChunks = await engine.getChunks(CHUNK_SLUG, { sourceId: 'testsrc' });
expect(testsrcChunks.length).toBe(1);
expect(testsrcChunks[0].chunk_text).toBe('testsrc chunk text');
});
test('updateSlug without opts targets source=default only (back-compat)', async () => {
// Default still has REN_FROM. Rename it without opts; testsrc REN_TO
// already exists, so a bare rename would fail (source_id, slug) UNIQUE
// when both default and testsrc converge on REN_TO. Source-scoped rename
// succeeds because testsrc is untouched.
const REN_TO_2 = 'topics/regression-rename-to-2';
await engine.updateSlug(REN_FROM, REN_TO_2);
const rows = await engine.executeRaw<{ source_id: string; slug: string }>(
`SELECT source_id, slug FROM pages WHERE slug IN ($1, $2) ORDER BY source_id`,
[REN_FROM, REN_TO_2],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('default');
expect(rows[0].slug).toBe(REN_TO_2);
});
});
// ─────────────────────────────────────────────────────────────────────────
// v0.31.8 — op-handler-layer threading (D7 + D11 + D16 + D20 + D21 + D22)
//
// The pre-v0.31.8 op handlers in src/core/operations.ts called engine methods
// bare-slug, ignoring ctx.sourceId. Result: a remote MCP token whose
// ctx.sourceId='X' calling put_page / add_tag / get_links / etc. silently
// landed on source 'default'. This block drives the actual op handlers
// (operations.ts) — not just the engine surface — through a mock
// OperationContext carrying sourceId='X' and asserts only the X-source row
// is mutated/read.
// ─────────────────────────────────────────────────────────────────────────
import { operations } from '../src/core/operations.ts';
import type { OperationContext } from '../src/core/operations.ts';
function makeCtx(eng: PGLiteEngine, overrides: Partial<OperationContext> = {}): OperationContext {
return {
engine: eng as unknown as OperationContext['engine'],
config: { engine: 'pglite' } as never,
logger: { info: () => {}, warn: () => {}, error: () => {} },
dryRun: false,
remote: false,
sourceId: 'default',
...overrides,
};
}
function getOp(name: string) {
const op = operations.find(o => o.name === name);
if (!op) throw new Error(`op not registered: ${name}`);
return op;
}
describe('v0.31.8 op-handler ctx.sourceId threading', () => {
// Two-source fixture seeded fresh for this block. Use unique slugs so the
// earlier engine-layer suite's leftover state doesn't pollute assertions.
const TAG_SLUG = 'topics/op-tag-target';
beforeAll(async () => {
// Page exists at BOTH sources (the v0.18.0 supported state).
await engine.putPage(TAG_SLUG, {
type: 'concept', title: 'Default tag target', compiled_truth: '.',
});
await engine.putPage(TAG_SLUG, {
type: 'concept', title: 'Testsrc tag target', compiled_truth: '.',
}, { sourceId: 'testsrc' });
});
test('add_tag handler with ctx.sourceId=testsrc tags only the testsrc row', async () => {
const op = getOp('add_tag');
await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), { slug: TAG_SLUG, tag: 'op-handler-test-1' });
const rows = await engine.executeRaw<{ source_id: string }>(
`SELECT p.source_id FROM tags t JOIN pages p ON p.id = t.page_id
WHERE p.slug = $1 AND t.tag = $2`,
[TAG_SLUG, 'op-handler-test-1'],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('testsrc');
});
test('add_tag handler without ctx.sourceId tags the default row (back-compat)', async () => {
const op = getOp('add_tag');
await op.handler(makeCtx(engine), { slug: TAG_SLUG, tag: 'op-handler-test-2' });
const rows = await engine.executeRaw<{ source_id: string }>(
`SELECT p.source_id FROM tags t JOIN pages p ON p.id = t.page_id
WHERE p.slug = $1 AND t.tag = $2`,
[TAG_SLUG, 'op-handler-test-2'],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('default');
});
test('get_tags handler with ctx.sourceId=testsrc returns only testsrc tags', async () => {
// Both rows should now have one tag each from the two tests above.
const op = getOp('get_tags');
const tags = await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), { slug: TAG_SLUG }) as string[];
expect(tags).toContain('op-handler-test-1');
expect(tags).not.toContain('op-handler-test-2');
});
test('get_tags handler without ctx.sourceId returns default-source tags (back-compat)', async () => {
const op = getOp('get_tags');
const tags = await op.handler(makeCtx(engine), { slug: TAG_SLUG }) as string[];
// getTags is a v0.18.0-era source-aware method: it already defaults to
// source='default' when opts is omitted (see engine.ts addTag/removeTag/
// getTags comment). It does NOT use the D16 two-branch pattern — the
// pre-v0.31.8 behavior for tags on multi-source brains was always
// "scoped to default unless told otherwise." So with ctx.sourceId unset,
// only the default-source tag surfaces. (D16 two-branch applies to
// getLinks/getBacklinks/getTimeline/getRawData/getVersions/getAllSlugs/
// revertToVersion — the methods that pre-D12 had no source filter at all.)
expect(tags).toContain('op-handler-test-2');
expect(tags).not.toContain('op-handler-test-1');
});
test('add_link handler with ctx.sourceId scopes both endpoints', async () => {
// Seed a target page at both sources so addLink's INTERSECT pre-check passes.
const TARGET = 'topics/op-link-target';
await engine.putPage(TARGET, { type: 'concept', title: 'Default target', compiled_truth: '.' });
await engine.putPage(TARGET, { type: 'concept', title: 'Testsrc target', compiled_truth: '.' }, { sourceId: 'testsrc' });
const op = getOp('add_link');
await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), {
from: TAG_SLUG, to: TARGET, link_type: 'mentions', context: 'op-test',
});
const rows = await engine.executeRaw<{ from_source: string; to_source: string }>(
`SELECT f.source_id AS from_source, t.source_id AS to_source
FROM links l JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
WHERE f.slug = $1 AND t.slug = $2 AND l.link_type = 'mentions'`,
[TAG_SLUG, TARGET],
);
expect(rows.length).toBe(1);
expect(rows[0].from_source).toBe('testsrc');
expect(rows[0].to_source).toBe('testsrc');
});
test('get_links handler scopes to ctx.sourceId; default source view (v0.34 STEP 0)', async () => {
const op = getOp('get_links');
const scoped = await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), { slug: TAG_SLUG }) as Array<{ to_slug: string }>;
const defaultCtx = await op.handler(makeCtx(engine), { slug: TAG_SLUG }) as Array<{ to_slug: string }>;
// testsrc has the link from add_link test above; the default-source view
// has none.
expect(scoped.length).toBeGreaterThanOrEqual(1);
// v0.34 STEP 0 (D4): OperationContext.sourceId is REQUIRED. makeCtx with
// no override falls back to 'default'. The pre-v0.34 back-compat
// "ctx.sourceId undefined → cross-source view" is gone by design —
// it's the exact cross-source-bleed bug class STEP 0 closed. Cross-
// source visibility is now an explicit caller decision (e.g. a sources
// admin running an explicit "all-sources" probe).
expect(defaultCtx.length).toBeLessThanOrEqual(scoped.length);
});
test('delete_page handler scopes to ctx.sourceId (soft-delete only the testsrc row)', async () => {
// Use a fresh slug so we don't impact other tests in this describe block.
const DEL_SLUG = 'topics/op-delete-target';
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'Default', compiled_truth: '.' });
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'Testsrc', compiled_truth: '.' }, { sourceId: 'testsrc' });
const op = getOp('delete_page');
await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), { slug: DEL_SLUG });
const rows = await engine.executeRaw<{ source_id: string; deleted_at: string | null }>(
`SELECT source_id, deleted_at FROM pages WHERE slug = $1 ORDER BY source_id`,
[DEL_SLUG],
);
expect(rows.length).toBe(2);
const def = rows.find(r => r.source_id === 'default')!;
const tst = rows.find(r => r.source_id === 'testsrc')!;
expect(def.deleted_at).toBeNull(); // default row untouched
expect(tst.deleted_at).not.toBeNull(); // testsrc row soft-deleted
});
test('put_raw_data handler threads ctx.sourceId (D21)', async () => {
const op = getOp('put_raw_data');
await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), {
slug: TAG_SLUG, source: 'unit-test', data: { variant: 'testsrc' },
});
// Read via the engine to assert which source row got the raw_data.
const rd = await engine.getRawData(TAG_SLUG, 'unit-test', { sourceId: 'testsrc' });
expect(rd.length).toBe(1);
expect((rd[0].data as { variant: string }).variant).toBe('testsrc');
// Default-source raw_data should be untouched.
const defRd = await engine.getRawData(TAG_SLUG, 'unit-test', { sourceId: 'default' });
expect(defRd.length).toBe(0);
});
});