Files
gbrain/test/source-id-tx-regression.test.ts
T
182900d071 v0.31.8 fix: multi-source threading + doctor wedge hint + voyage cap (P2 follow-ups) (#808)
* feat(multi-source): thread ctx.sourceId through op handlers + engine read-surface

Closes the multi-source threading gaps that the v0.31.1.1-fixwave codex
review caught. Multi-source brains were silently misrouting writes from
every CLI/MCP-driven op (put_page, add_tag, add_link, add_timeline_entry,
revert_version, put_raw_data, etc.) because the op handlers in
operations.ts ignored ctx.sourceId. Read-side ops were arbitrary-row
under same-slug-across-sources because the engine's read methods had no
source filter.

Engine layer (D12 + D16 + D21):
- engine.ts interface: getLinks/getBacklinks/getTimeline/getRawData/
  getVersions/getAllSlugs/revertToVersion/putRawData all take
  opts?: { sourceId?: string }.
- pglite-engine.ts + postgres-engine.ts: two-branch query for each
  read method. Without opts.sourceId, NO source filter applies
  (preserves pre-v0.31.8 cross-source semantics for back-link
  validators and any caller that hasn't threaded sourceId yet). With
  opts.sourceId, scoped to that source — the new path used by
  reconcileLinks and ctx.sourceId-aware op handlers.

Op-handler layer (D7 + D16 + D20):
- operations.ts threads ctx.sourceId through 16+ handler sites:
  put_page, revert_version, put_raw_data, add_tag, remove_tag,
  add_link, remove_link, add_timeline_entry, create_version,
  delete_page, restore_page, get_page, get_tags, get_links,
  get_backlinks, get_timeline, get_versions, get_raw_data,
  get_chunks, plus reconcileLinks's tx.getLinks/getBacklinks/
  addLink/removeLink and engine.getAllSlugs.
- Pattern: const sourceOpts = ctx.sourceId ? { sourceId: ctx.sourceId } : {};
  When ctx.sourceId is unset, engine falls through to cross-source
  view (back-compat). MCP callers populate ctx.sourceId via the
  transport layer.

CLI wiring (D11 + D22):
- cli.ts: makeContext is async, calls resolveSourceId() from
  src/core/source-resolver.ts:58 (the canonical 6-tier chain:
  --source flag → GBRAIN_SOURCE env → .gbrain-source dotfile →
  path-match → brain default → 'default'). Wrapped in try/catch
  so a fresh pre-init brain still returns a clean ctx with no
  sourceId set.
- commands/call.ts: runCall accepts --source <id> flag. Resolves
  through the same 6-tier chain and threads to handleToolCall
  via the new opts.sourceId param.
- mcp/server.ts: handleToolCall accepts opts.sourceId and threads
  to buildOperationContext.

Tests (D7 + D16 + D20 regression coverage):
- test/source-id-tx-regression.test.ts: 8 new op-handler-layer
  cases covering add_tag/get_tags/add_link/get_links/delete_page/
  put_raw_data routing under ctx.sourceId='X' vs unset, plus
  D16's two-branch back-compat invariant for getLinks (cross-
  source view preserved when ctx.sourceId is unset).

Closes the codex OV-1/OV-2/OV-3 findings from the v0.31.8 plan
review. Back-compat is strictly additive: callers that don't pass
opts.sourceId see the same results they did pre-v0.31.8.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(doctor): multi_source_drift check surfaces pre-v0.30.3 misroutes

Pre-v0.30.3 putPage misrouted multi-source writes from intended source X
to (default, slug). The fix-wave fixed forward-going writes but explicitly
deferred backfilling the misrouted rows. Operators have had no signal of
this silent corruption.

Adds src/core/multi-source-drift.ts exporting findMisroutedPages(engine,
sources, opts). The heuristic walks each non-default source's local_path
and surfaces slugs that exist at (default, slug) in DB but are MISSING
from (X, slug) — unambiguous evidence of the misroute shape.

Implementation notes (codex OV12 + OV13 + D17):
- FS walk handles BOTH .md and .mdx (matches src/core/sync.ts:133, which
  treats both as markdown). Walks own helper instead of importing from
  extract.ts so doctor doesn't crash if local_path is unreadable
  (try/catch on root statSync; ENOENT/EACCES yields zero files, NOT a
  thrown error that takes down doctor).
- Single batched SQL with VALUES clause: collect all candidate slugs
  into one array, then ONE LEFT JOIN against pages with source_id IN
  ('default', X). Materialize into Map<slug, Set<source_id>>. NOT a
  per-file 20K-round-trip loop.
- Bounded by limit (10K files) AND timeoutMs (5s). Bail with
  walk_truncated=true rather than letting doctor hang.
- Heuristic softened per OV12: "appears misrouted to default" with TWO
  possible causes flagged (pre-v0.30.3 misroute OR source X never
  completed initial sync). The doctor warning suggests verification
  ('gbrain sources status'), not a destructive action.

Wired into runDoctor (3b-multi-source slot, after sync_failures) AND
into doctorReportRemote (D14) so thin-client operators see the check
when 'gbrain doctor' routes through the remote MCP path. Single-source
brains skip the check entirely.

Tests: test/multi-source-drift.test.ts (7 PGLite cases) covers:
- Single-source brain → skip
- Multi-source no-misroutes → ok
- Multi-source 2 misrouted slugs → warn with sample
- Healthy same-slug-across-sources NOT a false positive (the codex
  OV4 redesign case — original heuristic would have false-positived)
- FS walk hits limit → walk_truncated=true
- Unreadable local_path doesn't crash
- .mdx files walked alongside .md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(doctor): wire multi_source_drift + wedge force-retry hint (D14 + D19)

Wires the new multi_source_drift check into both runDoctor (local) and
doctorReportRemote (thin-client remote MCP path), and extends the existing
minions_migration block to detect 3-consecutive-partials wedges and emit
gbrain apply-migrations --force-retry <v> hints (D19).

Pre-v0.31.8, operators wedged on v0.29.1 (or any future migration that
hits the apply-migrations runner's 3-consecutive-partials guard) got the
generic "Run: gbrain apply-migrations --yes" hint. That command refuses
to advance past the guard — so the hint was wrong. Codex OV-11 (and the
v0.31.1.1-fixwave commit message) flagged this, but the prior plan said
to delegate to apply-migrations.ts:statusForVersion(), which would have
re-opened a separate regression: the existing forward-progress override
at doctor.ts:303 (newer completion suppresses old partials) is
cross-version and statusForVersion is per-version only.

This commit extends the existing block in place rather than replacing it:

1. Keep the forward-progress override (lines 348-356) byte-identical so
   installs that moved past an old v0.11 partial don't light up with
   stale wedge alerts.
2. Add a 3-consecutive-partials detector after the stuck filter. Since
   `stuck` already excludes forward-progress-superseded versions, the
   wedge counter only fires on actual unresolved partials.
3. Branch the message:
   - wedged.length > 0 → "WEDGED MIGRATION(s): <v>. Run: gbrain
     apply-migrations --force-retry <v>" (chain with && for multiple)
   - else if stuck.length > 0 → existing --yes hint
   - else → no message

Same shape duplicated in doctorReportRemote so thin-client operators
see the right command on the brain host.

Plus the multi_source_drift wiring (D14): same heuristic from the
new src/core/multi-source-drift.ts library, called from both local and
remote doctor paths. Single-source brains skip. Engine-null guard on
the local path (--fast and DB-down branches pass null).

Tests: test/doctor.test.ts gains 4 wedge-hint regression cases:
- Both branches present in source (forward-progress override + 3-partials
  detection coexisting).
- Anti-regression guard: NO `import { statusForVersion }` from
  apply-migrations.ts. The prior plan would have introduced this
  import; keeping it out means doctor stays decoupled from the
  migration runner's per-version semantics.
- Multiple wedged versions chain force-retry calls with `&&`.
- Both branches present in doctorReportRemote (thin-client coverage,
  D14).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(voyage): Content-Length pre-check + per-item base64 cap (D2 + D10)

The voyage compat fetch wrapper at gateway.ts:294 called
\`await resp.clone().json()\` BEFORE iterating embeddings. A
malicious or compromised Voyage endpoint of arbitrary size was
fully parsed into the JS heap before any size check could fire.
The original v0.31.8 plan put the cap on per-item base64 length,
which fires AFTER the JSON parse — defeating the OOM defense
entirely (codex OV8).

Two-layer fix sized at MAX_VOYAGE_RESPONSE_BYTES = 256 MB
("unambiguously not legit" rather than tight against typical
batches; voyage-3-large × 16K embeddings ≈ 200 MB raw fits within
the cap):

Layer 1 (PRIMARY) — Content-Length header pre-check, fires
BEFORE resp.clone().json(). Throws a descriptive error if the
header reports a length over the cap. The JSON.parse OOM vector
is now gated.

Layer 2 (defense-in-depth) — per-embedding base64 length check
inside the iteration. Catches the rare case where Layer 1 was
skipped (chunked transfer encoding has no Content-Length) AND a
single embedding string is unreasonably large. Estimates decoded
size as 0.75 × base64 length (canonical base64 → bytes ratio).

Tests: test/voyage-response-cap.test.ts — 5 structural source-pin
cases including the critical D10 invariant: "Content-Length
pre-check appears BEFORE \`const json: any = await
resp.clone().json()\` in the inbound block". A future refactor
that moves the cap below the JSON parse fails this test loudly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.31.8)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ci): exclude *.serial.test.ts from sharded parallel run

scripts/test-shard.sh (the GitHub Actions runner) was including
*.serial.test.ts files alongside regular tests. Serial files use
top-level mock.module(...) which leaks across files in the same Bun
process — exactly what the .serial naming convention was meant to
quarantine.

Concretely: test/eval-takes-quality-runner.serial.test.ts mocks
src/core/ai/gateway.ts with `configureGateway: () => undefined`
(no-op). Because both files landed in shard 2, the mock leaked into
test/voyage-multimodal.test.ts: when its tests called
configureVoyageMultimodal() → configureGateway(), the no-op fired and
_config stayed null. Then embedMultimodal() called requireConfig()
which threw "AI gateway is not configured" — 18 tests failed at
gateway.ts:171 with [1.00ms] each.

Local fast loop (scripts/run-unit-shard.sh) already excludes
*.serial.test.ts AND *.slow.test.ts via the same find-arg pattern.
test-shard.sh just hadn't picked up the same exclusion when it was
written. This commit:

1. Mirrors run-unit-shard.sh's exclusion pattern in test-shard.sh
   (`-not -name '*.slow.test.ts' -not -name '*.serial.test.ts'`).
2. Adds a "Run *.serial.test.ts" step to .github/workflows/test.yml
   on shard 1 only, calling scripts/run-serial-tests.sh
   (--max-concurrency=1). Shard 1 already runs extra setup work
   (`bun run verify`), so it has the natural slot for the serial
   pass without slowing the parallel critical path.

Verified locally: shard 2 went from 18 voyage-multimodal failures to
0. Shard 2 file count: 81 → 78 (3 serial files removed). Total test
count after fix: 1438 (1437 pass + 1 pre-existing env-sensitive
warm-create speed gate flake — unrelated to v0.31.8 or this fix).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 13:33:05 -07:00

635 lines
28 KiB
TypeScript

/**
* v0.18.0+ Step 5+ regression — source_id threading through the per-page
* transaction surface (putPage / createVersion / getTags / addTag / removeTag /
* deleteChunks / upsertChunks / addLink / removeLink).
*
* Pre-fix bug:
* - putPage omitted source_id from its INSERT column list, so the schema
* DEFAULT 'default' was applied even when the caller meant to write under
* a non-default source (e.g. 'jarvis-memory'). When the same slug already
* existed under the intended source, putPage silently fabricated a
* duplicate row at (default, slug). Both rows then coexisted under the
* composite UNIQUE.
* - Subsequent bare-slug subqueries inside the same transaction —
* `(SELECT id FROM pages WHERE slug = $1)` in getTags / removeTag /
* deleteChunks / removeLink — returned 2 rows and crashed with Postgres
* 21000 ("more than one row returned by a subquery used as an expression"),
* rolling back the entire tx.
*
* Fix:
* - putPage adds source_id to the INSERT column list (defaults to 'default'
* when opts.sourceId is omitted, preserving back-compat).
* - Every bare-slug page-id subquery becomes source-qualified
* (`AND source_id = $X`), eliminating the multi-row fan-out.
* - addLink converts away from `FROM pages f, pages t` cross-product and
* mirrors addLinksBatch's VALUES + JOIN-on-(slug, source_id) shape.
*
* Backwards-compat: every method's opts param is optional. Existing callers
* that don't pass sourceId continue to target source 'default' (the schema
* default) and behave identically to pre-fix.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { runSources } from '../src/commands/sources.ts';
import { importFromContent } from '../src/core/import-file.ts';
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({ type: 'pglite' } as never);
await engine.initSchema();
// Add the second source up-front; tests below assume both 'default' and
// 'testsrc' exist.
await runSources(engine, ['add', 'testsrc', '--no-federated']);
}, 60_000);
afterAll(async () => {
if (engine) await engine.disconnect();
}, 60_000);
const SLUG = 'topics/source-id-regression';
describe('putPage threads source_id into the INSERT column list', () => {
test('putPage with opts.sourceId writes under the intended source', async () => {
await engine.putPage(SLUG, {
type: 'concept',
title: 'Default-source variant',
compiled_truth: 'Lives under source=default.',
});
await engine.putPage(SLUG, {
type: 'concept',
title: 'Testsrc-source variant',
compiled_truth: 'Lives under source=testsrc.',
}, { sourceId: 'testsrc' });
const rows = await engine.executeRaw<{ source_id: string; title: string }>(
`SELECT source_id, title FROM pages WHERE slug = $1 ORDER BY source_id`,
[SLUG],
);
expect(rows.length).toBe(2);
expect(rows[0].source_id).toBe('default');
expect(rows[0].title).toBe('Default-source variant');
expect(rows[1].source_id).toBe('testsrc');
expect(rows[1].title).toBe('Testsrc-source variant');
});
test('putPage without opts.sourceId still targets source=default (back-compat)', async () => {
// Call again under default to verify the no-opts path still hits the same
// (default, slug) row rather than fabricating a duplicate.
const updated = await engine.putPage(SLUG, {
type: 'concept',
title: 'Default-source updated',
compiled_truth: 'Updated content.',
});
expect(updated.title).toBe('Default-source updated');
const rows = await engine.executeRaw<{ source_id: string; title: string }>(
`SELECT source_id, title FROM pages WHERE slug = $1 ORDER BY source_id`,
[SLUG],
);
// Still exactly two rows — no duplicate fabricated.
expect(rows.length).toBe(2);
expect(rows.find(r => r.source_id === 'default')!.title).toBe('Default-source updated');
expect(rows.find(r => r.source_id === 'testsrc')!.title).toBe('Testsrc-source variant');
});
});
describe('Per-page tx methods source-qualify their bare-slug subqueries', () => {
test('getTags(slug, { sourceId }) returns scoped tags without 21000', async () => {
// Pre-fix: this call would crash because the bare-slug subquery
// `(SELECT id FROM pages WHERE slug = $1)` matched both rows.
await engine.addTag(SLUG, 'shared-by-default', { sourceId: 'default' });
await engine.addTag(SLUG, 'unique-to-testsrc', { sourceId: 'testsrc' });
await engine.addTag(SLUG, 'also-shared', { sourceId: 'default' });
await engine.addTag(SLUG, 'also-shared', { sourceId: 'testsrc' });
const defaultTags = await engine.getTags(SLUG, { sourceId: 'default' });
expect(defaultTags.sort()).toEqual(['also-shared', 'shared-by-default']);
const testsrcTags = await engine.getTags(SLUG, { sourceId: 'testsrc' });
expect(testsrcTags.sort()).toEqual(['also-shared', 'unique-to-testsrc']);
});
test('removeTag(slug, tag, { sourceId }) only removes from one source', async () => {
await engine.removeTag(SLUG, 'also-shared', { sourceId: 'testsrc' });
expect((await engine.getTags(SLUG, { sourceId: 'default' })).sort())
.toEqual(['also-shared', 'shared-by-default']);
expect((await engine.getTags(SLUG, { sourceId: 'testsrc' })).sort())
.toEqual(['unique-to-testsrc']);
});
test('deleteChunks(slug, { sourceId }) only deletes one source\'s chunks', async () => {
await engine.upsertChunks(SLUG, [
{ chunk_index: 0, chunk_text: 'default chunk 0', chunk_source: 'compiled_truth' },
], { sourceId: 'default' });
await engine.upsertChunks(SLUG, [
{ chunk_index: 0, chunk_text: 'testsrc chunk 0', chunk_source: 'compiled_truth' },
], { sourceId: 'testsrc' });
const beforeRows = await engine.executeRaw<{ source_id: string; chunk_text: string }>(
`SELECT p.source_id, cc.chunk_text
FROM content_chunks cc
JOIN pages p ON p.id = cc.page_id
WHERE p.slug = $1
ORDER BY p.source_id`,
[SLUG],
);
expect(beforeRows.length).toBe(2);
await engine.deleteChunks(SLUG, { sourceId: 'testsrc' });
const afterRows = await engine.executeRaw<{ source_id: string; chunk_text: string }>(
`SELECT p.source_id, cc.chunk_text
FROM content_chunks cc
JOIN pages p ON p.id = cc.page_id
WHERE p.slug = $1`,
[SLUG],
);
expect(afterRows.length).toBe(1);
expect(afterRows[0].source_id).toBe('default');
});
test('createVersion(slug, { sourceId }) snapshots the right row', async () => {
const v = await engine.createVersion(SLUG, { sourceId: 'testsrc' });
expect(v).toBeDefined();
const rows = await engine.executeRaw<{ source_id: string; compiled_truth: string }>(
`SELECT p.source_id, pv.compiled_truth
FROM page_versions pv
JOIN pages p ON p.id = pv.page_id
WHERE p.slug = $1
ORDER BY pv.snapshot_at DESC
LIMIT 1`,
[SLUG],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('testsrc');
expect(rows[0].compiled_truth).toBe('Lives under source=testsrc.');
});
});
describe('addLink rewrites the cross-product into a source-qualified JOIN', () => {
const FROM_SLUG = 'topics/regression-link-from';
const TO_SLUG = 'topics/regression-link-to';
test('addLink with opts.{from,to,origin}SourceId targets the right rows', async () => {
// Set up: same (from, to) slug pair under both default and testsrc.
await engine.putPage(FROM_SLUG, { type: 'concept', title: 'F default', compiled_truth: '' });
await engine.putPage(TO_SLUG, { type: 'concept', title: 'T default', compiled_truth: '' });
await engine.putPage(FROM_SLUG, { type: 'concept', title: 'F testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
await engine.putPage(TO_SLUG, { type: 'concept', title: 'T testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
// Add an edge under testsrc only.
await engine.addLink(
FROM_SLUG, TO_SLUG, 'testsrc edge', 'documents', 'markdown', undefined, undefined,
{ fromSourceId: 'testsrc', toSourceId: 'testsrc', originSourceId: 'testsrc' },
);
// Verify the link's endpoints both point at the testsrc rows, not the
// default rows. Pre-fix, the cross-product `FROM pages f, pages t` would
// pick whichever order Postgres returned; the source filter eliminates
// that fan-out.
const rows = await engine.executeRaw<{ from_src: string; to_src: string; context: string }>(
`SELECT f.source_id AS from_src, t.source_id AS to_src, l.context
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
WHERE l.context = 'testsrc edge'`,
);
expect(rows.length).toBe(1);
expect(rows[0].from_src).toBe('testsrc');
expect(rows[0].to_src).toBe('testsrc');
});
test('addLink with no opts defaults to source=default (back-compat)', async () => {
await engine.addLink(
FROM_SLUG, TO_SLUG, 'default edge', 'documents', 'markdown',
);
const rows = await engine.executeRaw<{ from_src: string; to_src: string }>(
`SELECT f.source_id AS from_src, t.source_id AS to_src
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
WHERE l.context = 'default edge'`,
);
expect(rows.length).toBe(1);
expect(rows[0].from_src).toBe('default');
expect(rows[0].to_src).toBe('default');
});
test('addLink fails fast when the source-qualified endpoint doesn\'t exist', async () => {
// Pre-fix: cross-product would silently fall back to the wrong source
// pair and succeed. Post-fix: missing-source-row → no JOIN match → no row
// inserted → INTERSECT pre-check throws.
let err: Error | null = null;
try {
await engine.addLink(
FROM_SLUG, TO_SLUG, 'phantom edge', 'documents', 'markdown', undefined, undefined,
{ fromSourceId: 'nonexistent-src', toSourceId: 'nonexistent-src' },
);
} catch (e) {
err = e as Error;
}
expect(err).not.toBeNull();
expect(err!.message).toMatch(/not found/);
});
});
describe('importFromContent threads sourceId through the entire transaction body', () => {
const IMP_SLUG = 'topics/regression-import-thread';
test('importFromContent under source=testsrc does not fabricate a (default, slug) duplicate', async () => {
// Pre-seed a default-source row at the same slug to prove the fix actually
// discriminates: pre-fix, importing under testsrc would have ALSO touched
// the default row (or duplicated it) and the bare-slug getTags inside the
// tx would crash with 21000.
await engine.putPage(IMP_SLUG, {
type: 'concept',
title: 'Default-source seed',
compiled_truth: 'pre-existing default row',
});
const md = `---
type: concept
title: Imported under testsrc
---
# Imported under testsrc
Body content; tags get reconciled inside the transaction.
`;
// No 21000, no duplicate. Pre-fix this call would have either crashed
// mid-tx (rolling back) OR fabricated a third row at (default, slug).
const result = await importFromContent(engine, IMP_SLUG, md, {
noEmbed: true,
sourceId: 'testsrc',
});
expect(result.status).toBe('imported');
const rows = await engine.executeRaw<{ source_id: string; title: string }>(
`SELECT source_id, title FROM pages WHERE slug = $1 ORDER BY source_id`,
[IMP_SLUG],
);
expect(rows.length).toBe(2);
expect(rows[0].source_id).toBe('default');
expect(rows[0].title).toBe('Default-source seed');
expect(rows[1].source_id).toBe('testsrc');
expect(rows[1].title).toBe('Imported under testsrc');
});
test('re-importing same content under same sourceId is idempotent (status=skipped)', async () => {
const md = `---
type: concept
title: Imported under testsrc
---
# Imported under testsrc
Body content; tags get reconciled inside the transaction.
`;
const result = await importFromContent(engine, IMP_SLUG, md, {
noEmbed: true,
sourceId: 'testsrc',
});
expect(result.status).toBe('skipped');
});
});
describe('addTimelineEntry source-scoping (Data R1 HIGH 2 fix)', () => {
const TL_SLUG = 'topics/regression-timeline';
test('addTimelineEntry with opts.sourceId only writes to the intended source', async () => {
// Set up: same slug under both default and testsrc.
await engine.putPage(TL_SLUG, { type: 'concept', title: 'TL default', compiled_truth: '' });
await engine.putPage(TL_SLUG, { type: 'concept', title: 'TL testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
// Pre-fix: bare-slug `INSERT ... SELECT id FROM pages WHERE slug = $1`
// would have inserted timeline rows for BOTH source rows, fanning out
// the entry across sources.
await engine.addTimelineEntry(TL_SLUG, {
date: '2026-05-07',
source: 'test',
summary: 'testsrc-only entry',
detail: 'Should land only under testsrc.',
}, { sourceId: 'testsrc' });
const rows = await engine.executeRaw<{ source_id: string; summary: string }>(
`SELECT p.source_id, te.summary
FROM timeline_entries te
JOIN pages p ON p.id = te.page_id
WHERE p.slug = $1`,
[TL_SLUG],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('testsrc');
expect(rows[0].summary).toBe('testsrc-only entry');
});
test('addTimelineEntry rejects missing source-qualified page', async () => {
let err: Error | null = null;
try {
await engine.addTimelineEntry(TL_SLUG, {
date: '2026-05-08',
source: 'test',
summary: 'bad source',
detail: '',
}, { sourceId: 'nonexistent-src' });
} catch (e) {
err = e as Error;
}
expect(err).not.toBeNull();
expect(err!.message).toMatch(/not found/);
});
test('addTimelineEntry without opts defaults to source=default (back-compat)', async () => {
await engine.addTimelineEntry(TL_SLUG, {
date: '2026-05-09',
source: 'test',
summary: 'default-source entry',
detail: '',
});
const rows = await engine.executeRaw<{ source_id: string; summary: string }>(
`SELECT p.source_id, te.summary
FROM timeline_entries te
JOIN pages p ON p.id = te.page_id
WHERE p.slug = $1 AND te.summary = 'default-source entry'`,
[TL_SLUG],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('default');
});
});
describe('deletePage + updateSlug source-scoping (Data R2 CRITICAL + HIGH fix)', () => {
const DEL_SLUG = 'topics/regression-delete';
const REN_FROM = 'topics/regression-rename-from';
const REN_TO = 'topics/regression-rename-to';
test('deletePage with opts.sourceId only deletes the intended source row', async () => {
// Set up: same slug under both default and testsrc.
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'D default', compiled_truth: '' });
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'D testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
// Pre-fix: bare `DELETE FROM pages WHERE slug = $1` would have hard-deleted
// BOTH rows across sources. Post-fix: only the testsrc row goes.
await engine.deletePage(DEL_SLUG, { sourceId: 'testsrc' });
const rows = await engine.executeRaw<{ source_id: string }>(
`SELECT source_id FROM pages WHERE slug = $1`,
[DEL_SLUG],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('default');
});
test('deletePage without opts targets source=default only (back-compat)', async () => {
// Recreate the testsrc row to test that default-source delete leaves it.
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'D testsrc back', compiled_truth: '' }, { sourceId: 'testsrc' });
await engine.deletePage(DEL_SLUG); // no opts → defaults to 'default'
const rows = await engine.executeRaw<{ source_id: string }>(
`SELECT source_id FROM pages WHERE slug = $1`,
[DEL_SLUG],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('testsrc');
});
test('updateSlug with opts.sourceId only renames the intended source row', async () => {
// Set up: same slug under both default and testsrc.
await engine.putPage(REN_FROM, { type: 'concept', title: 'R default', compiled_truth: '' });
await engine.putPage(REN_FROM, { type: 'concept', title: 'R testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
// Pre-fix: bare `UPDATE pages SET slug = $new WHERE slug = $old` would have
// hit both rows; if REN_TO already existed in either source, the (source_id,
// slug) UNIQUE would fail. Post-fix: only the testsrc row gets renamed.
await engine.updateSlug(REN_FROM, REN_TO, { sourceId: 'testsrc' });
const fromRows = await engine.executeRaw<{ source_id: string }>(
`SELECT source_id FROM pages WHERE slug = $1 ORDER BY source_id`,
[REN_FROM],
);
expect(fromRows.length).toBe(1);
expect(fromRows[0].source_id).toBe('default');
const toRows = await engine.executeRaw<{ source_id: string }>(
`SELECT source_id FROM pages WHERE slug = $1`,
[REN_TO],
);
expect(toRows.length).toBe(1);
expect(toRows[0].source_id).toBe('testsrc');
});
test('getChunks with opts.sourceId only returns the intended source\'s chunks', async () => {
// Set up: same slug under both default and testsrc, each with distinct chunks.
const CHUNK_SLUG = 'topics/regression-getchunks';
await engine.putPage(CHUNK_SLUG, { type: 'concept', title: 'C default', compiled_truth: '' });
await engine.putPage(CHUNK_SLUG, { type: 'concept', title: 'C testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
await engine.upsertChunks(CHUNK_SLUG, [
{ chunk_index: 0, chunk_text: 'default chunk text', chunk_source: 'compiled_truth' },
], { sourceId: 'default' });
await engine.upsertChunks(CHUNK_SLUG, [
{ chunk_index: 0, chunk_text: 'testsrc chunk text', chunk_source: 'compiled_truth' },
], { sourceId: 'testsrc' });
// Pre-fix: bare-slug `WHERE p.slug = $1` returned BOTH source's chunks
// mashed together. importCodeFile uses getChunks for incremental embedding
// reuse; pre-fix would have grabbed the wrong source's embeddings.
const defaultChunks = await engine.getChunks(CHUNK_SLUG, { sourceId: 'default' });
expect(defaultChunks.length).toBe(1);
expect(defaultChunks[0].chunk_text).toBe('default chunk text');
const testsrcChunks = await engine.getChunks(CHUNK_SLUG, { sourceId: 'testsrc' });
expect(testsrcChunks.length).toBe(1);
expect(testsrcChunks[0].chunk_text).toBe('testsrc chunk text');
});
test('updateSlug without opts targets source=default only (back-compat)', async () => {
// Default still has REN_FROM. Rename it without opts; testsrc REN_TO
// already exists, so a bare rename would fail (source_id, slug) UNIQUE
// when both default and testsrc converge on REN_TO. Source-scoped rename
// succeeds because testsrc is untouched.
const REN_TO_2 = 'topics/regression-rename-to-2';
await engine.updateSlug(REN_FROM, REN_TO_2);
const rows = await engine.executeRaw<{ source_id: string; slug: string }>(
`SELECT source_id, slug FROM pages WHERE slug IN ($1, $2) ORDER BY source_id`,
[REN_FROM, REN_TO_2],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('default');
expect(rows[0].slug).toBe(REN_TO_2);
});
});
// ─────────────────────────────────────────────────────────────────────────
// v0.31.8 — op-handler-layer threading (D7 + D11 + D16 + D20 + D21 + D22)
//
// The pre-v0.31.8 op handlers in src/core/operations.ts called engine methods
// bare-slug, ignoring ctx.sourceId. Result: a remote MCP token whose
// ctx.sourceId='X' calling put_page / add_tag / get_links / etc. silently
// landed on source 'default'. This block drives the actual op handlers
// (operations.ts) — not just the engine surface — through a mock
// OperationContext carrying sourceId='X' and asserts only the X-source row
// is mutated/read.
// ─────────────────────────────────────────────────────────────────────────
import { operations } from '../src/core/operations.ts';
import type { OperationContext } from '../src/core/operations.ts';
function makeCtx(eng: PGLiteEngine, overrides: Partial<OperationContext> = {}): OperationContext {
return {
engine: eng as unknown as OperationContext['engine'],
config: { engine: 'pglite' } as never,
logger: { info: () => {}, warn: () => {}, error: () => {} },
dryRun: false,
remote: false,
...overrides,
};
}
function getOp(name: string) {
const op = operations.find(o => o.name === name);
if (!op) throw new Error(`op not registered: ${name}`);
return op;
}
describe('v0.31.8 op-handler ctx.sourceId threading', () => {
// Two-source fixture seeded fresh for this block. Use unique slugs so the
// earlier engine-layer suite's leftover state doesn't pollute assertions.
const TAG_SLUG = 'topics/op-tag-target';
beforeAll(async () => {
// Page exists at BOTH sources (the v0.18.0 supported state).
await engine.putPage(TAG_SLUG, {
type: 'concept', title: 'Default tag target', compiled_truth: '.',
});
await engine.putPage(TAG_SLUG, {
type: 'concept', title: 'Testsrc tag target', compiled_truth: '.',
}, { sourceId: 'testsrc' });
});
test('add_tag handler with ctx.sourceId=testsrc tags only the testsrc row', async () => {
const op = getOp('add_tag');
await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), { slug: TAG_SLUG, tag: 'op-handler-test-1' });
const rows = await engine.executeRaw<{ source_id: string }>(
`SELECT p.source_id FROM tags t JOIN pages p ON p.id = t.page_id
WHERE p.slug = $1 AND t.tag = $2`,
[TAG_SLUG, 'op-handler-test-1'],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('testsrc');
});
test('add_tag handler without ctx.sourceId tags the default row (back-compat)', async () => {
const op = getOp('add_tag');
await op.handler(makeCtx(engine), { slug: TAG_SLUG, tag: 'op-handler-test-2' });
const rows = await engine.executeRaw<{ source_id: string }>(
`SELECT p.source_id FROM tags t JOIN pages p ON p.id = t.page_id
WHERE p.slug = $1 AND t.tag = $2`,
[TAG_SLUG, 'op-handler-test-2'],
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('default');
});
test('get_tags handler with ctx.sourceId=testsrc returns only testsrc tags', async () => {
// Both rows should now have one tag each from the two tests above.
const op = getOp('get_tags');
const tags = await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), { slug: TAG_SLUG }) as string[];
expect(tags).toContain('op-handler-test-1');
expect(tags).not.toContain('op-handler-test-2');
});
test('get_tags handler without ctx.sourceId returns default-source tags (back-compat)', async () => {
const op = getOp('get_tags');
const tags = await op.handler(makeCtx(engine), { slug: TAG_SLUG }) as string[];
// getTags is a v0.18.0-era source-aware method: it already defaults to
// source='default' when opts is omitted (see engine.ts addTag/removeTag/
// getTags comment). It does NOT use the D16 two-branch pattern — the
// pre-v0.31.8 behavior for tags on multi-source brains was always
// "scoped to default unless told otherwise." So with ctx.sourceId unset,
// only the default-source tag surfaces. (D16 two-branch applies to
// getLinks/getBacklinks/getTimeline/getRawData/getVersions/getAllSlugs/
// revertToVersion — the methods that pre-D12 had no source filter at all.)
expect(tags).toContain('op-handler-test-2');
expect(tags).not.toContain('op-handler-test-1');
});
test('add_link handler with ctx.sourceId scopes both endpoints', async () => {
// Seed a target page at both sources so addLink's INTERSECT pre-check passes.
const TARGET = 'topics/op-link-target';
await engine.putPage(TARGET, { type: 'concept', title: 'Default target', compiled_truth: '.' });
await engine.putPage(TARGET, { type: 'concept', title: 'Testsrc target', compiled_truth: '.' }, { sourceId: 'testsrc' });
const op = getOp('add_link');
await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), {
from: TAG_SLUG, to: TARGET, link_type: 'mentions', context: 'op-test',
});
const rows = await engine.executeRaw<{ from_source: string; to_source: string }>(
`SELECT f.source_id AS from_source, t.source_id AS to_source
FROM links l JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
WHERE f.slug = $1 AND t.slug = $2 AND l.link_type = 'mentions'`,
[TAG_SLUG, TARGET],
);
expect(rows.length).toBe(1);
expect(rows[0].from_source).toBe('testsrc');
expect(rows[0].to_source).toBe('testsrc');
});
test('get_links handler scopes to ctx.sourceId; back-compat cross-source view preserved (D16)', async () => {
const op = getOp('get_links');
const scoped = await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), { slug: TAG_SLUG }) as Array<{ to_slug: string }>;
const cross = await op.handler(makeCtx(engine), { slug: TAG_SLUG }) as Array<{ to_slug: string }>;
// testsrc has the link from add_link test above. default has none.
expect(scoped.length).toBeGreaterThanOrEqual(1);
// Cross-source view sees at least the same edges (and would see default's
// if we'd seeded any). Under the two-branch back-compat path, this is the
// pre-v0.31.8 semantic — no source filter on the engine join.
expect(cross.length).toBeGreaterThanOrEqual(scoped.length);
});
test('delete_page handler scopes to ctx.sourceId (soft-delete only the testsrc row)', async () => {
// Use a fresh slug so we don't impact other tests in this describe block.
const DEL_SLUG = 'topics/op-delete-target';
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'Default', compiled_truth: '.' });
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'Testsrc', compiled_truth: '.' }, { sourceId: 'testsrc' });
const op = getOp('delete_page');
await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), { slug: DEL_SLUG });
const rows = await engine.executeRaw<{ source_id: string; deleted_at: string | null }>(
`SELECT source_id, deleted_at FROM pages WHERE slug = $1 ORDER BY source_id`,
[DEL_SLUG],
);
expect(rows.length).toBe(2);
const def = rows.find(r => r.source_id === 'default')!;
const tst = rows.find(r => r.source_id === 'testsrc')!;
expect(def.deleted_at).toBeNull(); // default row untouched
expect(tst.deleted_at).not.toBeNull(); // testsrc row soft-deleted
});
test('put_raw_data handler threads ctx.sourceId (D21)', async () => {
const op = getOp('put_raw_data');
await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), {
slug: TAG_SLUG, source: 'unit-test', data: { variant: 'testsrc' },
});
// Read via the engine to assert which source row got the raw_data.
const rd = await engine.getRawData(TAG_SLUG, 'unit-test', { sourceId: 'testsrc' });
expect(rd.length).toBe(1);
expect((rd[0].data as { variant: string }).variant).toBe('testsrc');
// Default-source raw_data should be untouched.
const defRd = await engine.getRawData(TAG_SLUG, 'unit-test', { sourceId: 'default' });
expect(defRd.length).toBe(0);
});
});