mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
* feat(multi-source): thread ctx.sourceId through op handlers + engine read-surface
Closes the multi-source threading gaps that the v0.31.1.1-fixwave codex
review caught. Multi-source brains were silently misrouting writes from
every CLI/MCP-driven op (put_page, add_tag, add_link, add_timeline_entry,
revert_version, put_raw_data, etc.) because the op handlers in
operations.ts ignored ctx.sourceId. Read-side ops were arbitrary-row
under same-slug-across-sources because the engine's read methods had no
source filter.
Engine layer (D12 + D16 + D21):
- engine.ts interface: getLinks/getBacklinks/getTimeline/getRawData/
getVersions/getAllSlugs/revertToVersion/putRawData all take
opts?: { sourceId?: string }.
- pglite-engine.ts + postgres-engine.ts: two-branch query for each
read method. Without opts.sourceId, NO source filter applies
(preserves pre-v0.31.8 cross-source semantics for back-link
validators and any caller that hasn't threaded sourceId yet). With
opts.sourceId, scoped to that source — the new path used by
reconcileLinks and ctx.sourceId-aware op handlers.
Op-handler layer (D7 + D16 + D20):
- operations.ts threads ctx.sourceId through 16+ handler sites:
put_page, revert_version, put_raw_data, add_tag, remove_tag,
add_link, remove_link, add_timeline_entry, create_version,
delete_page, restore_page, get_page, get_tags, get_links,
get_backlinks, get_timeline, get_versions, get_raw_data,
get_chunks, plus reconcileLinks's tx.getLinks/getBacklinks/
addLink/removeLink and engine.getAllSlugs.
- Pattern: const sourceOpts = ctx.sourceId ? { sourceId: ctx.sourceId } : {};
When ctx.sourceId is unset, engine falls through to cross-source
view (back-compat). MCP callers populate ctx.sourceId via the
transport layer.
CLI wiring (D11 + D22):
- cli.ts: makeContext is async, calls resolveSourceId() from
src/core/source-resolver.ts:58 (the canonical 6-tier chain:
--source flag → GBRAIN_SOURCE env → .gbrain-source dotfile →
path-match → brain default → 'default'). Wrapped in try/catch
so a fresh pre-init brain still returns a clean ctx with no
sourceId set.
- commands/call.ts: runCall accepts --source <id> flag. Resolves
through the same 6-tier chain and threads to handleToolCall
via the new opts.sourceId param.
- mcp/server.ts: handleToolCall accepts opts.sourceId and threads
to buildOperationContext.
Tests (D7 + D16 + D20 regression coverage):
- test/source-id-tx-regression.test.ts: 8 new op-handler-layer
cases covering add_tag/get_tags/add_link/get_links/delete_page/
put_raw_data routing under ctx.sourceId='X' vs unset, plus
D16's two-branch back-compat invariant for getLinks (cross-
source view preserved when ctx.sourceId is unset).
Closes the codex OV-1/OV-2/OV-3 findings from the v0.31.8 plan
review. Back-compat is strictly additive: callers that don't pass
opts.sourceId see the same results they did pre-v0.31.8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(doctor): multi_source_drift check surfaces pre-v0.30.3 misroutes
Pre-v0.30.3 putPage misrouted multi-source writes from intended source X
to (default, slug). The fix-wave fixed forward-going writes but explicitly
deferred backfilling the misrouted rows. Operators have had no signal of
this silent corruption.
Adds src/core/multi-source-drift.ts exporting findMisroutedPages(engine,
sources, opts). The heuristic walks each non-default source's local_path
and surfaces slugs that exist at (default, slug) in DB but are MISSING
from (X, slug) — unambiguous evidence of the misroute shape.
Implementation notes (codex OV12 + OV13 + D17):
- FS walk handles BOTH .md and .mdx (matches src/core/sync.ts:133, which
treats both as markdown). Walks own helper instead of importing from
extract.ts so doctor doesn't crash if local_path is unreadable
(try/catch on root statSync; ENOENT/EACCES yields zero files, NOT a
thrown error that takes down doctor).
- Single batched SQL with VALUES clause: collect all candidate slugs
into one array, then ONE LEFT JOIN against pages with source_id IN
('default', X). Materialize into Map<slug, Set<source_id>>. NOT a
per-file 20K-round-trip loop.
- Bounded by limit (10K files) AND timeoutMs (5s). Bail with
walk_truncated=true rather than letting doctor hang.
- Heuristic softened per OV12: "appears misrouted to default" with TWO
possible causes flagged (pre-v0.30.3 misroute OR source X never
completed initial sync). The doctor warning suggests verification
('gbrain sources status'), not a destructive action.
Wired into runDoctor (3b-multi-source slot, after sync_failures) AND
into doctorReportRemote (D14) so thin-client operators see the check
when 'gbrain doctor' routes through the remote MCP path. Single-source
brains skip the check entirely.
Tests: test/multi-source-drift.test.ts (7 PGLite cases) covers:
- Single-source brain → skip
- Multi-source no-misroutes → ok
- Multi-source 2 misrouted slugs → warn with sample
- Healthy same-slug-across-sources NOT a false positive (the codex
OV4 redesign case — original heuristic would have false-positived)
- FS walk hits limit → walk_truncated=true
- Unreadable local_path doesn't crash
- .mdx files walked alongside .md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(doctor): wire multi_source_drift + wedge force-retry hint (D14 + D19)
Wires the new multi_source_drift check into both runDoctor (local) and
doctorReportRemote (thin-client remote MCP path), and extends the existing
minions_migration block to detect 3-consecutive-partials wedges and emit
gbrain apply-migrations --force-retry <v> hints (D19).
Pre-v0.31.8, operators wedged on v0.29.1 (or any future migration that
hits the apply-migrations runner's 3-consecutive-partials guard) got the
generic "Run: gbrain apply-migrations --yes" hint. That command refuses
to advance past the guard — so the hint was wrong. Codex OV-11 (and the
v0.31.1.1-fixwave commit message) flagged this, but the prior plan said
to delegate to apply-migrations.ts:statusForVersion(), which would have
re-opened a separate regression: the existing forward-progress override
at doctor.ts:303 (newer completion suppresses old partials) is
cross-version and statusForVersion is per-version only.
This commit extends the existing block in place rather than replacing it:
1. Keep the forward-progress override (lines 348-356) byte-identical so
installs that moved past an old v0.11 partial don't light up with
stale wedge alerts.
2. Add a 3-consecutive-partials detector after the stuck filter. Since
`stuck` already excludes forward-progress-superseded versions, the
wedge counter only fires on actual unresolved partials.
3. Branch the message:
- wedged.length > 0 → "WEDGED MIGRATION(s): <v>. Run: gbrain
apply-migrations --force-retry <v>" (chain with && for multiple)
- else if stuck.length > 0 → existing --yes hint
- else → no message
Same shape duplicated in doctorReportRemote so thin-client operators
see the right command on the brain host.
Plus the multi_source_drift wiring (D14): same heuristic from the
new src/core/multi-source-drift.ts library, called from both local and
remote doctor paths. Single-source brains skip. Engine-null guard on
the local path (--fast and DB-down branches pass null).
Tests: test/doctor.test.ts gains 4 wedge-hint regression cases:
- Both branches present in source (forward-progress override + 3-partials
detection coexisting).
- Anti-regression guard: NO `import { statusForVersion }` from
apply-migrations.ts. The prior plan would have introduced this
import; keeping it out means doctor stays decoupled from the
migration runner's per-version semantics.
- Multiple wedged versions chain force-retry calls with `&&`.
- Both branches present in doctorReportRemote (thin-client coverage,
D14).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(voyage): Content-Length pre-check + per-item base64 cap (D2 + D10)
The voyage compat fetch wrapper at gateway.ts:294 called
\`await resp.clone().json()\` BEFORE iterating embeddings. A
malicious or compromised Voyage endpoint of arbitrary size was
fully parsed into the JS heap before any size check could fire.
The original v0.31.8 plan put the cap on per-item base64 length,
which fires AFTER the JSON parse — defeating the OOM defense
entirely (codex OV8).
Two-layer fix sized at MAX_VOYAGE_RESPONSE_BYTES = 256 MB
("unambiguously not legit" rather than tight against typical
batches; voyage-3-large × 16K embeddings ≈ 200 MB raw fits within
the cap):
Layer 1 (PRIMARY) — Content-Length header pre-check, fires
BEFORE resp.clone().json(). Throws a descriptive error if the
header reports a length over the cap. The JSON.parse OOM vector
is now gated.
Layer 2 (defense-in-depth) — per-embedding base64 length check
inside the iteration. Catches the rare case where Layer 1 was
skipped (chunked transfer encoding has no Content-Length) AND a
single embedding string is unreasonably large. Estimates decoded
size as 0.75 × base64 length (canonical base64 → bytes ratio).
Tests: test/voyage-response-cap.test.ts — 5 structural source-pin
cases including the critical D10 invariant: "Content-Length
pre-check appears BEFORE \`const json: any = await
resp.clone().json()\` in the inbound block". A future refactor
that moves the cap below the JSON parse fails this test loudly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.31.8)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ci): exclude *.serial.test.ts from sharded parallel run
scripts/test-shard.sh (the GitHub Actions runner) was including
*.serial.test.ts files alongside regular tests. Serial files use
top-level mock.module(...) which leaks across files in the same Bun
process — exactly what the .serial naming convention was meant to
quarantine.
Concretely: test/eval-takes-quality-runner.serial.test.ts mocks
src/core/ai/gateway.ts with `configureGateway: () => undefined`
(no-op). Because both files landed in shard 2, the mock leaked into
test/voyage-multimodal.test.ts: when its tests called
configureVoyageMultimodal() → configureGateway(), the no-op fired and
_config stayed null. Then embedMultimodal() called requireConfig()
which threw "AI gateway is not configured" — 18 tests failed at
gateway.ts:171 with [1.00ms] each.
Local fast loop (scripts/run-unit-shard.sh) already excludes
*.serial.test.ts AND *.slow.test.ts via the same find-arg pattern.
test-shard.sh just hadn't picked up the same exclusion when it was
written. This commit:
1. Mirrors run-unit-shard.sh's exclusion pattern in test-shard.sh
(`-not -name '*.slow.test.ts' -not -name '*.serial.test.ts'`).
2. Adds a "Run *.serial.test.ts" step to .github/workflows/test.yml
on shard 1 only, calling scripts/run-serial-tests.sh
(--max-concurrency=1). Shard 1 already runs extra setup work
(`bun run verify`), so it has the natural slot for the serial
pass without slowing the parallel critical path.
Verified locally: shard 2 went from 18 voyage-multimodal failures to
0. Shard 2 file count: 81 → 78 (3 serial files removed). Total test
count after fix: 1438 (1437 pass + 1 pre-existing env-sensitive
warm-create speed gate flake — unrelated to v0.31.8 or this fix).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
635 lines
28 KiB
TypeScript
635 lines
28 KiB
TypeScript
/**
|
|
* v0.18.0+ Step 5+ regression — source_id threading through the per-page
|
|
* transaction surface (putPage / createVersion / getTags / addTag / removeTag /
|
|
* deleteChunks / upsertChunks / addLink / removeLink).
|
|
*
|
|
* Pre-fix bug:
|
|
* - putPage omitted source_id from its INSERT column list, so the schema
|
|
* DEFAULT 'default' was applied even when the caller meant to write under
|
|
* a non-default source (e.g. 'jarvis-memory'). When the same slug already
|
|
* existed under the intended source, putPage silently fabricated a
|
|
* duplicate row at (default, slug). Both rows then coexisted under the
|
|
* composite UNIQUE.
|
|
* - Subsequent bare-slug subqueries inside the same transaction —
|
|
* `(SELECT id FROM pages WHERE slug = $1)` in getTags / removeTag /
|
|
* deleteChunks / removeLink — returned 2 rows and crashed with Postgres
|
|
* 21000 ("more than one row returned by a subquery used as an expression"),
|
|
* rolling back the entire tx.
|
|
*
|
|
* Fix:
|
|
* - putPage adds source_id to the INSERT column list (defaults to 'default'
|
|
* when opts.sourceId is omitted, preserving back-compat).
|
|
* - Every bare-slug page-id subquery becomes source-qualified
|
|
* (`AND source_id = $X`), eliminating the multi-row fan-out.
|
|
* - addLink converts away from `FROM pages f, pages t` cross-product and
|
|
* mirrors addLinksBatch's VALUES + JOIN-on-(slug, source_id) shape.
|
|
*
|
|
* Backwards-compat: every method's opts param is optional. Existing callers
|
|
* that don't pass sourceId continue to target source 'default' (the schema
|
|
* default) and behave identically to pre-fix.
|
|
*/
|
|
|
|
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
|
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
|
import { runSources } from '../src/commands/sources.ts';
|
|
import { importFromContent } from '../src/core/import-file.ts';
|
|
|
|
let engine: PGLiteEngine;
|
|
|
|
beforeAll(async () => {
|
|
engine = new PGLiteEngine();
|
|
await engine.connect({ type: 'pglite' } as never);
|
|
await engine.initSchema();
|
|
// Add the second source up-front; tests below assume both 'default' and
|
|
// 'testsrc' exist.
|
|
await runSources(engine, ['add', 'testsrc', '--no-federated']);
|
|
}, 60_000);
|
|
|
|
afterAll(async () => {
|
|
if (engine) await engine.disconnect();
|
|
}, 60_000);
|
|
|
|
const SLUG = 'topics/source-id-regression';
|
|
|
|
describe('putPage threads source_id into the INSERT column list', () => {
|
|
test('putPage with opts.sourceId writes under the intended source', async () => {
|
|
await engine.putPage(SLUG, {
|
|
type: 'concept',
|
|
title: 'Default-source variant',
|
|
compiled_truth: 'Lives under source=default.',
|
|
});
|
|
await engine.putPage(SLUG, {
|
|
type: 'concept',
|
|
title: 'Testsrc-source variant',
|
|
compiled_truth: 'Lives under source=testsrc.',
|
|
}, { sourceId: 'testsrc' });
|
|
|
|
const rows = await engine.executeRaw<{ source_id: string; title: string }>(
|
|
`SELECT source_id, title FROM pages WHERE slug = $1 ORDER BY source_id`,
|
|
[SLUG],
|
|
);
|
|
expect(rows.length).toBe(2);
|
|
expect(rows[0].source_id).toBe('default');
|
|
expect(rows[0].title).toBe('Default-source variant');
|
|
expect(rows[1].source_id).toBe('testsrc');
|
|
expect(rows[1].title).toBe('Testsrc-source variant');
|
|
});
|
|
|
|
test('putPage without opts.sourceId still targets source=default (back-compat)', async () => {
|
|
// Call again under default to verify the no-opts path still hits the same
|
|
// (default, slug) row rather than fabricating a duplicate.
|
|
const updated = await engine.putPage(SLUG, {
|
|
type: 'concept',
|
|
title: 'Default-source updated',
|
|
compiled_truth: 'Updated content.',
|
|
});
|
|
expect(updated.title).toBe('Default-source updated');
|
|
|
|
const rows = await engine.executeRaw<{ source_id: string; title: string }>(
|
|
`SELECT source_id, title FROM pages WHERE slug = $1 ORDER BY source_id`,
|
|
[SLUG],
|
|
);
|
|
// Still exactly two rows — no duplicate fabricated.
|
|
expect(rows.length).toBe(2);
|
|
expect(rows.find(r => r.source_id === 'default')!.title).toBe('Default-source updated');
|
|
expect(rows.find(r => r.source_id === 'testsrc')!.title).toBe('Testsrc-source variant');
|
|
});
|
|
});
|
|
|
|
describe('Per-page tx methods source-qualify their bare-slug subqueries', () => {
|
|
test('getTags(slug, { sourceId }) returns scoped tags without 21000', async () => {
|
|
// Pre-fix: this call would crash because the bare-slug subquery
|
|
// `(SELECT id FROM pages WHERE slug = $1)` matched both rows.
|
|
await engine.addTag(SLUG, 'shared-by-default', { sourceId: 'default' });
|
|
await engine.addTag(SLUG, 'unique-to-testsrc', { sourceId: 'testsrc' });
|
|
await engine.addTag(SLUG, 'also-shared', { sourceId: 'default' });
|
|
await engine.addTag(SLUG, 'also-shared', { sourceId: 'testsrc' });
|
|
|
|
const defaultTags = await engine.getTags(SLUG, { sourceId: 'default' });
|
|
expect(defaultTags.sort()).toEqual(['also-shared', 'shared-by-default']);
|
|
|
|
const testsrcTags = await engine.getTags(SLUG, { sourceId: 'testsrc' });
|
|
expect(testsrcTags.sort()).toEqual(['also-shared', 'unique-to-testsrc']);
|
|
});
|
|
|
|
test('removeTag(slug, tag, { sourceId }) only removes from one source', async () => {
|
|
await engine.removeTag(SLUG, 'also-shared', { sourceId: 'testsrc' });
|
|
expect((await engine.getTags(SLUG, { sourceId: 'default' })).sort())
|
|
.toEqual(['also-shared', 'shared-by-default']);
|
|
expect((await engine.getTags(SLUG, { sourceId: 'testsrc' })).sort())
|
|
.toEqual(['unique-to-testsrc']);
|
|
});
|
|
|
|
test('deleteChunks(slug, { sourceId }) only deletes one source\'s chunks', async () => {
|
|
await engine.upsertChunks(SLUG, [
|
|
{ chunk_index: 0, chunk_text: 'default chunk 0', chunk_source: 'compiled_truth' },
|
|
], { sourceId: 'default' });
|
|
await engine.upsertChunks(SLUG, [
|
|
{ chunk_index: 0, chunk_text: 'testsrc chunk 0', chunk_source: 'compiled_truth' },
|
|
], { sourceId: 'testsrc' });
|
|
|
|
const beforeRows = await engine.executeRaw<{ source_id: string; chunk_text: string }>(
|
|
`SELECT p.source_id, cc.chunk_text
|
|
FROM content_chunks cc
|
|
JOIN pages p ON p.id = cc.page_id
|
|
WHERE p.slug = $1
|
|
ORDER BY p.source_id`,
|
|
[SLUG],
|
|
);
|
|
expect(beforeRows.length).toBe(2);
|
|
|
|
await engine.deleteChunks(SLUG, { sourceId: 'testsrc' });
|
|
|
|
const afterRows = await engine.executeRaw<{ source_id: string; chunk_text: string }>(
|
|
`SELECT p.source_id, cc.chunk_text
|
|
FROM content_chunks cc
|
|
JOIN pages p ON p.id = cc.page_id
|
|
WHERE p.slug = $1`,
|
|
[SLUG],
|
|
);
|
|
expect(afterRows.length).toBe(1);
|
|
expect(afterRows[0].source_id).toBe('default');
|
|
});
|
|
|
|
test('createVersion(slug, { sourceId }) snapshots the right row', async () => {
|
|
const v = await engine.createVersion(SLUG, { sourceId: 'testsrc' });
|
|
expect(v).toBeDefined();
|
|
const rows = await engine.executeRaw<{ source_id: string; compiled_truth: string }>(
|
|
`SELECT p.source_id, pv.compiled_truth
|
|
FROM page_versions pv
|
|
JOIN pages p ON p.id = pv.page_id
|
|
WHERE p.slug = $1
|
|
ORDER BY pv.snapshot_at DESC
|
|
LIMIT 1`,
|
|
[SLUG],
|
|
);
|
|
expect(rows.length).toBe(1);
|
|
expect(rows[0].source_id).toBe('testsrc');
|
|
expect(rows[0].compiled_truth).toBe('Lives under source=testsrc.');
|
|
});
|
|
});
|
|
|
|
describe('addLink rewrites the cross-product into a source-qualified JOIN', () => {
|
|
const FROM_SLUG = 'topics/regression-link-from';
|
|
const TO_SLUG = 'topics/regression-link-to';
|
|
|
|
test('addLink with opts.{from,to,origin}SourceId targets the right rows', async () => {
|
|
// Set up: same (from, to) slug pair under both default and testsrc.
|
|
await engine.putPage(FROM_SLUG, { type: 'concept', title: 'F default', compiled_truth: '' });
|
|
await engine.putPage(TO_SLUG, { type: 'concept', title: 'T default', compiled_truth: '' });
|
|
await engine.putPage(FROM_SLUG, { type: 'concept', title: 'F testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
|
|
await engine.putPage(TO_SLUG, { type: 'concept', title: 'T testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
|
|
|
|
// Add an edge under testsrc only.
|
|
await engine.addLink(
|
|
FROM_SLUG, TO_SLUG, 'testsrc edge', 'documents', 'markdown', undefined, undefined,
|
|
{ fromSourceId: 'testsrc', toSourceId: 'testsrc', originSourceId: 'testsrc' },
|
|
);
|
|
|
|
// Verify the link's endpoints both point at the testsrc rows, not the
|
|
// default rows. Pre-fix, the cross-product `FROM pages f, pages t` would
|
|
// pick whichever order Postgres returned; the source filter eliminates
|
|
// that fan-out.
|
|
const rows = await engine.executeRaw<{ from_src: string; to_src: string; context: string }>(
|
|
`SELECT f.source_id AS from_src, t.source_id AS to_src, l.context
|
|
FROM links l
|
|
JOIN pages f ON f.id = l.from_page_id
|
|
JOIN pages t ON t.id = l.to_page_id
|
|
WHERE l.context = 'testsrc edge'`,
|
|
);
|
|
expect(rows.length).toBe(1);
|
|
expect(rows[0].from_src).toBe('testsrc');
|
|
expect(rows[0].to_src).toBe('testsrc');
|
|
});
|
|
|
|
test('addLink with no opts defaults to source=default (back-compat)', async () => {
|
|
await engine.addLink(
|
|
FROM_SLUG, TO_SLUG, 'default edge', 'documents', 'markdown',
|
|
);
|
|
const rows = await engine.executeRaw<{ from_src: string; to_src: string }>(
|
|
`SELECT f.source_id AS from_src, t.source_id AS to_src
|
|
FROM links l
|
|
JOIN pages f ON f.id = l.from_page_id
|
|
JOIN pages t ON t.id = l.to_page_id
|
|
WHERE l.context = 'default edge'`,
|
|
);
|
|
expect(rows.length).toBe(1);
|
|
expect(rows[0].from_src).toBe('default');
|
|
expect(rows[0].to_src).toBe('default');
|
|
});
|
|
|
|
test('addLink fails fast when the source-qualified endpoint doesn\'t exist', async () => {
|
|
// Pre-fix: cross-product would silently fall back to the wrong source
|
|
// pair and succeed. Post-fix: missing-source-row → no JOIN match → no row
|
|
// inserted → INTERSECT pre-check throws.
|
|
let err: Error | null = null;
|
|
try {
|
|
await engine.addLink(
|
|
FROM_SLUG, TO_SLUG, 'phantom edge', 'documents', 'markdown', undefined, undefined,
|
|
{ fromSourceId: 'nonexistent-src', toSourceId: 'nonexistent-src' },
|
|
);
|
|
} catch (e) {
|
|
err = e as Error;
|
|
}
|
|
expect(err).not.toBeNull();
|
|
expect(err!.message).toMatch(/not found/);
|
|
});
|
|
});
|
|
|
|
describe('importFromContent threads sourceId through the entire transaction body', () => {
|
|
const IMP_SLUG = 'topics/regression-import-thread';
|
|
|
|
test('importFromContent under source=testsrc does not fabricate a (default, slug) duplicate', async () => {
|
|
// Pre-seed a default-source row at the same slug to prove the fix actually
|
|
// discriminates: pre-fix, importing under testsrc would have ALSO touched
|
|
// the default row (or duplicated it) and the bare-slug getTags inside the
|
|
// tx would crash with 21000.
|
|
await engine.putPage(IMP_SLUG, {
|
|
type: 'concept',
|
|
title: 'Default-source seed',
|
|
compiled_truth: 'pre-existing default row',
|
|
});
|
|
|
|
const md = `---
|
|
type: concept
|
|
title: Imported under testsrc
|
|
---
|
|
|
|
# Imported under testsrc
|
|
|
|
Body content; tags get reconciled inside the transaction.
|
|
`;
|
|
|
|
// No 21000, no duplicate. Pre-fix this call would have either crashed
|
|
// mid-tx (rolling back) OR fabricated a third row at (default, slug).
|
|
const result = await importFromContent(engine, IMP_SLUG, md, {
|
|
noEmbed: true,
|
|
sourceId: 'testsrc',
|
|
});
|
|
expect(result.status).toBe('imported');
|
|
|
|
const rows = await engine.executeRaw<{ source_id: string; title: string }>(
|
|
`SELECT source_id, title FROM pages WHERE slug = $1 ORDER BY source_id`,
|
|
[IMP_SLUG],
|
|
);
|
|
expect(rows.length).toBe(2);
|
|
expect(rows[0].source_id).toBe('default');
|
|
expect(rows[0].title).toBe('Default-source seed');
|
|
expect(rows[1].source_id).toBe('testsrc');
|
|
expect(rows[1].title).toBe('Imported under testsrc');
|
|
});
|
|
|
|
test('re-importing same content under same sourceId is idempotent (status=skipped)', async () => {
|
|
const md = `---
|
|
type: concept
|
|
title: Imported under testsrc
|
|
---
|
|
|
|
# Imported under testsrc
|
|
|
|
Body content; tags get reconciled inside the transaction.
|
|
`;
|
|
const result = await importFromContent(engine, IMP_SLUG, md, {
|
|
noEmbed: true,
|
|
sourceId: 'testsrc',
|
|
});
|
|
expect(result.status).toBe('skipped');
|
|
});
|
|
});
|
|
|
|
describe('addTimelineEntry source-scoping (Data R1 HIGH 2 fix)', () => {
|
|
const TL_SLUG = 'topics/regression-timeline';
|
|
|
|
test('addTimelineEntry with opts.sourceId only writes to the intended source', async () => {
|
|
// Set up: same slug under both default and testsrc.
|
|
await engine.putPage(TL_SLUG, { type: 'concept', title: 'TL default', compiled_truth: '' });
|
|
await engine.putPage(TL_SLUG, { type: 'concept', title: 'TL testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
|
|
|
|
// Pre-fix: bare-slug `INSERT ... SELECT id FROM pages WHERE slug = $1`
|
|
// would have inserted timeline rows for BOTH source rows, fanning out
|
|
// the entry across sources.
|
|
await engine.addTimelineEntry(TL_SLUG, {
|
|
date: '2026-05-07',
|
|
source: 'test',
|
|
summary: 'testsrc-only entry',
|
|
detail: 'Should land only under testsrc.',
|
|
}, { sourceId: 'testsrc' });
|
|
|
|
const rows = await engine.executeRaw<{ source_id: string; summary: string }>(
|
|
`SELECT p.source_id, te.summary
|
|
FROM timeline_entries te
|
|
JOIN pages p ON p.id = te.page_id
|
|
WHERE p.slug = $1`,
|
|
[TL_SLUG],
|
|
);
|
|
expect(rows.length).toBe(1);
|
|
expect(rows[0].source_id).toBe('testsrc');
|
|
expect(rows[0].summary).toBe('testsrc-only entry');
|
|
});
|
|
|
|
test('addTimelineEntry rejects missing source-qualified page', async () => {
|
|
let err: Error | null = null;
|
|
try {
|
|
await engine.addTimelineEntry(TL_SLUG, {
|
|
date: '2026-05-08',
|
|
source: 'test',
|
|
summary: 'bad source',
|
|
detail: '',
|
|
}, { sourceId: 'nonexistent-src' });
|
|
} catch (e) {
|
|
err = e as Error;
|
|
}
|
|
expect(err).not.toBeNull();
|
|
expect(err!.message).toMatch(/not found/);
|
|
});
|
|
|
|
test('addTimelineEntry without opts defaults to source=default (back-compat)', async () => {
|
|
await engine.addTimelineEntry(TL_SLUG, {
|
|
date: '2026-05-09',
|
|
source: 'test',
|
|
summary: 'default-source entry',
|
|
detail: '',
|
|
});
|
|
|
|
const rows = await engine.executeRaw<{ source_id: string; summary: string }>(
|
|
`SELECT p.source_id, te.summary
|
|
FROM timeline_entries te
|
|
JOIN pages p ON p.id = te.page_id
|
|
WHERE p.slug = $1 AND te.summary = 'default-source entry'`,
|
|
[TL_SLUG],
|
|
);
|
|
expect(rows.length).toBe(1);
|
|
expect(rows[0].source_id).toBe('default');
|
|
});
|
|
});
|
|
|
|
describe('deletePage + updateSlug source-scoping (Data R2 CRITICAL + HIGH fix)', () => {
|
|
const DEL_SLUG = 'topics/regression-delete';
|
|
const REN_FROM = 'topics/regression-rename-from';
|
|
const REN_TO = 'topics/regression-rename-to';
|
|
|
|
test('deletePage with opts.sourceId only deletes the intended source row', async () => {
|
|
// Set up: same slug under both default and testsrc.
|
|
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'D default', compiled_truth: '' });
|
|
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'D testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
|
|
|
|
// Pre-fix: bare `DELETE FROM pages WHERE slug = $1` would have hard-deleted
|
|
// BOTH rows across sources. Post-fix: only the testsrc row goes.
|
|
await engine.deletePage(DEL_SLUG, { sourceId: 'testsrc' });
|
|
|
|
const rows = await engine.executeRaw<{ source_id: string }>(
|
|
`SELECT source_id FROM pages WHERE slug = $1`,
|
|
[DEL_SLUG],
|
|
);
|
|
expect(rows.length).toBe(1);
|
|
expect(rows[0].source_id).toBe('default');
|
|
});
|
|
|
|
test('deletePage without opts targets source=default only (back-compat)', async () => {
|
|
// Recreate the testsrc row to test that default-source delete leaves it.
|
|
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'D testsrc back', compiled_truth: '' }, { sourceId: 'testsrc' });
|
|
await engine.deletePage(DEL_SLUG); // no opts → defaults to 'default'
|
|
|
|
const rows = await engine.executeRaw<{ source_id: string }>(
|
|
`SELECT source_id FROM pages WHERE slug = $1`,
|
|
[DEL_SLUG],
|
|
);
|
|
expect(rows.length).toBe(1);
|
|
expect(rows[0].source_id).toBe('testsrc');
|
|
});
|
|
|
|
test('updateSlug with opts.sourceId only renames the intended source row', async () => {
|
|
// Set up: same slug under both default and testsrc.
|
|
await engine.putPage(REN_FROM, { type: 'concept', title: 'R default', compiled_truth: '' });
|
|
await engine.putPage(REN_FROM, { type: 'concept', title: 'R testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
|
|
|
|
// Pre-fix: bare `UPDATE pages SET slug = $new WHERE slug = $old` would have
|
|
// hit both rows; if REN_TO already existed in either source, the (source_id,
|
|
// slug) UNIQUE would fail. Post-fix: only the testsrc row gets renamed.
|
|
await engine.updateSlug(REN_FROM, REN_TO, { sourceId: 'testsrc' });
|
|
|
|
const fromRows = await engine.executeRaw<{ source_id: string }>(
|
|
`SELECT source_id FROM pages WHERE slug = $1 ORDER BY source_id`,
|
|
[REN_FROM],
|
|
);
|
|
expect(fromRows.length).toBe(1);
|
|
expect(fromRows[0].source_id).toBe('default');
|
|
|
|
const toRows = await engine.executeRaw<{ source_id: string }>(
|
|
`SELECT source_id FROM pages WHERE slug = $1`,
|
|
[REN_TO],
|
|
);
|
|
expect(toRows.length).toBe(1);
|
|
expect(toRows[0].source_id).toBe('testsrc');
|
|
});
|
|
|
|
test('getChunks with opts.sourceId only returns the intended source\'s chunks', async () => {
|
|
// Set up: same slug under both default and testsrc, each with distinct chunks.
|
|
const CHUNK_SLUG = 'topics/regression-getchunks';
|
|
await engine.putPage(CHUNK_SLUG, { type: 'concept', title: 'C default', compiled_truth: '' });
|
|
await engine.putPage(CHUNK_SLUG, { type: 'concept', title: 'C testsrc', compiled_truth: '' }, { sourceId: 'testsrc' });
|
|
await engine.upsertChunks(CHUNK_SLUG, [
|
|
{ chunk_index: 0, chunk_text: 'default chunk text', chunk_source: 'compiled_truth' },
|
|
], { sourceId: 'default' });
|
|
await engine.upsertChunks(CHUNK_SLUG, [
|
|
{ chunk_index: 0, chunk_text: 'testsrc chunk text', chunk_source: 'compiled_truth' },
|
|
], { sourceId: 'testsrc' });
|
|
|
|
// Pre-fix: bare-slug `WHERE p.slug = $1` returned BOTH source's chunks
|
|
// mashed together. importCodeFile uses getChunks for incremental embedding
|
|
// reuse; pre-fix would have grabbed the wrong source's embeddings.
|
|
const defaultChunks = await engine.getChunks(CHUNK_SLUG, { sourceId: 'default' });
|
|
expect(defaultChunks.length).toBe(1);
|
|
expect(defaultChunks[0].chunk_text).toBe('default chunk text');
|
|
|
|
const testsrcChunks = await engine.getChunks(CHUNK_SLUG, { sourceId: 'testsrc' });
|
|
expect(testsrcChunks.length).toBe(1);
|
|
expect(testsrcChunks[0].chunk_text).toBe('testsrc chunk text');
|
|
});
|
|
|
|
test('updateSlug without opts targets source=default only (back-compat)', async () => {
|
|
// Default still has REN_FROM. Rename it without opts; testsrc REN_TO
|
|
// already exists, so a bare rename would fail (source_id, slug) UNIQUE
|
|
// when both default and testsrc converge on REN_TO. Source-scoped rename
|
|
// succeeds because testsrc is untouched.
|
|
const REN_TO_2 = 'topics/regression-rename-to-2';
|
|
await engine.updateSlug(REN_FROM, REN_TO_2);
|
|
|
|
const rows = await engine.executeRaw<{ source_id: string; slug: string }>(
|
|
`SELECT source_id, slug FROM pages WHERE slug IN ($1, $2) ORDER BY source_id`,
|
|
[REN_FROM, REN_TO_2],
|
|
);
|
|
expect(rows.length).toBe(1);
|
|
expect(rows[0].source_id).toBe('default');
|
|
expect(rows[0].slug).toBe(REN_TO_2);
|
|
});
|
|
});
|
|
|
|
// ─────────────────────────────────────────────────────────────────────────
|
|
// v0.31.8 — op-handler-layer threading (D7 + D11 + D16 + D20 + D21 + D22)
|
|
//
|
|
// The pre-v0.31.8 op handlers in src/core/operations.ts called engine methods
|
|
// bare-slug, ignoring ctx.sourceId. Result: a remote MCP token whose
|
|
// ctx.sourceId='X' calling put_page / add_tag / get_links / etc. silently
|
|
// landed on source 'default'. This block drives the actual op handlers
|
|
// (operations.ts) — not just the engine surface — through a mock
|
|
// OperationContext carrying sourceId='X' and asserts only the X-source row
|
|
// is mutated/read.
|
|
// ─────────────────────────────────────────────────────────────────────────
|
|
|
|
import { operations } from '../src/core/operations.ts';
|
|
import type { OperationContext } from '../src/core/operations.ts';
|
|
|
|
function makeCtx(eng: PGLiteEngine, overrides: Partial<OperationContext> = {}): OperationContext {
|
|
return {
|
|
engine: eng as unknown as OperationContext['engine'],
|
|
config: { engine: 'pglite' } as never,
|
|
logger: { info: () => {}, warn: () => {}, error: () => {} },
|
|
dryRun: false,
|
|
remote: false,
|
|
...overrides,
|
|
};
|
|
}
|
|
|
|
function getOp(name: string) {
|
|
const op = operations.find(o => o.name === name);
|
|
if (!op) throw new Error(`op not registered: ${name}`);
|
|
return op;
|
|
}
|
|
|
|
describe('v0.31.8 op-handler ctx.sourceId threading', () => {
|
|
// Two-source fixture seeded fresh for this block. Use unique slugs so the
|
|
// earlier engine-layer suite's leftover state doesn't pollute assertions.
|
|
const TAG_SLUG = 'topics/op-tag-target';
|
|
|
|
beforeAll(async () => {
|
|
// Page exists at BOTH sources (the v0.18.0 supported state).
|
|
await engine.putPage(TAG_SLUG, {
|
|
type: 'concept', title: 'Default tag target', compiled_truth: '.',
|
|
});
|
|
await engine.putPage(TAG_SLUG, {
|
|
type: 'concept', title: 'Testsrc tag target', compiled_truth: '.',
|
|
}, { sourceId: 'testsrc' });
|
|
});
|
|
|
|
test('add_tag handler with ctx.sourceId=testsrc tags only the testsrc row', async () => {
|
|
const op = getOp('add_tag');
|
|
await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), { slug: TAG_SLUG, tag: 'op-handler-test-1' });
|
|
|
|
const rows = await engine.executeRaw<{ source_id: string }>(
|
|
`SELECT p.source_id FROM tags t JOIN pages p ON p.id = t.page_id
|
|
WHERE p.slug = $1 AND t.tag = $2`,
|
|
[TAG_SLUG, 'op-handler-test-1'],
|
|
);
|
|
expect(rows.length).toBe(1);
|
|
expect(rows[0].source_id).toBe('testsrc');
|
|
});
|
|
|
|
test('add_tag handler without ctx.sourceId tags the default row (back-compat)', async () => {
|
|
const op = getOp('add_tag');
|
|
await op.handler(makeCtx(engine), { slug: TAG_SLUG, tag: 'op-handler-test-2' });
|
|
|
|
const rows = await engine.executeRaw<{ source_id: string }>(
|
|
`SELECT p.source_id FROM tags t JOIN pages p ON p.id = t.page_id
|
|
WHERE p.slug = $1 AND t.tag = $2`,
|
|
[TAG_SLUG, 'op-handler-test-2'],
|
|
);
|
|
expect(rows.length).toBe(1);
|
|
expect(rows[0].source_id).toBe('default');
|
|
});
|
|
|
|
test('get_tags handler with ctx.sourceId=testsrc returns only testsrc tags', async () => {
|
|
// Both rows should now have one tag each from the two tests above.
|
|
const op = getOp('get_tags');
|
|
const tags = await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), { slug: TAG_SLUG }) as string[];
|
|
expect(tags).toContain('op-handler-test-1');
|
|
expect(tags).not.toContain('op-handler-test-2');
|
|
});
|
|
|
|
test('get_tags handler without ctx.sourceId returns default-source tags (back-compat)', async () => {
|
|
const op = getOp('get_tags');
|
|
const tags = await op.handler(makeCtx(engine), { slug: TAG_SLUG }) as string[];
|
|
// getTags is a v0.18.0-era source-aware method: it already defaults to
|
|
// source='default' when opts is omitted (see engine.ts addTag/removeTag/
|
|
// getTags comment). It does NOT use the D16 two-branch pattern — the
|
|
// pre-v0.31.8 behavior for tags on multi-source brains was always
|
|
// "scoped to default unless told otherwise." So with ctx.sourceId unset,
|
|
// only the default-source tag surfaces. (D16 two-branch applies to
|
|
// getLinks/getBacklinks/getTimeline/getRawData/getVersions/getAllSlugs/
|
|
// revertToVersion — the methods that pre-D12 had no source filter at all.)
|
|
expect(tags).toContain('op-handler-test-2');
|
|
expect(tags).not.toContain('op-handler-test-1');
|
|
});
|
|
|
|
test('add_link handler with ctx.sourceId scopes both endpoints', async () => {
|
|
// Seed a target page at both sources so addLink's INTERSECT pre-check passes.
|
|
const TARGET = 'topics/op-link-target';
|
|
await engine.putPage(TARGET, { type: 'concept', title: 'Default target', compiled_truth: '.' });
|
|
await engine.putPage(TARGET, { type: 'concept', title: 'Testsrc target', compiled_truth: '.' }, { sourceId: 'testsrc' });
|
|
|
|
const op = getOp('add_link');
|
|
await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), {
|
|
from: TAG_SLUG, to: TARGET, link_type: 'mentions', context: 'op-test',
|
|
});
|
|
|
|
const rows = await engine.executeRaw<{ from_source: string; to_source: string }>(
|
|
`SELECT f.source_id AS from_source, t.source_id AS to_source
|
|
FROM links l JOIN pages f ON f.id = l.from_page_id
|
|
JOIN pages t ON t.id = l.to_page_id
|
|
WHERE f.slug = $1 AND t.slug = $2 AND l.link_type = 'mentions'`,
|
|
[TAG_SLUG, TARGET],
|
|
);
|
|
expect(rows.length).toBe(1);
|
|
expect(rows[0].from_source).toBe('testsrc');
|
|
expect(rows[0].to_source).toBe('testsrc');
|
|
});
|
|
|
|
test('get_links handler scopes to ctx.sourceId; back-compat cross-source view preserved (D16)', async () => {
|
|
const op = getOp('get_links');
|
|
const scoped = await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), { slug: TAG_SLUG }) as Array<{ to_slug: string }>;
|
|
const cross = await op.handler(makeCtx(engine), { slug: TAG_SLUG }) as Array<{ to_slug: string }>;
|
|
// testsrc has the link from add_link test above. default has none.
|
|
expect(scoped.length).toBeGreaterThanOrEqual(1);
|
|
// Cross-source view sees at least the same edges (and would see default's
|
|
// if we'd seeded any). Under the two-branch back-compat path, this is the
|
|
// pre-v0.31.8 semantic — no source filter on the engine join.
|
|
expect(cross.length).toBeGreaterThanOrEqual(scoped.length);
|
|
});
|
|
|
|
test('delete_page handler scopes to ctx.sourceId (soft-delete only the testsrc row)', async () => {
|
|
// Use a fresh slug so we don't impact other tests in this describe block.
|
|
const DEL_SLUG = 'topics/op-delete-target';
|
|
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'Default', compiled_truth: '.' });
|
|
await engine.putPage(DEL_SLUG, { type: 'concept', title: 'Testsrc', compiled_truth: '.' }, { sourceId: 'testsrc' });
|
|
|
|
const op = getOp('delete_page');
|
|
await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), { slug: DEL_SLUG });
|
|
|
|
const rows = await engine.executeRaw<{ source_id: string; deleted_at: string | null }>(
|
|
`SELECT source_id, deleted_at FROM pages WHERE slug = $1 ORDER BY source_id`,
|
|
[DEL_SLUG],
|
|
);
|
|
expect(rows.length).toBe(2);
|
|
const def = rows.find(r => r.source_id === 'default')!;
|
|
const tst = rows.find(r => r.source_id === 'testsrc')!;
|
|
expect(def.deleted_at).toBeNull(); // default row untouched
|
|
expect(tst.deleted_at).not.toBeNull(); // testsrc row soft-deleted
|
|
});
|
|
|
|
test('put_raw_data handler threads ctx.sourceId (D21)', async () => {
|
|
const op = getOp('put_raw_data');
|
|
await op.handler(makeCtx(engine, { sourceId: 'testsrc' }), {
|
|
slug: TAG_SLUG, source: 'unit-test', data: { variant: 'testsrc' },
|
|
});
|
|
|
|
// Read via the engine to assert which source row got the raw_data.
|
|
const rd = await engine.getRawData(TAG_SLUG, 'unit-test', { sourceId: 'testsrc' });
|
|
expect(rd.length).toBe(1);
|
|
expect((rd[0].data as { variant: string }).variant).toBe('testsrc');
|
|
|
|
// Default-source raw_data should be untouched.
|
|
const defRd = await engine.getRawData(TAG_SLUG, 'unit-test', { sourceId: 'default' });
|
|
expect(defRd.length).toBe(0);
|
|
});
|
|
});
|