Files
gbrain/test/pglite-engine.test.ts
T
24881f60fc v0.34.4.0 fix(embed): cursor-paginated --stale hardening wave (D2/D3/D4/D6/D7/D8 + regression test) (#991)
* perf(embed): cursor-paginated stale loading + rate-limit backoff + partial index

Three fixes for embed --stale on large brains (300K+ chunks):

## 1. Cursor-paginated listStaleChunks (embed timeout fix)

The previous implementation pulled ALL stale rows (up to 100K) in one
query. On a 373K-row content_chunks table with 48K stale rows, this
query took >2 min and hit Supabase's 2-min statement_timeout, causing
embed --stale to silently fail with zero progress.

Fix: keyset pagination on (page_id, chunk_index) with a default batch
size of 2000 rows. Each query finishes in <1s. The embedAllStale loop
pages through batches, embeds each batch, then advances the cursor.

## 2. Rate-limit-aware retry (429 backoff)

The OpenAI SDK's built-in retry has a ~4s max backoff window, which is
too short for TPM (tokens-per-minute) limits on large pages (~90K
tokens). The embed loop would fail after 3 SDK retries and skip the
page entirely.

Fix: embedBatchWithBackoff wrapper parses the retry delay from the
429 error message (e.g. 'try again in 248ms') and sleeps for that
duration + 500ms padding. Up to 5 retries with parsed delays (60s
fallback when unparseable).

## 3. Migration v58: partial index for NULL embeddings

`CREATE INDEX idx_chunks_embedding_null ON content_chunks (page_id,
chunk_index) WHERE embedding IS NULL` — makes countStaleChunks() and
the paginated listStaleChunks() instant instead of full-table-scanning
373K rows.

## Testing

Verified on a 99K-page / 373K-chunk brain with 48K stale chunks.
Before: embed --stale hung for 2+ min then timed out (0 progress).
After: loads 2K rows in <1s, embeds concurrently, pages through all
stale chunks without timeout.

* fix(embed): wave of hardening + tests on cursor-paginated --stale path

Lands the 9 decisions + regression test set from /plan-eng-review on PR #991's
embed-perf cherry-pick. Implements the codex outside-voice findings folded in
during plan review.

Architecture / correctness:
- D2 jitter on the parsed retry-after delay (±30%) so 20 concurrent workers
  don't relock on the next 429 wave (thundering herd fix).
- D3 + D3a + D8 wall-clock budget (GBRAIN_EMBED_TIME_BUDGET_MS, default 30
  min) threaded as an AbortSignal into THREE places: the retry sleep
  (abortableSleep), the per-key worker claim loop, and the gateway embed
  call itself (so a worker mid-fetch on a ~30s OpenAI HTTP timeout cancels
  within seconds instead of waiting it out).
- D4 structured 429 detection that unwraps the gateway's AITransientError
  wrap via cause chain (depth-limited to 5). Naive `e.status === 429` was
  silently false against normalized errors; message-match stays as
  fallback. detect429FromCause exported as @internal helper.
- D4a `maxRetries: 0` passthrough through embedBatch → gateway →
  embedMany so the AI SDK's default 2-retry stack doesn't multiply this
  wrapper's 5 attempts (was up to 15 total cycles per call).
- D6 migration v59 (embed_stale_partial_index) rewritten to use
  CREATE INDEX CONCURRENTLY + handler-based engine-branching (mirrors v14
  invalid-remnant pattern). Plain CREATE INDEX would have taken ShareLock
  on the 373K-row content_chunks table for the duration of the build.
- D7 sourceId threaded through countStaleChunks + listStaleChunks +
  embedAllStale. `gbrain embed --stale --source X` was silently dropping
  the flag pre-fix and counting/embedding across every source. Both
  Postgres and PGLite engines updated.

Tests added:
- D5 8 unit cases for embedBatchWithBackoff in test/embed.serial.test.ts:
  ms / s retry-after parse, fallback, non-rate-limit rethrow, jitter
  variance, budget abort during sleep+fetch, normalized-error cause
  unwrap, maxRetries:0 passthrough verification.
- D5a fixed every pre-existing stale-row mock to include source_id +
  page_id (required on StaleChunkRow as of v0.33.3 cursor pagination —
  TypeScript's structural typing was hiding these).
- D7 unit cases asserting CLI `--source X` parses + threads sourceId.
- Gap scan: end-to-end wall-clock budget firing in the outer pagination
  loop via runEmbedCore.
- D6 migration v59 test cases in test/migrate.test.ts: source-shape
  assertion (CONCURRENTLY + invalid-remnant DROP-before-CREATE ordering),
  PGLite handler-branch idempotency, partial-index materialization.
- REGRESSION: new test/e2e/embed-stale-pagination.test.ts covering
  static (every chunk visited exactly once), failed-page (cursor advances
  past failures, next run picks up), page-split-across-batches,
  source-scoped scan, duplicate-slug-across-sources.
- PGLite parity cases for cursor pagination, page split, source filter
  in test/pglite-engine.test.ts (pins tuple-compare against WASM build).

Gate:
- bun run test: 6305 pass / 0 fail / 0 skip across all 8 shards + serial.
- DATABASE_URL=... bun run test:e2e: 90 files, 603 tests, 0 failures.

Plan: ~/.claude/plans/system-instruction-you-are-working-iterative-torvalds.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.34.3.0)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: garrytan-agents <garrytan-agents@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 21:22:10 -07:00

1321 lines
58 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/**
* PGLite Engine Tests — validates all 37 BrainEngine methods against PGLite (in-memory).
*
* No Docker, no DATABASE_URL, no external dependencies. Runs instantly in CI.
*/
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import type { BrainEngine } from '../src/core/engine.ts';
import type { PageInput, ChunkInput } from '../src/core/types.ts';
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({}); // in-memory
await engine.initSchema();
});
afterAll(async () => {
await engine.disconnect();
});
// Helper to reset data between test groups
async function truncateAll() {
const tables = [
'content_chunks', 'links', 'tags', 'raw_data',
'timeline_entries', 'page_versions', 'ingest_log', 'pages',
];
for (const t of tables) {
await (engine as any).db.exec(`DELETE FROM ${t}`);
}
}
const testPage: PageInput = {
type: 'concept',
title: 'Test Page',
compiled_truth: 'This is a test page about NovaMind AI agents.',
timeline: '2024-01-15: Founded NovaMind',
};
// ─────────────────────────────────────────────────────────────────
// Pages CRUD
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: Pages', () => {
beforeEach(truncateAll);
test('putPage + getPage round trip', async () => {
const page = await engine.putPage('test/hello', testPage);
expect(page.slug).toBe('test/hello');
expect(page.title).toBe('Test Page');
expect(page.type).toBe('concept');
expect(page.compiled_truth).toContain('NovaMind');
const fetched = await engine.getPage('test/hello');
expect(fetched).not.toBeNull();
expect(fetched!.title).toBe('Test Page');
expect(fetched!.content_hash).toBeTruthy();
});
test('putPage upserts on conflict', async () => {
await engine.putPage('test/upsert', testPage);
const updated = await engine.putPage('test/upsert', {
...testPage,
title: 'Updated Title',
});
expect(updated.title).toBe('Updated Title');
const all = await engine.listPages();
const matches = all.filter(p => p.slug === 'test/upsert');
expect(matches.length).toBe(1);
});
test('getPage returns null for missing slug', async () => {
const result = await engine.getPage('nonexistent/slug');
expect(result).toBeNull();
});
test('deletePage removes page', async () => {
await engine.putPage('test/delete-me', testPage);
await engine.deletePage('test/delete-me');
const result = await engine.getPage('test/delete-me');
expect(result).toBeNull();
});
test('listPages with type filter', async () => {
await engine.putPage('people/alice', { ...testPage, type: 'person', title: 'Alice' });
await engine.putPage('concepts/rag', { ...testPage, type: 'concept', title: 'RAG' });
const people = await engine.listPages({ type: 'person' });
expect(people.length).toBe(1);
expect(people[0].title).toBe('Alice');
});
test('listPages with tag filter', async () => {
await engine.putPage('test/tagged', testPage);
await engine.addTag('test/tagged', 'special');
const tagged = await engine.listPages({ tag: 'special' });
expect(tagged.length).toBe(1);
expect(tagged[0].slug).toBe('test/tagged');
});
test('listPages with slugPrefix filter (Issue #13)', async () => {
await truncateAll();
await engine.putPage('media/x/tweet-1', { ...testPage, type: 'concept' });
await engine.putPage('media/x/tweet-2', { ...testPage, type: 'concept' });
await engine.putPage('media/articles/post-1', { ...testPage, type: 'concept' });
await engine.putPage('people/alice', { ...testPage, type: 'person' });
const xOnly = await engine.listPages({ slugPrefix: 'media/x/', limit: 100 });
expect(xOnly.map((p) => p.slug).sort()).toEqual(['media/x/tweet-1', 'media/x/tweet-2']);
const allMedia = await engine.listPages({ slugPrefix: 'media/', limit: 100 });
expect(allMedia.length).toBe(3);
// Path-segment risk: 'media/x' (no trailing /) would also match 'media/xerox'.
// The matcher in storage-config.ts is responsible for trailing-/ semantics
// (step 6); the engine treats slugPrefix as a literal string prefix.
expect((await engine.listPages({ slugPrefix: 'media/x', limit: 100 })).length).toBe(2);
});
test('listPages slugPrefix escapes LIKE metacharacters', async () => {
await truncateAll();
await engine.putPage('safe/foo', { ...testPage, type: 'concept' });
// A user prefix containing % or _ would otherwise match unintended slugs
// if not escaped. We can't easily insert a slug with % in it (most slugs
// are url-safe), but we can confirm the escape logic doesn't break the
// happy path.
const result = await engine.listPages({ slugPrefix: 'safe/', limit: 10 });
expect(result.length).toBe(1);
expect(result[0].slug).toBe('safe/foo');
});
test('resolveSlugs exact match', async () => {
await engine.putPage('test/exact', testPage);
const slugs = await engine.resolveSlugs('test/exact');
expect(slugs).toEqual(['test/exact']);
});
test('resolveSlugs fuzzy match via pg_trgm', async () => {
await engine.putPage('people/sarah-chen', { ...testPage, title: 'Sarah Chen' });
const slugs = await engine.resolveSlugs('sarah');
expect(slugs.length).toBeGreaterThan(0);
expect(slugs).toContain('people/sarah-chen');
});
test('updateSlug renames page', async () => {
await engine.putPage('test/old-name', testPage);
await engine.updateSlug('test/old-name', 'test/new-name');
expect(await engine.getPage('test/old-name')).toBeNull();
expect((await engine.getPage('test/new-name'))?.title).toBe('Test Page');
});
test('validateSlug rejects path traversal', async () => {
expect(() => engine.putPage('../etc/passwd', testPage)).toThrow();
});
test('validateSlug rejects leading slash', async () => {
expect(() => engine.putPage('/absolute/path', testPage)).toThrow();
});
test('validateSlug normalizes to lowercase', async () => {
const page = await engine.putPage('Test/UPPER', testPage);
expect(page.slug).toBe('test/upper');
});
});
// ─────────────────────────────────────────────────────────────────
// Search (tsvector triggers + FTS)
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: Search', () => {
beforeAll(async () => {
await truncateAll();
await engine.putPage('companies/novamind', {
type: 'company', title: 'NovaMind',
compiled_truth: 'NovaMind builds AI agents for enterprise automation.',
});
await engine.upsertChunks('companies/novamind', [
{ chunk_index: 0, chunk_text: 'NovaMind builds AI agents for enterprise', chunk_source: 'compiled_truth' },
]);
await engine.putPage('concepts/rag', {
type: 'concept', title: 'Retrieval-Augmented Generation',
compiled_truth: 'RAG combines retrieval with generation for better answers.',
});
await engine.upsertChunks('concepts/rag', [
{ chunk_index: 0, chunk_text: 'RAG combines retrieval with generation', chunk_source: 'compiled_truth' },
]);
});
test('searchKeyword returns results for matching term', async () => {
const results = await engine.searchKeyword('NovaMind');
expect(results.length).toBeGreaterThan(0);
expect(results[0].slug).toBe('companies/novamind');
});
test('searchKeyword returns empty for non-matching term', async () => {
const results = await engine.searchKeyword('xyznonexistent');
expect(results.length).toBe(0);
});
test('tsvector trigger populates search_vector on insert', async () => {
// Verify the PL/pgSQL trigger fires and content_chunks.search_vector is
// populated from chunk_text. v0.20.0 Cathedral II Layer 3 moved FTS from
// pages.search_vector to content_chunks.search_vector — the chunk-grain
// vector is built from chunk_text (+ optional doc_comment + qualified
// symbol name). 'AI agents' is a phrase inside the chunk_text so it
// stresses the chunk-grain tsvector directly.
const results = await engine.searchKeyword('AI agents');
expect(results.length).toBeGreaterThan(0);
});
test('searchVector returns empty when no embeddings', async () => {
const fakeEmbedding = new Float32Array(1536);
const results = await engine.searchVector(fakeEmbedding);
expect(results.length).toBe(0);
});
});
// ─────────────────────────────────────────────────────────────────
// CJK keyword fallback (v0.32.7)
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: CJK keyword fallback (v0.32.7)', () => {
beforeEach(async () => {
await truncateAll();
// Three pages with Chinese / Japanese / Korean content; the Chinese
// page contains the substring 测试 three times so bigram ranking
// can rank it above the others.
await engine.putPage('originals/chinese-essay', {
type: 'concept', title: 'Chinese essay',
compiled_truth: '测试 内容 测试 测试 多次',
});
await engine.upsertChunks('originals/chinese-essay', [
{ chunk_index: 0, chunk_text: '测试 内容 测试 测试 多次', chunk_source: 'compiled_truth' },
]);
await engine.putPage('originals/japanese-essay', {
type: 'concept', title: 'Japanese essay',
compiled_truth: '今日は晴れです。明日は雨です。',
});
await engine.upsertChunks('originals/japanese-essay', [
{ chunk_index: 0, chunk_text: '今日は晴れです。明日は雨です。', chunk_source: 'compiled_truth' },
]);
await engine.putPage('originals/korean-essay', {
type: 'concept', title: 'Korean essay',
compiled_truth: '한글 테스트 문서 입니다',
});
await engine.upsertChunks('originals/korean-essay', [
{ chunk_index: 0, chunk_text: '한글 테스트 문서 입니다', chunk_source: 'compiled_truth' },
]);
// Plus an English page so we can verify the ASCII path still works.
await engine.putPage('originals/english-essay', {
type: 'concept', title: 'English essay',
compiled_truth: 'NovaMind builds AI agents for enterprise automation.',
});
await engine.upsertChunks('originals/english-essay', [
{ chunk_index: 0, chunk_text: 'NovaMind builds AI agents for enterprise', chunk_source: 'compiled_truth' },
]);
});
test('CJK query routes to LIKE branch and finds Chinese substring', async () => {
const results = await engine.searchKeyword('测试');
expect(results.length).toBeGreaterThan(0);
expect(results[0].slug).toBe('originals/chinese-essay');
});
test('CJK query finds Japanese substring', async () => {
const results = await engine.searchKeyword('晴れ');
expect(results.length).toBeGreaterThan(0);
expect(results[0].slug).toBe('originals/japanese-essay');
});
test('CJK query finds Korean Hangul substring', async () => {
const results = await engine.searchKeyword('한글');
expect(results.length).toBeGreaterThan(0);
expect(results[0].slug).toBe('originals/korean-essay');
});
test('bigram ranking: 3-hit page outranks 1-hit page', async () => {
// Add another Chinese page with only ONE occurrence of 测试.
await engine.putPage('originals/chinese-one-hit', {
type: 'concept', title: 'One-hit',
compiled_truth: '只有一个 测试 in this page',
});
await engine.upsertChunks('originals/chinese-one-hit', [
{ chunk_index: 0, chunk_text: '只有一个 测试 in this page', chunk_source: 'compiled_truth' },
]);
const results = await engine.searchKeyword('测试');
// 3-occurrence page should rank ahead of 1-occurrence page.
const idxThreeHits = results.findIndex(r => r.slug === 'originals/chinese-essay');
const idxOneHit = results.findIndex(r => r.slug === 'originals/chinese-one-hit');
expect(idxThreeHits).toBeGreaterThanOrEqual(0);
expect(idxOneHit).toBeGreaterThanOrEqual(0);
expect(idxThreeHits).toBeLessThan(idxOneHit);
});
test('REGRESSION: ASCII query still uses FTS path and returns English hits', async () => {
const results = await engine.searchKeyword('NovaMind');
expect(results.length).toBeGreaterThan(0);
expect(results[0].slug).toBe('originals/english-essay');
});
test('REGRESSION: ASCII query does NOT match CJK pages', async () => {
// English query against a brain containing Chinese pages should not
// return Chinese hits (English tokenizer + Chinese text → no FTS match).
const results = await engine.searchKeyword('NovaMind');
expect(results.every(r => !r.slug.includes('chinese') && !r.slug.includes('japanese') && !r.slug.includes('korean'))).toBe(true);
});
test('LIKE-meta-char escape: query with literal % does not wildcard-match', async () => {
// After our escape pass, ILIKE '%' || '\%' || '%' ESCAPE '\' looks
// for a literal `%` character — which our seeded CJK pages don't
// contain. So results should be empty (or at least not all 3 CJK pages).
const results = await engine.searchKeyword('100% 测试');
// Either the literal "100% 测试" exists nowhere (expected empty), or
// only the exact-substring pages match. None of our seeded pages
// contain this exact string.
expect(results.length).toBe(0);
});
test('empty CJK query returns no results', async () => {
const results = await engine.searchKeyword('');
// Empty query: our CJK branch detects hasCJK('') === false, so it falls
// to the ASCII FTS path which also returns nothing for empty.
expect(results).toEqual([]);
});
});
// ─────────────────────────────────────────────────────────────────
// Chunks
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: Chunks', () => {
beforeEach(truncateAll);
test('upsertChunks + getChunks round trip', async () => {
await engine.putPage('test/chunks', testPage);
await engine.upsertChunks('test/chunks', [
{ chunk_index: 0, chunk_text: 'Chunk zero', chunk_source: 'compiled_truth' },
{ chunk_index: 1, chunk_text: 'Chunk one', chunk_source: 'compiled_truth' },
]);
const chunks = await engine.getChunks('test/chunks');
expect(chunks.length).toBe(2);
expect(chunks[0].chunk_text).toBe('Chunk zero');
expect(chunks[1].chunk_text).toBe('Chunk one');
});
test('upsertChunks removes orphan chunks', async () => {
await engine.putPage('test/orphan', testPage);
await engine.upsertChunks('test/orphan', [
{ chunk_index: 0, chunk_text: 'Keep', chunk_source: 'compiled_truth' },
{ chunk_index: 1, chunk_text: 'Remove', chunk_source: 'compiled_truth' },
]);
// Re-upsert with only index 0
await engine.upsertChunks('test/orphan', [
{ chunk_index: 0, chunk_text: 'Updated', chunk_source: 'compiled_truth' },
]);
const chunks = await engine.getChunks('test/orphan');
expect(chunks.length).toBe(1);
expect(chunks[0].chunk_text).toBe('Updated');
});
test('upsertChunks throws for missing page', async () => {
await expect(
engine.upsertChunks('nonexistent/page', [
{ chunk_index: 0, chunk_text: 'test', chunk_source: 'compiled_truth' },
])
).rejects.toThrow('Page not found');
});
test('deleteChunks removes all chunks for page', async () => {
await engine.putPage('test/delete-chunks', testPage);
await engine.upsertChunks('test/delete-chunks', [
{ chunk_index: 0, chunk_text: 'Gone', chunk_source: 'compiled_truth' },
]);
await engine.deleteChunks('test/delete-chunks');
const chunks = await engine.getChunks('test/delete-chunks');
expect(chunks.length).toBe(0);
});
test('getChunksWithEmbeddings returns embedding data', async () => {
await engine.putPage('test/embed', testPage);
const embedding = new Float32Array(1536).fill(0.1);
await engine.upsertChunks('test/embed', [
{ chunk_index: 0, chunk_text: 'With embedding', chunk_source: 'compiled_truth', embedding },
]);
const chunks = await engine.getChunksWithEmbeddings('test/embed');
expect(chunks.length).toBe(1);
expect(chunks[0].embedding).not.toBeNull();
});
});
// ────────────────────────────────────────────────────────────────────────
// v0.33.4 D7 + IRON RULE — countStaleChunks + listStaleChunks contract
// PGLite parity for the Postgres E2E in test/e2e/embed-stale-pagination.
// Pins the tuple-compare `(cc.page_id, cc.chunk_index) > ($1, $2)` against
// the WASM build (Postgres 17.5 in WASM has had quirks here historically).
// ────────────────────────────────────────────────────────────────────────
describe('PGLiteEngine: stale chunk pagination (D7 + REGRESSION)', () => {
beforeEach(truncateAll);
test('countStaleChunks: zero-state baseline', async () => {
expect(await engine.countStaleChunks()).toBe(0);
});
test('countStaleChunks counts chunks with NULL embedding only', async () => {
await engine.putPage('test/stale-a', testPage);
await engine.upsertChunks('test/stale-a', [
{ chunk_index: 0, chunk_text: 'no embed', chunk_source: 'compiled_truth' },
{ chunk_index: 1, chunk_text: 'has embed', chunk_source: 'compiled_truth', embedding: new Float32Array(1536).fill(0.1) },
]);
expect(await engine.countStaleChunks()).toBe(1);
});
test('listStaleChunks: cursor pagination across page boundaries', async () => {
// Seed 3 pages × 3 chunks = 9 stale rows; walk with batchSize=2.
for (const slug of ['c-a', 'c-b', 'c-c']) {
await engine.putPage(`test/${slug}`, testPage);
await engine.upsertChunks(`test/${slug}`, [
{ chunk_index: 0, chunk_text: 'a', chunk_source: 'compiled_truth' },
{ chunk_index: 1, chunk_text: 'b', chunk_source: 'compiled_truth' },
{ chunk_index: 2, chunk_text: 'c', chunk_source: 'compiled_truth' },
]);
}
const visited = new Set<string>();
let after_pid = 0;
let after_idx = -1;
let lastPid = -1;
let lastIdx = -1;
let cursorMonotonic = true;
// eslint-disable-next-line no-constant-condition
while (true) {
const batch = await engine.listStaleChunks({
batchSize: 2,
afterPageId: after_pid,
afterChunkIndex: after_idx,
});
if (batch.length === 0) break;
for (const row of batch) {
const key = `${row.page_id}::${row.chunk_index}`;
expect(visited.has(key)).toBe(false);
visited.add(key);
const advance = row.page_id > lastPid
|| (row.page_id === lastPid && row.chunk_index > lastIdx);
if (!advance) cursorMonotonic = false;
lastPid = row.page_id;
lastIdx = row.chunk_index;
}
const tail = batch[batch.length - 1];
after_pid = tail.page_id;
after_idx = tail.chunk_index;
if (batch.length < 2) break;
}
expect(visited.size).toBe(9);
expect(cursorMonotonic).toBe(true);
});
test('listStaleChunks: page split across batches (1 page, 5 chunks, batchSize=2)', async () => {
await engine.putPage('test/split', testPage);
await engine.upsertChunks('test/split', [
{ chunk_index: 0, chunk_text: 'a', chunk_source: 'compiled_truth' },
{ chunk_index: 1, chunk_text: 'b', chunk_source: 'compiled_truth' },
{ chunk_index: 2, chunk_text: 'c', chunk_source: 'compiled_truth' },
{ chunk_index: 3, chunk_text: 'd', chunk_source: 'compiled_truth' },
{ chunk_index: 4, chunk_text: 'e', chunk_source: 'compiled_truth' },
]);
const collected: number[] = [];
let after_pid = 0;
let after_idx = -1;
// eslint-disable-next-line no-constant-condition
while (true) {
const batch = await engine.listStaleChunks({
batchSize: 2,
afterPageId: after_pid,
afterChunkIndex: after_idx,
});
if (batch.length === 0) break;
for (const r of batch) collected.push(r.chunk_index);
const tail = batch[batch.length - 1];
after_pid = tail.page_id;
after_idx = tail.chunk_index;
if (batch.length < 2) break;
}
expect(collected).toEqual([0, 1, 2, 3, 4]);
});
test('countStaleChunks + listStaleChunks honor sourceId filter (D7)', async () => {
// PGLite default seed has 'default' source. Add 'other-source' and
// seed identical slugs in both.
await engine.executeRaw(
`INSERT INTO sources (id, name, local_path) VALUES ('other', 'other', '/tmp/other') ON CONFLICT (id) DO NOTHING`,
);
// Default source page via the engine API.
await engine.putPage('test/shared', testPage, { sourceId: 'default' });
await engine.upsertChunks('test/shared', [
{ chunk_index: 0, chunk_text: 'default-0', chunk_source: 'compiled_truth' },
{ chunk_index: 1, chunk_text: 'default-1', chunk_source: 'compiled_truth' },
], { sourceId: 'default' });
// Same slug, different source.
await engine.putPage('test/shared', testPage, { sourceId: 'other' });
await engine.upsertChunks('test/shared', [
{ chunk_index: 0, chunk_text: 'other-0', chunk_source: 'compiled_truth' },
{ chunk_index: 1, chunk_text: 'other-1', chunk_source: 'compiled_truth' },
{ chunk_index: 2, chunk_text: 'other-2', chunk_source: 'compiled_truth' },
], { sourceId: 'other' });
expect(await engine.countStaleChunks()).toBe(5);
expect(await engine.countStaleChunks({ sourceId: 'default' })).toBe(2);
expect(await engine.countStaleChunks({ sourceId: 'other' })).toBe(3);
const defaultRows = await engine.listStaleChunks({ sourceId: 'default', batchSize: 100 });
expect(defaultRows).toHaveLength(2);
for (const r of defaultRows) expect(r.source_id).toBe('default');
const otherRows = await engine.listStaleChunks({ sourceId: 'other', batchSize: 100 });
expect(otherRows).toHaveLength(3);
for (const r of otherRows) expect(r.source_id).toBe('other');
});
});
// ─────────────────────────────────────────────────────────────────
// Links + Graph
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: Links', () => {
beforeEach(async () => {
await truncateAll();
await engine.putPage('people/alice', { ...testPage, type: 'person', title: 'Alice' });
await engine.putPage('companies/acme', { ...testPage, type: 'company', title: 'ACME' });
await engine.putPage('companies/beta', { ...testPage, type: 'company', title: 'Beta' });
});
test('addLink + getLinks', async () => {
await engine.addLink('people/alice', 'companies/acme', 'works at', 'employment');
const links = await engine.getLinks('people/alice');
expect(links.length).toBe(1);
expect(links[0].to_slug).toBe('companies/acme');
});
test('getBacklinks', async () => {
await engine.addLink('people/alice', 'companies/acme');
const backlinks = await engine.getBacklinks('companies/acme');
expect(backlinks.length).toBe(1);
expect(backlinks[0].from_slug).toBe('people/alice');
});
test('removeLink', async () => {
await engine.addLink('people/alice', 'companies/acme');
await engine.removeLink('people/alice', 'companies/acme');
const links = await engine.getLinks('people/alice');
expect(links.length).toBe(0);
});
test('traverseGraph with depth', async () => {
await engine.addLink('people/alice', 'companies/acme');
await engine.addLink('companies/acme', 'companies/beta');
const graph = await engine.traverseGraph('people/alice', 2);
expect(graph.length).toBeGreaterThanOrEqual(2);
const slugs = graph.map(n => n.slug);
expect(slugs).toContain('people/alice');
expect(slugs).toContain('companies/acme');
});
});
// ─────────────────────────────────────────────────────────────────
// Tags
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: Tags', () => {
beforeEach(async () => {
await truncateAll();
await engine.putPage('test/tags', testPage);
});
test('addTag + getTags', async () => {
await engine.addTag('test/tags', 'alpha');
await engine.addTag('test/tags', 'beta');
const tags = await engine.getTags('test/tags');
expect(tags).toEqual(['alpha', 'beta']);
});
test('removeTag', async () => {
await engine.addTag('test/tags', 'remove-me');
await engine.removeTag('test/tags', 'remove-me');
const tags = await engine.getTags('test/tags');
expect(tags).not.toContain('remove-me');
});
test('duplicate tag is idempotent', async () => {
await engine.addTag('test/tags', 'dup');
await engine.addTag('test/tags', 'dup');
const tags = await engine.getTags('test/tags');
expect(tags.filter(t => t === 'dup').length).toBe(1);
});
});
// ─────────────────────────────────────────────────────────────────
// Timeline
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: Timeline', () => {
beforeEach(async () => {
await truncateAll();
await engine.putPage('test/timeline', testPage);
});
test('addTimelineEntry + getTimeline', async () => {
await engine.addTimelineEntry('test/timeline', {
date: '2024-01-15', summary: 'Founded', detail: 'Company founded',
});
const entries = await engine.getTimeline('test/timeline');
expect(entries.length).toBe(1);
expect(entries[0].summary).toBe('Founded');
});
test('getTimeline with date range', async () => {
await engine.addTimelineEntry('test/timeline', { date: '2024-01-01', summary: 'Jan' });
await engine.addTimelineEntry('test/timeline', { date: '2024-06-01', summary: 'Jun' });
await engine.addTimelineEntry('test/timeline', { date: '2024-12-01', summary: 'Dec' });
const filtered = await engine.getTimeline('test/timeline', {
after: '2024-03-01', before: '2024-09-01',
});
expect(filtered.length).toBe(1);
expect(filtered[0].summary).toBe('Jun');
});
});
// ─────────────────────────────────────────────────────────────────
// Batch methods (addLinksBatch / addTimelineEntriesBatch)
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: addLinksBatch', () => {
beforeEach(async () => {
await truncateAll();
await engine.putPage('a', { type: 'concept', title: 'A', compiled_truth: '', timeline: '' });
await engine.putPage('b', { type: 'concept', title: 'B', compiled_truth: '', timeline: '' });
await engine.putPage('c', { type: 'concept', title: 'C', compiled_truth: '', timeline: '' });
});
test('empty batch returns 0 with no DB call', async () => {
expect(await engine.addLinksBatch([])).toBe(0);
});
test('batch of 1 with missing optional fields inserts row with empty defaults', async () => {
const inserted = await engine.addLinksBatch([{ from_slug: 'a', to_slug: 'b' }]);
expect(inserted).toBe(1);
const links = await engine.getLinks('a');
expect(links.length).toBe(1);
expect(links[0].context).toBe('');
expect(links[0].link_type).toBe('');
});
test('within-batch duplicates are deduped via ON CONFLICT (no 21000 error)', async () => {
const inserted = await engine.addLinksBatch([
{ from_slug: 'a', to_slug: 'b', link_type: 'mention' },
{ from_slug: 'a', to_slug: 'b', link_type: 'mention' },
{ from_slug: 'a', to_slug: 'c', link_type: 'mention' },
]);
expect(inserted).toBe(2);
});
test('rows with missing slug are silently dropped by JOIN', async () => {
const inserted = await engine.addLinksBatch([
{ from_slug: 'doesnt-exist', to_slug: 'b' },
{ from_slug: 'a', to_slug: 'b' },
]);
expect(inserted).toBe(1);
});
test('half-existing batch returns count of new only', async () => {
await engine.addLink('a', 'b', '', 'mention');
const inserted = await engine.addLinksBatch([
{ from_slug: 'a', to_slug: 'b', link_type: 'mention' },
{ from_slug: 'a', to_slug: 'c', link_type: 'mention' },
]);
expect(inserted).toBe(1);
});
test('batch of 100 fresh rows returns 100', async () => {
// Create 100 target pages
for (let i = 0; i < 100; i++) {
await engine.putPage(`target/${i}`, { type: 'concept', title: `T${i}`, compiled_truth: '', timeline: '' });
}
const batch = Array.from({ length: 100 }, (_, i) => ({
from_slug: 'a', to_slug: `target/${i}`, link_type: 'mention',
}));
expect(await engine.addLinksBatch(batch)).toBe(100);
});
});
describe('PGLiteEngine: addTimelineEntriesBatch', () => {
beforeEach(async () => {
await truncateAll();
await engine.putPage('p1', { type: 'concept', title: 'P1', compiled_truth: '', timeline: '' });
await engine.putPage('p2', { type: 'concept', title: 'P2', compiled_truth: '', timeline: '' });
});
test('empty batch returns 0', async () => {
expect(await engine.addTimelineEntriesBatch([])).toBe(0);
});
test('batch of 1 with missing optionals inserts with empty defaults', async () => {
const inserted = await engine.addTimelineEntriesBatch([
{ slug: 'p1', date: '2024-01-15', summary: 'Founded' },
]);
expect(inserted).toBe(1);
const entries = await engine.getTimeline('p1');
expect(entries.length).toBe(1);
expect(entries[0].source).toBe('');
expect(entries[0].detail).toBe('');
});
test('within-batch duplicates are deduped via ON CONFLICT', async () => {
const inserted = await engine.addTimelineEntriesBatch([
{ slug: 'p1', date: '2024-01-15', summary: 'Founded' },
{ slug: 'p1', date: '2024-01-15', summary: 'Founded' },
{ slug: 'p1', date: '2024-02-01', summary: 'Launched' },
]);
expect(inserted).toBe(2);
});
test('rows with missing slug are silently dropped by JOIN', async () => {
const inserted = await engine.addTimelineEntriesBatch([
{ slug: 'no-such-page', date: '2024-01-15', summary: 'Phantom' },
{ slug: 'p1', date: '2024-01-15', summary: 'Real' },
]);
expect(inserted).toBe(1);
});
test('mix of new + existing returns count of new only', async () => {
await engine.addTimelineEntry('p1', { date: '2024-01-15', summary: 'Founded' });
const inserted = await engine.addTimelineEntriesBatch([
{ slug: 'p1', date: '2024-01-15', summary: 'Founded' },
{ slug: 'p1', date: '2024-02-01', summary: 'Launched' },
{ slug: 'p2', date: '2024-03-01', summary: 'Spun off' },
]);
expect(inserted).toBe(2);
});
});
// v0.18.0: regression guards for the cross-source JOIN fan-out.
// Before the fix, addLinksBatch/addTimelineEntriesBatch JOINed on pages.slug
// only — so a page with the same slug in two sources would fan out and
// silently create duplicate edges / entries. Source-id-qualified JOINs
// eliminate the fan-out.
describe('PGLiteEngine: batch ops source-awareness (v0.18.0)', () => {
beforeEach(async () => {
await truncateAll();
// Register a second source and populate the same slugs in both.
const db = (engine as any).db;
await db.query(
`INSERT INTO sources (id, name) VALUES ('alt', 'alt')
ON CONFLICT (id) DO NOTHING`
);
// default-source rows via putPage (schema DEFAULT 'default').
await engine.putPage('topics/ai', { type: 'concept', title: 'AI (default)', compiled_truth: '', timeline: '' });
await engine.putPage('topics/ml', { type: 'concept', title: 'ML (default)', compiled_truth: '', timeline: '' });
// alt-source rows with the same slugs, inserted via raw SQL.
await db.query(
`INSERT INTO pages (slug, type, title, compiled_truth, timeline, frontmatter, content_hash, source_id, updated_at)
VALUES ('topics/ai', 'concept', 'AI (alt)', '', '', '{}'::jsonb, 'h1', 'alt', now()),
('topics/ml', 'concept', 'ML (alt)', '', '', '{}'::jsonb, 'h2', 'alt', now())`
);
});
test('addLinksBatch default source_id does NOT fan out across sources', async () => {
const inserted = await engine.addLinksBatch([
{ from_slug: 'topics/ai', to_slug: 'topics/ml', link_type: 'mention' },
]);
// Exactly one edge, not two. Before the fix this was 2.
expect(inserted).toBe(1);
const db = (engine as any).db;
const { rows } = await db.query(
`SELECT f.source_id AS from_src, t.source_id AS to_src
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id`
);
expect(rows.length).toBe(1);
expect(rows[0].from_src).toBe('default');
expect(rows[0].to_src).toBe('default');
});
test('addLinksBatch with explicit alt source_id lands in alt only', async () => {
const inserted = await engine.addLinksBatch([
{
from_slug: 'topics/ai', to_slug: 'topics/ml', link_type: 'mention',
from_source_id: 'alt', to_source_id: 'alt',
},
]);
expect(inserted).toBe(1);
const db = (engine as any).db;
const { rows } = await db.query(
`SELECT f.source_id AS from_src, t.source_id AS to_src
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id`
);
expect(rows.length).toBe(1);
expect(rows[0].from_src).toBe('alt');
expect(rows[0].to_src).toBe('alt');
});
test('addLinksBatch supports cross-source edges', async () => {
const inserted = await engine.addLinksBatch([
{
from_slug: 'topics/ai', to_slug: 'topics/ml', link_type: 'mention',
from_source_id: 'default', to_source_id: 'alt',
},
]);
expect(inserted).toBe(1);
const db = (engine as any).db;
const { rows } = await db.query(
`SELECT f.source_id AS from_src, t.source_id AS to_src
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id`
);
expect(rows.length).toBe(1);
expect(rows[0].from_src).toBe('default');
expect(rows[0].to_src).toBe('alt');
});
test('addTimelineEntriesBatch default source_id does NOT fan out across sources', async () => {
const inserted = await engine.addTimelineEntriesBatch([
{ slug: 'topics/ai', date: '2024-01-15', summary: 'Founded' },
]);
// Exactly one entry (default source), not two. Before the fix this was 2.
expect(inserted).toBe(1);
const db = (engine as any).db;
const { rows } = await db.query(
`SELECT p.source_id FROM timeline_entries te
JOIN pages p ON p.id = te.page_id`
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('default');
});
test('addTimelineEntriesBatch with explicit alt source_id lands in alt only', async () => {
const inserted = await engine.addTimelineEntriesBatch([
{ slug: 'topics/ai', date: '2024-01-15', summary: 'Founded', source_id: 'alt' },
]);
expect(inserted).toBe(1);
const db = (engine as any).db;
const { rows } = await db.query(
`SELECT p.source_id FROM timeline_entries te
JOIN pages p ON p.id = te.page_id`
);
expect(rows.length).toBe(1);
expect(rows[0].source_id).toBe('alt');
});
});
// ─────────────────────────────────────────────────────────────────
// Raw Data, Versions, Config, IngestLog
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: RawData', () => {
beforeEach(async () => {
await truncateAll();
await engine.putPage('test/raw', testPage);
});
test('putRawData + getRawData', async () => {
await engine.putRawData('test/raw', 'crunchbase', { funding: '$10M' });
const data = await engine.getRawData('test/raw', 'crunchbase');
expect(data.length).toBe(1);
expect((data[0].data as any).funding).toBe('$10M');
});
});
describe('PGLiteEngine: Versions', () => {
beforeEach(async () => {
await truncateAll();
await engine.putPage('test/version', testPage);
});
test('createVersion + getVersions', async () => {
const v = await engine.createVersion('test/version');
expect(v.compiled_truth).toBe(testPage.compiled_truth);
const versions = await engine.getVersions('test/version');
expect(versions.length).toBe(1);
});
test('revertToVersion restores content', async () => {
await engine.createVersion('test/version');
await engine.putPage('test/version', { ...testPage, compiled_truth: 'Changed' });
const versions = await engine.getVersions('test/version');
await engine.revertToVersion('test/version', versions[0].id);
const page = await engine.getPage('test/version');
expect(page!.compiled_truth).toBe(testPage.compiled_truth);
});
});
describe('PGLiteEngine: Config', () => {
test('getConfig + setConfig', async () => {
await engine.setConfig('test_key', 'test_value');
const val = await engine.getConfig('test_key');
expect(val).toBe('test_value');
});
test('getConfig returns null for missing key', async () => {
const val = await engine.getConfig('nonexistent_key');
expect(val).toBeNull();
});
});
describe('PGLiteEngine: IngestLog', () => {
test('logIngest + getIngestLog', async () => {
await engine.logIngest({
source_type: 'git', source_ref: '/tmp/test-repo',
pages_updated: ['test/a', 'test/b'], summary: 'Imported 2 pages',
});
const log = await engine.getIngestLog({ limit: 10 });
expect(log.length).toBeGreaterThan(0);
expect(log[0].source_type).toBe('git');
});
});
// ─────────────────────────────────────────────────────────────────
// Stats + Health
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: Stats & Health', () => {
beforeAll(async () => {
await truncateAll();
await engine.putPage('test/stats', testPage);
await engine.upsertChunks('test/stats', [
{ chunk_index: 0, chunk_text: 'chunk', chunk_source: 'compiled_truth' },
]);
await engine.addTag('test/stats', 'stat-tag');
});
test('getStats returns correct counts', async () => {
const stats = await engine.getStats();
expect(stats.page_count).toBe(1);
expect(stats.chunk_count).toBe(1);
expect(stats.tag_count).toBe(1);
expect(stats.pages_by_type.concept).toBe(1);
});
test('getHealth returns coverage metrics', async () => {
const health = await engine.getHealth();
expect(health.page_count).toBe(1);
expect(health.missing_embeddings).toBe(1); // chunk has no embedding
expect(health.embed_coverage).toBe(0);
});
});
// ─────────────────────────────────────────────────────────────────
// Transactions
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: Transactions', () => {
beforeEach(truncateAll);
test('transaction commits on success', async () => {
await engine.transaction(async (tx) => {
await tx.putPage('test/tx-ok', testPage);
});
const page = await engine.getPage('test/tx-ok');
expect(page).not.toBeNull();
});
test('transaction rolls back on error', async () => {
try {
await engine.transaction(async (tx) => {
await tx.putPage('test/tx-fail', testPage);
throw new Error('Deliberate rollback');
});
} catch { /* expected */ }
const page = await engine.getPage('test/tx-fail');
expect(page).toBeNull();
});
});
// ─────────────────────────────────────────────────────────────────
// Cascade deletes
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: Cascade deletes', () => {
test('deleting a page cascades to chunks, tags, links', async () => {
await engine.putPage('test/cascade', testPage);
await engine.upsertChunks('test/cascade', [
{ chunk_index: 0, chunk_text: 'cascade chunk', chunk_source: 'compiled_truth' },
]);
await engine.addTag('test/cascade', 'cascade-tag');
await engine.deletePage('test/cascade');
const chunks = await engine.getChunks('test/cascade');
expect(chunks.length).toBe(0);
const tags = await engine.getTags('test/cascade');
expect(tags.length).toBe(0);
});
});
// ─────────────────────────────────────────────────────────────────
// v0.10.1: Knowledge graph layer
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: getAllSlugs', () => {
beforeEach(async () => {
await truncateAll();
await engine.putPage('people/alice', { ...testPage, type: 'person', title: 'Alice' });
await engine.putPage('people/bob', { ...testPage, type: 'person', title: 'Bob' });
await engine.putPage('companies/acme', { ...testPage, type: 'company', title: 'Acme' });
});
test('returns Set of all page slugs', async () => {
const slugs = await engine.getAllSlugs();
expect(slugs).toBeInstanceOf(Set);
expect(slugs.size).toBe(3);
expect(slugs.has('people/alice')).toBe(true);
expect(slugs.has('companies/acme')).toBe(true);
});
test('empty brain returns empty Set', async () => {
await truncateAll();
const slugs = await engine.getAllSlugs();
expect(slugs.size).toBe(0);
});
});
describe('PGLiteEngine: listPages updated_after filter', () => {
beforeEach(async () => {
await truncateAll();
});
test('filters pages by updated_at > given date', async () => {
await engine.putPage('test/old', testPage);
// Sleep briefly so the second page has a strictly later updated_at.
await new Promise(r => setTimeout(r, 10));
const cutoff = new Date().toISOString();
await new Promise(r => setTimeout(r, 10));
await engine.putPage('test/new', testPage);
const recent = await engine.listPages({ updated_after: cutoff, limit: 100 });
const recentSlugs = recent.map(p => p.slug);
expect(recentSlugs).toContain('test/new');
expect(recentSlugs).not.toContain('test/old');
});
test('without updated_after, returns all pages (regression)', async () => {
await engine.putPage('test/a', testPage);
await engine.putPage('test/b', testPage);
const all = await engine.listPages({ limit: 100 });
expect(all.length).toBe(2);
});
});
describe('PGLiteEngine: Multi-type links (v5 migration)', () => {
beforeEach(async () => {
await truncateAll();
await engine.putPage('people/alice', { ...testPage, type: 'person', title: 'Alice' });
await engine.putPage('companies/acme', { ...testPage, type: 'company', title: 'Acme' });
});
test('same (from, to) with different link_types both stored', async () => {
await engine.addLink('people/alice', 'companies/acme', 'CEO', 'works_at');
await engine.addLink('people/alice', 'companies/acme', 'on the board', 'advises');
const links = await engine.getLinks('people/alice');
expect(links.length).toBe(2);
const types = links.map(l => l.link_type).sort();
expect(types).toEqual(['advises', 'works_at']);
});
test('upsert on same (from, to, type) updates context', async () => {
await engine.addLink('people/alice', 'companies/acme', 'old context', 'works_at');
await engine.addLink('people/alice', 'companies/acme', 'new context', 'works_at');
const links = await engine.getLinks('people/alice');
expect(links.length).toBe(1);
expect(links[0].context).toBe('new context');
});
test('removeLink without linkType removes ALL types for the pair (regression)', async () => {
await engine.addLink('people/alice', 'companies/acme', 'a', 'works_at');
await engine.addLink('people/alice', 'companies/acme', 'b', 'advises');
await engine.removeLink('people/alice', 'companies/acme');
const links = await engine.getLinks('people/alice');
expect(links.length).toBe(0);
});
test('removeLink with linkType removes only that type', async () => {
await engine.addLink('people/alice', 'companies/acme', 'a', 'works_at');
await engine.addLink('people/alice', 'companies/acme', 'b', 'advises');
await engine.removeLink('people/alice', 'companies/acme', 'works_at');
const links = await engine.getLinks('people/alice');
expect(links.length).toBe(1);
expect(links[0].link_type).toBe('advises');
});
});
describe('PGLiteEngine: Timeline dedup constraint (v6 migration)', () => {
beforeEach(async () => {
await truncateAll();
await engine.putPage('test/timeline-dedup', testPage);
});
test('inserting same (date, summary) twice is silent no-op (idempotent)', async () => {
await engine.addTimelineEntry('test/timeline-dedup', { date: '2026-01-15', summary: 'Event A' });
await engine.addTimelineEntry('test/timeline-dedup', { date: '2026-01-15', summary: 'Event A' });
const entries = await engine.getTimeline('test/timeline-dedup');
expect(entries.length).toBe(1);
});
test('different summary on same date: both inserted', async () => {
await engine.addTimelineEntry('test/timeline-dedup', { date: '2026-01-15', summary: 'Morning' });
await engine.addTimelineEntry('test/timeline-dedup', { date: '2026-01-15', summary: 'Evening' });
const entries = await engine.getTimeline('test/timeline-dedup');
expect(entries.length).toBe(2);
});
test('throws on missing page (default behavior preserved)', async () => {
await expect(engine.addTimelineEntry('does/not-exist', { date: '2026-01-15', summary: 'X' }))
.rejects.toThrow();
});
test('skipExistenceCheck=true: silent no-op on missing page', async () => {
// No throw, but also nothing inserted (subquery returns no rows).
await engine.addTimelineEntry(
'does/not-exist',
{ date: '2026-01-15', summary: 'X' },
{ skipExistenceCheck: true },
);
// No assertion needed beyond "did not throw".
});
});
describe('PGLiteEngine: getBacklinkCounts', () => {
beforeEach(async () => {
await truncateAll();
await engine.putPage('people/alice', { ...testPage, type: 'person', title: 'Alice' });
await engine.putPage('people/bob', { ...testPage, type: 'person', title: 'Bob' });
await engine.putPage('companies/acme', { ...testPage, type: 'company', title: 'Acme' });
});
test('returns Map<slug, count> for given slugs', async () => {
await engine.addLink('people/alice', 'companies/acme', '', 'works_at');
await engine.addLink('people/bob', 'companies/acme', '', 'invested_in');
const counts = await engine.getBacklinkCounts(['companies/acme', 'people/alice']);
expect(counts.get('companies/acme')).toBe(2);
expect(counts.get('people/alice')).toBe(0);
});
test('empty input -> empty Map', async () => {
const counts = await engine.getBacklinkCounts([]);
expect(counts.size).toBe(0);
});
test('slugs with zero links: present in Map with 0', async () => {
const counts = await engine.getBacklinkCounts(['people/alice']);
expect(counts.get('people/alice')).toBe(0);
});
});
describe('PGLiteEngine: traversePaths (v0.10.1)', () => {
beforeEach(async () => {
await truncateAll();
await engine.putPage('people/alice', { ...testPage, type: 'person', title: 'Alice' });
await engine.putPage('people/bob', { ...testPage, type: 'person', title: 'Bob' });
await engine.putPage('people/carol', { ...testPage, type: 'person', title: 'Carol' });
await engine.putPage('companies/acme', { ...testPage, type: 'company', title: 'Acme' });
await engine.putPage('meetings/standup', { ...testPage, type: 'meeting', title: 'Standup' });
// Build a small typed graph
await engine.addLink('meetings/standup', 'people/alice', '', 'attended');
await engine.addLink('meetings/standup', 'people/bob', '', 'attended');
await engine.addLink('meetings/standup', 'people/carol', '', 'attended');
await engine.addLink('people/alice', 'companies/acme', '', 'works_at');
await engine.addLink('people/bob', 'companies/acme', '', 'invested_in');
});
test('out direction (default): follows from->to edges', async () => {
const paths = await engine.traversePaths('meetings/standup', { depth: 1 });
expect(paths.length).toBe(3);
expect(new Set(paths.map(p => p.to_slug))).toEqual(new Set(['people/alice', 'people/bob', 'people/carol']));
expect(paths.every(p => p.link_type === 'attended')).toBe(true);
});
test('in direction: follows to->from edges', async () => {
const paths = await engine.traversePaths('companies/acme', { depth: 1, direction: 'in' });
expect(paths.length).toBe(2);
expect(new Set(paths.map(p => p.from_slug))).toEqual(new Set(['people/alice', 'people/bob']));
});
test('linkType per-edge filter: only follows matching edges', async () => {
const paths = await engine.traversePaths('companies/acme', {
depth: 1, direction: 'in', linkType: 'works_at',
});
expect(paths.length).toBe(1);
expect(paths[0].from_slug).toBe('people/alice');
});
test('depth 2: multi-hop traversal', async () => {
const paths = await engine.traversePaths('meetings/standup', { depth: 2 });
// alice/bob/carol direct + alice->acme + bob->acme
expect(paths.length).toBeGreaterThanOrEqual(5);
const acmePaths = paths.filter(p => p.to_slug === 'companies/acme');
expect(acmePaths.length).toBe(2);
expect(acmePaths.every(p => p.depth === 2)).toBe(true);
});
test('non-existent slug returns empty', async () => {
const paths = await engine.traversePaths('does/not-exist', { depth: 5 });
expect(paths).toEqual([]);
});
});
describe('PGLiteEngine: traverseGraph cycle prevention', () => {
beforeEach(async () => {
await truncateAll();
await engine.putPage('people/a', { ...testPage, type: 'person', title: 'A' });
await engine.putPage('people/b', { ...testPage, type: 'person', title: 'B' });
// Create a 2-cycle: A -> B -> A
await engine.addLink('people/a', 'people/b', '', 'mentions');
await engine.addLink('people/b', 'people/a', '', 'mentions');
});
test('does not amplify on cyclic graphs', async () => {
// Without cycle prevention, depth 5 on a 2-cycle would loop indefinitely
// (or at least produce many duplicate nodes). With the visited array, each
// node appears at most once.
const graph = await engine.traverseGraph('people/a', 5);
const slugs = graph.map(n => n.slug);
// Each slug should appear at most twice (once at depth 0, possibly once
// again at a deeper level via the cycle, but bounded by visited check).
const counts = new Map<string, number>();
for (const s of slugs) counts.set(s, (counts.get(s) ?? 0) + 1);
for (const [slug, count] of counts) {
expect(count).toBeLessThanOrEqual(2); // tolerate root + 1 traversal entry
void slug;
}
});
});
describe('PGLiteEngine: getHealth graph metrics', () => {
beforeEach(async () => {
await truncateAll();
await engine.putPage('people/alice', { ...testPage, type: 'person', title: 'Alice' });
await engine.putPage('people/bob', { ...testPage, type: 'person', title: 'Bob' });
await engine.putPage('companies/acme', { ...testPage, type: 'company', title: 'Acme' });
});
test('link_coverage = 0 when no links exist', async () => {
const h = await engine.getHealth();
expect(h.link_coverage).toBe(0);
});
test('link_coverage = % of entity pages with >= 1 inbound link', async () => {
// Acme gets 1 inbound link (from Alice), Alice/Bob get 0 inbound.
// 1 of 3 entity pages has inbound links -> 33%.
await engine.addLink('people/alice', 'companies/acme', '', 'works_at');
const h = await engine.getHealth();
expect(h.link_coverage).toBeCloseTo(1 / 3, 2);
});
test('timeline_coverage = % with >= 1 timeline entry', async () => {
await engine.addTimelineEntry('people/alice', { date: '2026-01-15', summary: 'Joined' });
const h = await engine.getHealth();
expect(h.timeline_coverage).toBeCloseTo(1 / 3, 2);
});
test('most_connected lists top entities by link count', async () => {
await engine.addLink('people/alice', 'companies/acme', '', 'works_at');
await engine.addLink('people/bob', 'companies/acme', '', 'invested_in');
const h = await engine.getHealth();
expect(h.most_connected.length).toBeGreaterThan(0);
expect(h.most_connected[0].slug).toBe('companies/acme');
expect(h.most_connected[0].link_count).toBe(2);
});
test('orphan_pages: pages with neither inbound nor outbound links', async () => {
// All 3 pages start with no links. Expect 3 orphans.
const h = await engine.getHealth();
expect(h.orphan_pages).toBe(3);
// Add alice -> acme. Alice has outbound, acme has inbound, only Bob is orphan.
await engine.addLink('people/alice', 'companies/acme', '', 'works_at');
const h2 = await engine.getHealth();
expect(h2.orphan_pages).toBe(1);
});
});
// ─────────────────────────────────────────────────────────────────
// v0.13.1 — PGLite.create() error-wrap (structural guard for #223)
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: v0.13.1 error-wrap on connect() (#223)', () => {
test('pglite-engine.ts source contains the wrap with #223 hint and nested original error', async () => {
const { readFileSync } = await import('fs');
const src = readFileSync('src/core/pglite-engine.ts', 'utf-8');
// Structural: the try/catch block must wrap PGlite.create() (the actual
// abort site, NOT engine-factory.ts). The error message must name the
// issue and suggest gbrain doctor. Must NOT suggest "missing migrations"
// as a cause (that was conflating #218 and #223 — migrations run AFTER
// create()).
expect(src).toContain('this._db = await PGlite.create');
expect(src).toContain('https://github.com/garrytan/gbrain/issues/223');
expect(src).toContain('gbrain doctor');
expect(src).toContain('Original error:');
// Regression guard: the user-visible error MESSAGE must not re-introduce
// the misleading "missing migrations" hint. (A source comment explaining
// *why* we removed it is fine — match only inside the wrapped Error body.)
const wrapStart = src.indexOf('const wrapped = new Error(');
expect(wrapStart).toBeGreaterThan(-1);
const wrapEnd = src.indexOf(');', wrapStart);
const errBody = src.slice(wrapStart, wrapEnd);
expect(errBody).not.toContain('missing migrations');
expect(errBody).not.toContain('apply-migrations');
});
});
// ─────────────────────────────────────────────────────────────────
// v0.13.1 — Engine kind discriminator
// ─────────────────────────────────────────────────────────────────
describe('PGLiteEngine: v0.13.1 kind discriminator', () => {
test('exposes readonly kind = pglite', () => {
expect(engine.kind).toBe('pglite');
});
});