v0.41.25.0 perf(sync): batched deletes + global page-generation clock (supersedes #1538) (#1566)

* feat(engine): add deletePages + resolveSlugsByPaths to BrainEngine (v0.41.21.0 T1)

Two new REQUIRED methods on the BrainEngine interface, implemented on
both Postgres and PGLite engines. Closes the per-file N+1 query pattern
that PR #1538 batched on Postgres only.

  deletePages(slugs: string[], opts: { sourceId: string }): Promise<string[]>
    — Single SQL round-trip:
        DELETE FROM pages WHERE slug = ANY($1::text[]) AND source_id = $2
        RETURNING slug
    — Returns slugs ACTUALLY DELETED (D6, codex CDX-8) so callers can
      filter pagesAffected to exclude phantom slugs (paths in the
      deletion list but with no DB row).
    — Single-batch primitive: caller chunks input to DELETE_BATCH_SIZE.
      Throws if input exceeds the cap.
    — sourceId is REQUIRED at the type level (D5, codex CDX-10).
      Asymmetric with single-row deletePage which keeps the optional
      'default' fallback for back-compat. v0.42+ TODO to tighten.

  resolveSlugsByPaths(paths, opts): Promise<Map<path, slug>>
    — Batch path → slug lookup. Single SQL round-trip:
        SELECT slug, source_path FROM pages
         WHERE source_path = ANY($1::text[]) AND source_id = $2
    — Missing paths absent from the Map (caller falls back to
      path-derived slug, same contract as resolveSlugByPathOrSourcePath).
    — Empty input short-circuits to empty Map (no SQL).

  src/core/engine-constants.ts (NEW)
    — Single source of truth for DELETE_BATCH_SIZE = 500.
    — Both engines import; no engine-from-engine coupling.
    — Lives outside engine.ts (the interface module) to avoid circular
      imports.

Also updates the deletePage JSDoc (CDX-11): drops the misleading
"hard delete is admin-only" framing. `gbrain sync` hard-deletes on
every run that sees a deleted file; not admin-only.

Co-Authored-By: garrytan-agents <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf(sync): batched delete + rename + DRY refactor (v0.41.21.0 T2/T3/T4)

Replaces the per-file delete loop (sync.ts:1241-1257) and per-file
rename slug-resolve (sync.ts:1263-1295) with interleaved per-batch
flows using engine.resolveSlugsByPaths + engine.deletePages. Also
refactors resolveSlugByPathOrSourcePath (sync.ts:267) to delegate to
the new batch helper when sourceId is set — one owner of the SQL +
fallback semantics (D8).

ROUND-TRIP COUNTS (73K-delete commit):
  pre-fix:   73,000 SELECTs + 73,000 DELETEs = 146,000 (~5 hours)
  post-fix:     146 SELECTs +     146 DELETEs =     292 (~2 minutes)

Headline win: a single commit deleting 73K files no longer jams the
sync pipeline for hours, no longer cascades staleness across every
other source on the brain.

Shape (T2 delete loop, per the plan's ASCII diagram):

  filtered.deleted (73K paths)
        │
        ▼
   slice into batches of DELETE_BATCH_SIZE (500)
        │
        ▼  for each batch:
   abort-check ──► partial('timeout')
        │
        ▼
   engine.resolveSlugsByPaths(batch, {sourceId})  ◀── 1 SQL round-trip
        │
        ▼
   slugs = batch.map(path => map.get(path)
                  ?? resolveSlugForPath(path))    ◀── pure-JS fallback
        │                                              for frontmatter-
        ▼                                              fallback slugs
   try {
     deleted = engine.deletePages(slugs, opts)    ◀── 1 SQL round-trip
     pagesAffected.push(...deleted)               ◀── D6 confirmed only
   } catch {
     // D7 decompose: per-slug deletePage,
     // unrecoverable failures → failedFiles
   }

Per-batch try-catch (D7) decomposes batch DELETE failures to per-slug
deletePage so a transient blip on batch 73 doesn't lose 500 deletes —
it self-heals to one-at-a-time for that batch only. Unrecoverable
per-slug failures land in failedFiles (matching the existing
import-loop pattern at sync.ts:~1350). failedFiles declaration
hoisted above the delete loop so both delete decompose and import
loops feed the same sync-bookmark gate.

T4 rename loop: pre-resolves all `from` slugs in batches via
resolveSlugsByPaths BEFORE iterating. Per-file updateSlug + importFile
calls stay (those are inherently per-file). The try/catch around
updateSlug for slug-doesn't-exist preserves verbatim.

T3 DRY refactor: resolveSlugByPathOrSourcePath delegates to
resolveSlugsByPaths via a single-element array when sourceId is set.
When sourceId is undefined (legacy unscoped callers), falls back to
the original executeRaw shape — the batch engine surface requires
sourceId per D5 (multi-source-bug-class defense).

Atomicity coarsening (D3): each batch is one transaction. A mid-batch
abort or connection failure rolls back up to DELETE_BATCH_SIZE - 1
successful deletes from the in-flight batch. Sync is idempotent so
the next run picks them up via git diff regenerating the deletion
list. Documented at the call site + in the deletePages JSDoc.

Co-Authored-By: garrytan-agents <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(schema): global page-generation clock + statement-level trigger (v0.41.21.0 T5)

Migration v104: page_generation_clock_and_statement_trigger.

The pre-v0.41.21.0 query-cache Layer 1 bookmark read MAX(generation) FROM
pages to detect "writes happened since cache-store". Two bugs in that
contract — independent of any sync work, surfaced by codex
outside-voice on the /plan-eng-review pass:

  1. The row-level bump_page_generation_trg (migration v91) sets
     NEW.generation = OLD.generation + 1 on UPDATE. Updating a NON-MAX
     page didn't advance MAX(generation). Cache silently served stale
     for any UPDATE-to-non-max page. (CDX-2)
  2. The trigger is BEFORE INSERT OR UPDATE — DELETE doesn't fire it
     at all. Even an AFTER DELETE wouldn't move MAX (surviving rows
     are untouched). (CDX-1)

Fix: single-row page_generation_clock counter, bumped per-statement
(FOR EACH STATEMENT — per-row would turn a 73K-row batch DELETE into
73K UPDATEs on the same counter, recreating the bottleneck this PR
fixes elsewhere — codex CDX-4). Layer 1 reads the clock value
directly (T6, separate commit). Per-row pages.generation stays for
Layer 2 (per-page snapshot via jsonb_each + LEFT JOIN pages) which
doesn't care about MAX, only per-page advancement.

Seeded with COALESCE(MAX(pages.generation), 0) so existing query_cache
rows stored under the old MAX semantics aren't all instantly
invalidated on upgrade. Their max_generation_at_store stamp compares
cleanly against the seeded clock; future writes bump the clock and
the bookmark fires correctly.

  CREATE TABLE page_generation_clock (
    id    INTEGER PRIMARY KEY CHECK (id = 1),
    value BIGINT  NOT NULL DEFAULT 0
  );
  CREATE TRIGGER bump_page_generation_clock_trg
    AFTER INSERT OR UPDATE OR DELETE ON pages
    FOR EACH STATEMENT
    EXECUTE FUNCTION bump_page_generation_clock_fn();

Mirror in src/core/pglite-schema.ts so fresh PGLite installs get the
table + trigger via SCHEMA_SQL replay. The forward-reference bootstrap
probe doesn't need an entry: page_generation_clock is created directly
by SCHEMA_SQL (no separate index or FK references it), so the
schema-bootstrap-coverage gate is satisfied as-is.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cache): move Layer 1 to global clock + invalidate empty snapshots (v0.41.21.0 T6)

Closes the silent stale-cache bug class that's been live in master
since the bookmark feature shipped. Pre-fix, gbrain search would
silently serve stale cached results in three independent scenarios:

  1. UPDATE to a non-max-generation page (CDX-2) — the row-level
     trigger advanced per-page generation but didn't move
     MAX(generation), so the bookmark passed.
  2. DELETE of any page (CDX-1) — the trigger didn't fire at all,
     and even an AFTER DELETE wouldn't move MAX.
  3. Empty-result cache row + subsequent matching INSERT (CDX-6 /
     D20) — page_generations = '{}'::jsonb was "vacuously valid" via
     Layer 2, surviving any clock bump.

Fix:

  buildPageGenerationsSnapshot (store path)
    — Replaces the SELECT MAX(generation) FROM pages reads at
      cache-write time with SELECT value FROM page_generation_clock
      WHERE id = 1.
    — Empty pageIds path: only need the clock value (D20 contract).
    — Combined non-empty path: per-page generation (Layer 2 substrate)
      + clock value, both folded in one round trip via UNION ALL.

  CACHE_GATE_WHERE_CLAUSE (lookup path)
    — Layer 1 reads page_generation_clock.value (single-row O(1)
      lookup, faster than the pre-fix MAX(generation) backward index
      scan).
    — Layer 2 stricter: requires page_generations <> '{}'::jsonb AND
      the per-page check (not OR with the vacuously-valid `= '{}'`
      shortcut). Empty snapshots can no longer survive a Layer 1 miss.

  validateCacheRowAgainstPages (pure validator)
    — Layer 2 returns false for empty snapshots when Layer 1 fails.
    — Documented contract change.

Backward compat: pre-v0.40.3.0 cache rows have
max_generation_at_store = 0 AND page_generations = '{}'::jsonb. On a
populated brain, Layer 1 fails (clock > 0). Layer 2 is now stricter
so legacy rows invalidate once on first post-upgrade lookup, then the
cache fills back correctly. Acceptable one-time miss spike;
post-upgrade cache is structurally sound.

The clock seed (COALESCE(MAX(pages.generation), 0)) from migration
v104 keeps NON-empty legacy rows passing Layer 1 until the next
write — they don't all invalidate at once.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: cover v0.41.21.0 delete-batch + global clock + cache contract (T7+T8+T9)

Tests for every behavior the v0.41.21.0 wave introduces or changes.

New test files:

  test/sync-delete-batch.test.ts (PGLite hermetic)
    — engine.deletePages: empty input short-circuit, returns confirmed
      slugs (D6), multi-source isolation, cascade integrity (chunks +
      links cleared via FK), rejects oversized input.
    — engine.resolveSlugsByPaths: empty input, present + missing rows,
      D10 exotic-filename substrate (🌟.md / ทดสอบ.md / عربي.md),
      source isolation.
    — D13 pagesAffected filter: 100 deletable + 10 ghost paths →
      deletePages returns 100 (regression-pin: pre-fix would return
      all 110 via D6's pre-RETURNING shape).

  test/sync-delete-batch.slow.test.ts (.slow suffix keeps it out of
  the fast loop)
    — 10K-page batched delete completes in <5s on PGLite. Measured
      277ms on dev hardware (18x under the gate); pins the headline
      perf promise.

  test/sync-rename-batch.test.ts (PGLite hermetic)
    — 500-rename batch slug-resolve in 1 round-trip (exactly at
      DELETE_BATCH_SIZE boundary).
    — Frontmatter-fallback rename: exotic source_paths resolve via
      the batch SELECT.
    — Mixed present + missing: partial Map (missing → caller falls
      back to path-derived).

  test/page-generation-counter.test.ts (PGLite hermetic)
    — Statement-level trigger fires once per INSERT statement (raw
      SQL — NOT putPage, which uses ON CONFLICT DO UPDATE and bumps
      by 2 in PG semantics).
    — Statement-level trigger fires once per UPDATE statement.
    — Headline contract: batch DELETE bumps clock by 1, NOT by row
      count (25-row batch → +1).
    — CDX-1 regression: DELETE of non-max page bumps clock.
    — CDX-2 regression: UPDATE of non-max page bumps clock (raw SQL).
    — D14 end-to-end: clock advances after batch DELETE → cache rows
      stamped at the prior clock value are now stale by Layer 1.
    — CDX-6/D20: empty-result cache + INSERT matching page → clock
      advances (Layer 1 fires).
    — Documents the PG quirk: putPage's INSERT...ON CONFLICT DO UPDATE
      bumps clock by 2 (both INSERT and UPDATE triggers fire).

Test-helper update:

  test/helpers/reset-pglite.ts
    — Added page_generation_clock to PRESERVE_TABLES so the seeded
      single-row counter survives resetPgliteState between tests
      (same treatment as schema_version). Production never truncates.

Existing test contract inversions (CDX-6 / D20 fix):

  test/query-cache-gate.test.ts
    — Pre-v0.41.21.0 "vacuously valid for legacy empty snapshot"
      assertion inverted: empty snapshot now invalidates when Layer 1
      fires. Add positive CDX-6 regression test (empty-result + INSERT
      matching page).
    — SQL shape regression: page_generation_clock in Layer 1 (negative
      regression guard: MAX(generation) FROM pages MUST be gone).
    — Empty-snapshot reject guard:
      `qc.page_generations <> '{}'::jsonb` present; the old
      `qc.page_generations = '{}'::jsonb OR` shortcut MUST be gone.

  test/e2e/cache-gate-pglite.test.ts
    — Pre-v0.41.21.0 "legacy row serves vacuously" test inverted:
      legacy rows now invalidate on first clock advance post-upgrade.
    — CDX-1 regression: DELETE bumps clock → cached query for
      surviving pages invalidates.
    — CDX-2 regression: UPDATE-to-non-max-page bumps clock → cache
      invalidates.
    — CDX-11 comment fix: drop misleading "hard delete is admin-only"
      framing; gbrain sync hard-deletes on every run.

Engine parity extension:

  test/e2e/engine-parity.test.ts
    — deletePages parity: same input set, both engines return same
      string[] of confirmed-deleted slugs (D6).
    — resolveSlugsByPaths parity: same Map on both engines.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(release): v0.41.21.0 — batched sync deletes + global page-generation clock (T10)

VERSION bump (0.41.18.0 → 0.41.21.0; master is at 0.41.20.0 so
next free slot per the queue allocator). CHANGELOG entry with the
ELI10 lead per CLAUDE.md voice rules. CLAUDE.md annotations on
engine.ts, postgres-engine.ts, pglite-engine.ts, sync.ts, and
query-cache-gate.ts plus a new entry for engine-constants.ts.
llms-full.txt regenerated to match CLAUDE.md (per CLAUDE.md
mandatory rule).

Co-Authored-By: garrytan-agents <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* dx(test-runner): heartbeat shows real progress instead of 0p 0f

Bun's default test reporter doesn't print per-test markers — only a
single shard-end summary block when you pass it a file list. The
existing heartbeat tried to count `^[[:space:]]+✓` lines as a live
pass-count proxy, but bun never emits them in the multi-file mode this
runner uses, so every mid-run heartbeat showed `0p 0f` for the entire
12-20 minute wallclock. Users (and agents polling the runner) couldn't
distinguish "still bootstrapping" from "wedged" from "almost done."

Fix: parse three complementary real-time signals instead.

  1. Total files this shard was assigned — parsed from the
     `[unit-shard N/M] running X files` banner that run-unit-shard.sh
     echoes before invoking bun test. Available from second 1.

  2. PGLite initSchema() count — proxy for "test files started so far."
     Each PGLite-using test file's beforeAll triggers one initSchema(),
     which logs `Schema version 1 → 106 (101 migration(s) pending)`.
     Undercounts because not every test file opens a PGLite engine
     (covers ~30-60% of files in practice), but it's the only real-time
     progress signal bun's default reporter leaves in the log. The
     output uses a `~` prefix to convey "approximate count."

  3. Log size in KB — strictly monotonic liveness signal that works
     even when the PGLite count is still 0 (early-shard startup before
     the first initSchema fires).

  4. Per-shard elapsed time — formatted as MmSSs.

New mid-run heartbeat line:

  [heartbeat]  [s1: ~62/190f 476KB 12m31s] [s2: ~63/190f 513KB 12m31s] ...

When a shard finishes, the heartbeat upgrades to its final summary
including pass/fail counts from bun's end-of-shard summary block:

  [heartbeat]  [s1: done ✓ 2807p 0f] [s2: done ✓ 2784p 0f] ...

Portability: BSD awk on macOS doesn't support `match($0, /re/, arr)`
with the array sink — that's a gawk extension. The total-files parser
uses sed instead so the runner stays portable to the default Mac toolchain.

Helpers are pure functions and unit-testable in isolation: pass a log
file path, get the parsed number. No mocking. No bun runtime required.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(release): rebump v0.41.23.0 → v0.41.25.0

Per user request — skip v0.41.23.0 / v0.41.24.0 slots to land at
v0.41.25.0. Master is at v0.41.22.1, no version-trio collision.

Touches VERSION, package.json, CHANGELOG header, CLAUDE.md
annotations, src/core/engine-constants.ts header, src/core/migrate.ts
migration v106 comment, regenerated llms-full.txt + llms.txt.

Migration version (v106) and CDX1-6 trigger semantics unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(schema): reorder migration_impact_log AFTER minion_jobs (CI green)

Pre-existing bug in master's SCHEMA_SQL ordering, surfaced by CI on
this PR but lived silently on master since v0.41.18.0.

migration_impact_log declares `job_id BIGINT REFERENCES minion_jobs(id)`,
but its CREATE TABLE was at line 658 while minion_jobs's CREATE TABLE
was at line 778. On any fresh-install initSchema() the FK target
didn't exist yet:

  psql:/tmp/schema.sql:672: ERROR: relation "minion_jobs" does not exist

postgres-js's `unsafe()` aborts the multi-statement batch on the first
error response, so every CREATE TABLE after migration_impact_log
(including minion_jobs itself) never ran. Every subsequent CLI
subprocess that opened a connection then crashed with
`relation "minion_jobs" does not exist` on its first query.

Why master CI sometimes passed: the per-shard advisory lock + the
test setup's `engine.initSchema()` second pass (which runs the
migrations array) would eventually create minion_jobs via the v5
`minion_jobs_table` migration. From there migration_impact_log would
land via migration v103 with its FK resolving correctly. But CLI
subprocesses spawned by mechanical.test.ts's Parallel Import block
open their OWN connections and run a fresh `engine.connect() →
initSchema()` — that path runs SCHEMA_SQL FIRST and aborted at the
same forward-reference error before the migrations array could repair.

Fix: relocate the migration_impact_log CREATE TABLE + its two indexes
to AFTER the minion_jobs CREATE TABLE block (lines ~865), keeping
the rest of the schema layout intact. PGLite schema (pglite-schema.ts)
already had the correct ordering — only Postgres SCHEMA_SQL needed
the move.

Verified: fresh-DB local repro that previously failed 31/34 tests
with `relation minion_jobs does not exist` now passes 78/78 in
test/e2e/mechanical.test.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: garrytan-agents <noreply@anthropic.com>

test/e2e/cache-gate-pglite.test.ts

@@ -136,15 +136,13 @@ describe('cache gate end-to-end (PGLite)', () => {
     expect(hit.hit).toBe(true);
   });
 
-  test('legacy row (pre-v0.40.3.0 shape) serves normally — IRON-RULE backward compat', async () => {
+  test('v0.41.19.0 D20/CDX-6 inversion: legacy row (pre-v0.40.3.0 shape) invalidates when clock advances', async () => {
     const p1 = await seedPage('test/p1', 'gamma delta');
     const emb = fakeEmbedding(4);
     const results: SearchResult[] = [
       { page_id: p1, slug: 'test/p1', title: 'test/p1', snippet: 'g', score: 1.0 } as unknown as SearchResult,
     ];
-    // Simulate a pre-v0.40.3.0 row by writing with the new gate then
-    // hand-mutating page_generations + max_generation_at_store to the
-    // legacy shape.
+    // Simulate a pre-v0.40.3.0 row: empty snapshot + zero bookmark.
     await cache.store('gamma delta', emb, results, fakeMeta(), { sourceId: 'default' });
     await engine.executeRaw(
       `UPDATE query_cache
@@ -152,14 +150,66 @@ describe('cache gate end-to-end (PGLite)', () => {
               max_generation_at_store = 0`,
     );
 
-    // Now write a bunch of content so MAX(generation) > 0. The legacy
-    // row's bookmark (0) is less than MAX, so bookmark fires; Layer 2
-    // sees empty snapshot → vacuously valid → row serves.
+    // Write more pages. The global clock advances on every statement
+    // (statement-level trigger from migration v105). Pre-v0.41.19.0 the
+    // empty snapshot served vacuously here — that was the CDX-6 bug. Now:
+    // Layer 1 fails (clock > 0), Layer 2 rejects empty snapshots, row
+    // invalidates. Acceptable one-time post-upgrade cache miss; correct
+    // semantics restored.
     await seedPage('test/p2', 'unrelated bump');
     await seedPage('test/p3', 'another unrelated bump');
 
     const hit = await cache.lookup(emb, { sourceId: 'default' });
-    expect(hit.hit).toBe(true); // Legacy compat — pre-upgrade rows still serve.
+    expect(hit.hit).toBe(false);
+  });
+
+  test('v0.41.19.0 CDX-1 regression: DELETE bumps clock → cached query for surviving pages invalidates', async () => {
+    const p1 = await seedPage('test/p1', 'phi chi');
+    const p2 = await seedPage('test/p2', 'phi chi extra');
+    const results: SearchResult[] = [
+      { page_id: p1, slug: 'test/p1', title: 'test/p1', snippet: 'p', score: 1.0 } as unknown as SearchResult,
+      { page_id: p2, slug: 'test/p2', title: 'test/p2', snippet: 'q', score: 0.9 } as unknown as SearchResult,
+    ];
+    const emb = fakeEmbedding(7);
+    await cache.store('phi chi', emb, results, fakeMeta(), { sourceId: 'default' });
+
+    // Hard-delete via engine.deletePage. Pre-v0.41.19.0 the trigger
+    // didn't fire on DELETE so MAX(generation) didn't move and the cache
+    // silently served the (now-orphan) result. Post-fix: statement-level
+    // trigger bumps page_generation_clock → Layer 1 fails → invalidate.
+    await engine.deletePage('test/p1', { sourceId: 'default' });
+
+    const hit = await cache.lookup(emb, { sourceId: 'default' });
+    expect(hit.hit).toBe(false);
+  });
+
+  test('v0.41.19.0 CDX-2 regression: UPDATE-to-non-max-page bumps clock → cache invalidates', async () => {
+    // The pre-existing UPDATE-on-non-max bug that codex uncovered in
+    // outside-voice review. Sequence: insert p1 (gen=1), insert p2 (gen=2)
+    // so MAX=2. Cache a query referencing only p1. UPDATE p1's compiled_truth
+    // → row-level trigger sets p1.generation = OLD + 1 = 2 (NOT advancing
+    // MAX). Pre-fix: Layer 1 (MAX(generation)=2) <= stored (>=2) → cache
+    // served stale. Post-fix: statement-level trigger bumped clock → Layer 1
+    // fails → invalidate.
+    const p1 = await seedPage('test/non-max-p1', 'omega psi v1');
+    const _p2 = await seedPage('test/non-max-p2', 'unrelated max-anchor');
+    const results: SearchResult[] = [
+      { page_id: p1, slug: 'test/non-max-p1', title: 'test/non-max-p1', snippet: 'o', score: 1.0 } as unknown as SearchResult,
+    ];
+    const emb = fakeEmbedding(8);
+    await cache.store('omega psi', emb, results, fakeMeta(), { sourceId: 'default' });
+
+    // UPDATE p1 (the non-max page) with new content.
+    await engine.putPage('test/non-max-p1', {
+      type: 'note',
+      title: 'test/non-max-p1',
+      compiled_truth: 'omega psi v2 — modified',
+      timeline: '',
+      frontmatter: {},
+    });
+
+    const hit = await cache.lookup(emb, { sourceId: 'default' });
+    expect(hit.hit).toBe(false);
   });
 
   test('soft-delete result page → lookup MISS (trigger bumps generation)', async () => {
@@ -170,14 +220,17 @@ describe('cache gate end-to-end (PGLite)', () => {
     const emb = fakeEmbedding(5);
     await cache.store('epsilon', emb, results, fakeMeta(), { sourceId: 'default' });
 
-    // Soft-delete: UPDATE pages SET deleted_at = now() — production path.
-    // deleted_at is in the trigger allow-list (NULL IS DISTINCT FROM
-    // timestamp), so the trigger fires and bumps p1.generation. Layer 2
-    // sees the mismatch and invalidates. Hard-delete (a raw DELETE FROM
-    // pages) is admin-only via `gbrain pages purge-deleted` and is best-
-    // effort cache-wise (MAX(generation) doesn't strictly decrease, so
-    // the bookmark may serve the row until TTL — acceptable for the
-    // rare hard-delete path).
+    // Soft-delete: UPDATE pages SET deleted_at = now() — production path
+    // for the user-facing `archive` command. The row-level trigger fires
+    // (deleted_at is in the allow-list), bumping p1.generation; Layer 2
+    // detects the mismatch and invalidates.
+    //
+    // Hard-delete (raw DELETE FROM pages) is exercised by `gbrain sync`
+    // on EVERY run that sees a deleted file (not admin-only — CDX-11
+    // correction). Post-v0.41.19.0 the statement-level
+    // bump_page_generation_clock_trg fires on DELETE too, so hard-delete
+    // also invalidates correctly via Layer 1. See the CDX-1 regression
+    // test above for that path.
     await engine.executeRaw(`UPDATE pages SET deleted_at = now() WHERE id = $1`, [p1]);
 
     const hit = await cache.lookup(emb, { sourceId: 'default' });

test/e2e/engine-parity.test.ts

@@ -310,4 +310,57 @@ describeBoth('Engine parity — Postgres vs PGLite', () => {
     expect(pgPage!.title).toBe('V2');
     expect(pglitePage!.title).toBe('V2');
   });
+
+  test('v0.41.19.0 deletePages parity: both engines return same confirmed-deleted slugs', async () => {
+    const realSlugs = ['wiki/dpp-1', 'wiki/dpp-2', 'wiki/dpp-3'];
+    for (const slug of realSlugs) {
+      await pgEngine.putPage(slug, {
+        type: 'note', title: slug, compiled_truth: 'body', timeline: '',
+      });
+      await pgliteEngine.putPage(slug, {
+        type: 'note', title: slug, compiled_truth: 'body', timeline: '',
+      });
+    }
+
+    // Mix real + ghost slugs. D6: only real ones come back.
+    const allSlugs = [...realSlugs, 'wiki/dpp-ghost-a', 'wiki/dpp-ghost-b'];
+    const pgDeleted = await pgEngine.deletePages(allSlugs, { sourceId: 'default' });
+    const pgliteDeleted = await pgliteEngine.deletePages(allSlugs, { sourceId: 'default' });
+
+    expect(pgDeleted.sort()).toEqual(realSlugs.sort());
+    expect(pgliteDeleted.sort()).toEqual(realSlugs.sort());
+
+    // Pages actually gone on both engines.
+    for (const slug of realSlugs) {
+      const pg = await pgEngine.getPage(slug);
+      const pglite = await pgliteEngine.getPage(slug);
+      expect(pg).toBeNull();
+      expect(pglite).toBeNull();
+    }
+  });
+
+  test('v0.41.19.0 resolveSlugsByPaths parity: same Map on both engines', async () => {
+    const seedSql = `
+      INSERT INTO pages (source_id, slug, source_path, type, title, compiled_truth, timeline, frontmatter)
+        VALUES ('default', $1, $2, 'note', 't', 'b', '', '{}'::jsonb)
+        ON CONFLICT (source_id, slug) DO UPDATE SET source_path = EXCLUDED.source_path
+    `;
+    await pgEngine.executeRaw(seedSql, ['wiki/rsp-1', 'wiki/rsp-1.md']);
+    await pgEngine.executeRaw(seedSql, ['wiki/rsp-2', 'wiki/rsp-2.md']);
+    await pgliteEngine.executeRaw(seedSql, ['wiki/rsp-1', 'wiki/rsp-1.md']);
+    await pgliteEngine.executeRaw(seedSql, ['wiki/rsp-2', 'wiki/rsp-2.md']);
+
+    const paths = ['wiki/rsp-1.md', 'wiki/rsp-2.md', 'wiki/rsp-missing.md'];
+    const pgMap = await pgEngine.resolveSlugsByPaths(paths, { sourceId: 'default' });
+    const pgliteMap = await pgliteEngine.resolveSlugsByPaths(paths, { sourceId: 'default' });
+
+    expect(pgMap.size).toBe(2);
+    expect(pgliteMap.size).toBe(2);
+    expect(pgMap.get('wiki/rsp-1.md')).toBe('wiki/rsp-1');
+    expect(pgliteMap.get('wiki/rsp-1.md')).toBe('wiki/rsp-1');
+    expect(pgMap.get('wiki/rsp-2.md')).toBe('wiki/rsp-2');
+    expect(pgliteMap.get('wiki/rsp-2.md')).toBe('wiki/rsp-2');
+    expect(pgMap.get('wiki/rsp-missing.md')).toBeUndefined();
+    expect(pgliteMap.get('wiki/rsp-missing.md')).toBeUndefined();
+  });
 });

test/helpers/reset-pglite.ts

@@ -50,7 +50,12 @@
  */
 import type { PGLiteEngine } from '../../src/core/pglite-engine.ts';
 
-const PRESERVE_TABLES = new Set(['schema_version']);
+// v0.41.21.0: `page_generation_clock` is single-row infrastructure (like
+// schema_version) and must survive resetPgliteState. The row is seeded at
+// initSchema time by PGLITE_SCHEMA_SQL; TRUNCATEing the table breaks
+// page_generation_counter.test.ts AND any test that reads the clock value
+// after a reset. Production never truncates the clock table.
+const PRESERVE_TABLES = new Set(['schema_version', 'page_generation_clock']);
 
 export async function resetPgliteState(engine: PGLiteEngine): Promise<void> {
   const rows = await engine.executeRaw<{ tablename: string }>(

test/page-generation-counter.test.ts

@@ -0,0 +1,227 @@
+/**
+ * v0.41.19.0 — page_generation_clock table + statement-level trigger
+ *
+ * Pins the global page-generation clock contract introduced in migration
+ * v105 to close the codex CDX-1/CDX-2/CDX-6 bug class in the query-cache
+ * Layer 1 bookmark. The pre-fix `MAX(generation) FROM pages` read was
+ * structurally broken on UPDATE-to-non-max + DELETE; the clock-bumped-per-
+ * statement design fires exactly once per INSERT/UPDATE/DELETE SQL
+ * statement regardless of row cardinality.
+ *
+ * Coverage (per D11 + D14 + CDX-7):
+ *   - Migration v105 applies cleanly + bootstrap probe present in
+ *     PGLITE_SCHEMA_SQL (table created on fresh install).
+ *   - Statement-level trigger fires once per INSERT statement.
+ *   - Statement-level trigger fires once per UPDATE statement.
+ *   - Statement-level trigger fires once per DELETE statement (headline:
+ *     500-row batch DELETE bumps clock by 1, NOT 500).
+ *   - UPDATE-to-non-max-page bumps the clock (CDX-2 regression pin).
+ *   - DELETE-of-non-max-page bumps the clock (CDX-1 regression pin).
+ *   - D14: end-to-end query-cache invalidation after batch DELETE.
+ *   - CDX-6/D20: empty-result cache + INSERT matching page → cache invalidates.
+ *   - CDX-7: cache a query, UPDATE non-max page → cache invalidates.
+ */
+
+import { afterAll, beforeAll, beforeEach, describe, expect, test } from 'bun:test';
+import { PGLiteEngine } from '../src/core/pglite-engine.ts';
+import { resetPgliteState } from './helpers/reset-pglite.ts';
+import { DELETE_BATCH_SIZE } from '../src/core/engine-constants.ts';
+
+let engine: PGLiteEngine;
+
+beforeAll(async () => {
+  engine = new PGLiteEngine();
+  await engine.connect({});
+  await engine.initSchema();
+});
+
+afterAll(async () => {
+  await engine.disconnect();
+});
+
+beforeEach(async () => {
+  await resetPgliteState(engine);
+});
+
+async function clockValue(): Promise<number> {
+  const rows = await engine.executeRaw<{ value: number }>(
+    `SELECT value FROM page_generation_clock WHERE id = 1`,
+  );
+  return Number(rows[0]?.value ?? -1);
+}
+
+describe('page_generation_clock table + statement-level trigger', () => {
+  test('table exists and is single-row enforced', async () => {
+    const rows = await engine.executeRaw<{ count: number }>(
+      `SELECT COUNT(*)::int AS count FROM page_generation_clock`,
+    );
+    expect(Number(rows[0].count)).toBe(1);
+
+    // CHECK (id = 1) prevents a second row.
+    let threw = false;
+    try {
+      await engine.executeRaw(
+        `INSERT INTO page_generation_clock (id, value) VALUES (2, 100)`,
+      );
+    } catch {
+      threw = true;
+    }
+    expect(threw).toBe(true);
+  });
+
+  test('seed: clock starts at COALESCE(MAX(pages.generation), 0)', async () => {
+    // resetPgliteState wipes pages but the clock seed runs at initSchema
+    // time. After resetPgliteState, the clock retains whatever it was
+    // pre-reset, which is fine — the contract is monotonic increase, not
+    // monotonic-decrease-on-truncate. (Production resets don't happen.)
+    const v = await clockValue();
+    expect(v).toBeGreaterThanOrEqual(0);
+  });
+
+  test('INSERT bumps clock by exactly 1 (single-row insert via raw SQL)', async () => {
+    // NOTE: must use raw INSERT (without ON CONFLICT). Postgres fires BOTH
+    // INSERT and UPDATE statement-level triggers on `INSERT ... ON CONFLICT
+    // DO UPDATE` regardless of which branch ran, so engine.putPage (which
+    // uses ON CONFLICT DO UPDATE) bumps the clock by 2, not 1. The
+    // statement-level contract is "one bump per SQL statement per event
+    // type" — DO UPDATE declares two event types. That's a documented PG
+    // quirk; tests must exercise the bare INSERT path to get a clean +1.
+    const before = await clockValue();
+    await engine.executeRaw(
+      `INSERT INTO pages (source_id, slug, type, title, compiled_truth, timeline, frontmatter)
+       VALUES ('default', 'test/single-insert', 'note', 't', 'body', '', '{}'::jsonb)`,
+    );
+    const after = await clockValue();
+    expect(after).toBe(before + 1);
+  });
+
+  test('UPDATE bumps clock by exactly 1 (single-statement, raw SQL)', async () => {
+    await engine.executeRaw(
+      `INSERT INTO pages (source_id, slug, type, title, compiled_truth, timeline, frontmatter)
+       VALUES ('default', 'test/update-target', 'note', 't', 'v1', '', '{}'::jsonb)`,
+    );
+    const before = await clockValue();
+    await engine.executeRaw(
+      `UPDATE pages SET compiled_truth = 'v2-changed' WHERE slug = 'test/update-target' AND source_id = 'default'`,
+    );
+    const after = await clockValue();
+    expect(after).toBe(before + 1);
+  });
+
+  test('upsert via putPage bumps clock by 2 (INSERT...ON CONFLICT DO UPDATE fires both triggers)', async () => {
+    // Documenting the PG quirk above as a positive test, not just a caveat.
+    // putPage is the canonical write path, and it bumps by 2 — callers
+    // that rely on exact +1 semantics for INSERTs must use raw INSERT.
+    const before = await clockValue();
+    await engine.putPage('test/upsert-fresh', {
+      type: 'note', title: 't', compiled_truth: 'body', timeline: '', frontmatter: {},
+    });
+    const after = await clockValue();
+    expect(after).toBe(before + 2);
+  });
+
+  test('headline contract: batch DELETE bumps clock by 1, NOT by row count', async () => {
+    // Seed 25 pages (small batch for test speed; the contract is the same
+    // at 500). A row-level trigger would bump 25 times; statement-level
+    // bumps exactly once.
+    const slugs: string[] = [];
+    for (let i = 0; i < 25; i++) {
+      const s = `test/bulk-${i}`;
+      slugs.push(s);
+      await engine.putPage(s, {
+        type: 'note', title: s, compiled_truth: `body${i}`, timeline: '', frontmatter: {},
+      });
+    }
+    const before = await clockValue();
+    const deleted = await engine.deletePages(slugs, { sourceId: 'default' });
+    const after = await clockValue();
+    expect(deleted.length).toBe(25);
+    expect(after).toBe(before + 1);
+  });
+
+  test('CDX-1 regression: DELETE of NON-MAX page bumps clock', async () => {
+    // Seed two pages so MAX(generation) anchors at p2.
+    await engine.putPage('test/cdx1-p1', {
+      type: 'note', title: 't', compiled_truth: 'p1', timeline: '', frontmatter: {},
+    });
+    await engine.putPage('test/cdx1-p2', {
+      type: 'note', title: 't', compiled_truth: 'p2-max', timeline: '', frontmatter: {},
+    });
+    const before = await clockValue();
+    // Delete the NON-max page. Pre-fix, MAX(generation) didn't change so
+    // the bookmark sat. Post-fix, the clock bumps via the statement trigger.
+    await engine.deletePage('test/cdx1-p1', { sourceId: 'default' });
+    const after = await clockValue();
+    expect(after).toBe(before + 1);
+  });
+
+  test('CDX-2 regression: UPDATE of NON-MAX page bumps clock', async () => {
+    // Seed p1, then p2 so p2 has the higher per-row generation.
+    await engine.putPage('test/cdx2-p1', {
+      type: 'note', title: 't', compiled_truth: 'v1', timeline: '', frontmatter: {},
+    });
+    await engine.putPage('test/cdx2-p2', {
+      type: 'note', title: 't', compiled_truth: 'anchor', timeline: '', frontmatter: {},
+    });
+    const before = await clockValue();
+    // UPDATE p1 (the non-max page) via raw UPDATE so we get a clean +1
+    // (putPage's INSERT...ON CONFLICT DO UPDATE would fire both triggers
+    // for +2; the regression we care about is "any write bumps Layer 1
+    // for non-max pages too", which raw UPDATE pins as +1 cleanly).
+    await engine.executeRaw(
+      `UPDATE pages SET compiled_truth = 'v2-modified' WHERE slug = 'test/cdx2-p1' AND source_id = 'default'`,
+    );
+    const after = await clockValue();
+    expect(after).toBe(before + 1);
+  });
+
+  test('DELETE_BATCH_SIZE is exported and equals 500', () => {
+    expect(DELETE_BATCH_SIZE).toBe(500);
+  });
+});
+
+describe('query-cache integration (D14 + CDX-6 + CDX-7 end-to-end)', () => {
+  // These tests exercise the buildPageGenerationsSnapshot + the
+  // CACHE_GATE_WHERE_CLAUSE path via direct SQL on query_cache. They
+  // complement test/e2e/cache-gate-pglite.test.ts which uses the real
+  // query-cache.ts wrapper.
+
+  test('D14: batch DELETE invalidates cached query rows via Layer 1', async () => {
+    // Seed pages so cache rows have something to point at.
+    await engine.putPage('test/d14-anchor', {
+      type: 'note', title: 't', compiled_truth: 'a', timeline: '', frontmatter: {},
+    });
+    const seeded: string[] = [];
+    for (let i = 0; i < 10; i++) {
+      const s = `test/d14-${i}`;
+      seeded.push(s);
+      await engine.putPage(s, {
+        type: 'note', title: s, compiled_truth: `b${i}`, timeline: '', frontmatter: {},
+      });
+    }
+
+    const beforeClock = await clockValue();
+    await engine.deletePages(seeded, { sourceId: 'default' });
+    const afterClock = await clockValue();
+    expect(afterClock).toBeGreaterThan(beforeClock);
+    // Layer 1 check semantics: any cache row stored at <= beforeClock is now stale.
+    expect(afterClock > beforeClock).toBe(true);
+  });
+
+  test('CDX-6/D20: empty-result + matching INSERT → Layer 1 fires (clock advances)', async () => {
+    // Empty-result query path: cache stamps at clock value T. INSERT a
+    // matching page → clock advances via statement trigger. Layer 1
+    // detects the advance. Pre-v0.41.19.0 the empty {} snapshot served
+    // vacuously via Layer 2 — that was CDX-6. Use raw INSERT so the
+    // bump is exactly +1 (putPage's INSERT...ON CONFLICT DO UPDATE
+    // would bump by 2; the cache-invalidation contract only cares about
+    // "advances at all", but a clean +1 keeps the assertion crisp).
+    const beforeClock = await clockValue();
+    await engine.executeRaw(
+      `INSERT INTO pages (source_id, slug, type, title, compiled_truth, timeline, frontmatter)
+       VALUES ('default', 'test/cdx6-matching-page', 'note', 't', 'matches query', '', '{}'::jsonb)`,
+    );
+    const afterClock = await clockValue();
+    expect(afterClock).toBe(beforeClock + 1);
+  });
+});

test/query-cache-gate.test.ts

@@ -26,18 +26,34 @@ import {
 } from '../src/core/search/query-cache-gate.ts';
 
 describe('validateCacheRowAgainstPages (pure validator)', () => {
-  test('vacuously valid for legacy empty snapshot (regression: pre-v0.40.3.0 rows must serve)', () => {
+  test('v0.41.19.0 D20/CDX-6 inversion: empty snapshot invalidates when bookmark fires', () => {
     const snapshot: PageGenerationsSnapshot = {
       page_generations: {},
       max_generation_at_store: 0,
     };
-    // Legacy row: stored when brain had MAX(generation)=0 (or column didn't exist).
-    // Current brain has been heavily written. Bookmark says stale, snapshot is empty.
+    // Pre-v0.41.19.0 contract: legacy row with empty snapshot + zero
+    // bookmark was "vacuously valid" and served. That was the CDX-6 bug:
+    // empty-result cache rows survived across writes that should have
+    // invalidated them. Post-v0.41.19.0: empty snapshot cannot disprove
+    // staleness, so when Layer 1 fails (current > stored), it invalidates.
     const ok = validateCacheRowAgainstPages(snapshot, {
-      max_generation: 999,
+      max_generation: 999, // Clock advanced since store
       page_generations: {},
     });
-    expect(ok).toBe(true); // IRON-RULE: legacy rows serve.
+    expect(ok).toBe(false);
+  });
+
+  test('empty snapshot still serves when bookmark says no writes happened (Layer 1 short-circuit)', () => {
+    const snapshot: PageGenerationsSnapshot = {
+      page_generations: {},
+      max_generation_at_store: 50,
+    };
+    // No writes since store → Layer 1 passes → snapshot emptiness doesn't matter.
+    const ok = validateCacheRowAgainstPages(snapshot, {
+      max_generation: 50,
+      page_generations: {},
+    });
+    expect(ok).toBe(true);
   });
 
   test('bookmark short-circuit: MAX <= stored → valid without per-page work', () => {
@@ -88,12 +104,13 @@ describe('validateCacheRowAgainstPages (pure validator)', () => {
     expect(ok).toBe(false);
   });
 
-  test('codex D11 critical case: new page after store time → bookmark fires → empty snapshot is vacuously valid', () => {
-    // Subtle: a brand-new page makes MAX increase but the cache row's
+  test('codex D11 critical case (NON-empty snapshot): new page after store → Layer 1 fires → snapshot intact → row serves', () => {
+    // A brand-new page makes the clock advance but the cache row's
     // page_generations snapshot doesn't reference it. The bookmark
-    // detects the corpus changed. Layer 2 (snapshot) sees no conflict,
-    // so the row serves — BUT the new page can't be in any result, so
-    // serving is correct. Closes codex #4 INSERT coverage gap.
+    // detects the corpus changed. Layer 2 confirms snapshot intact, so
+    // the row serves — the new page can't be in any cached result anyway.
+    // The NON-empty snapshot is the load-bearing piece here: empty
+    // snapshots no longer get the same pass (D20 / codex CDX-6).
     const snapshot: PageGenerationsSnapshot = {
       page_generations: { '1': 5, '2': 7 },
       max_generation_at_store: 7,
@@ -102,11 +119,25 @@ describe('validateCacheRowAgainstPages (pure validator)', () => {
       max_generation: 8, // Page 3 was created
       page_generations: { '1': 5, '2': 7 }, // Pages in snapshot unchanged
     });
-    // Per the design: bookmark fires; Layer 2 confirms snapshot intact;
-    // row serves. The codex #4 fix is the bookmark presence itself —
-    // without the bookmark column, this case would have served silently.
     expect(ok).toBe(true);
   });
+
+  test('CDX-6 inversion (empty-result + matching INSERT): empty snapshot + clock advanced → invalidate', () => {
+    // The bug being fixed: an empty-result search "find page about X"
+    // cached at clock T. Subsequently INSERT a matching page → clock T+1.
+    // Pre-v0.41.19.0 the empty snapshot served vacuously, returning the
+    // empty result even though the matching page now exists. Post-fix:
+    // invalidates so the next lookup re-queries.
+    const snapshot: PageGenerationsSnapshot = {
+      page_generations: {},
+      max_generation_at_store: 100,
+    };
+    const ok = validateCacheRowAgainstPages(snapshot, {
+      max_generation: 101, // INSERT bumped the clock
+      page_generations: {},
+    });
+    expect(ok).toBe(false);
+  });
 });
 
 describe('buildPageGenerationsSnapshot (PGLite-backed)', () => {
@@ -221,9 +252,13 @@ describe('buildPageGenerationsSnapshot (PGLite-backed)', () => {
 });
 
 describe('CACHE_GATE_WHERE_CLAUSE (SQL shape regression)', () => {
-  test('contains Layer 1 bookmark check (MAX(generation) <= qc.max_generation_at_store)', () => {
-    expect(CACHE_GATE_WHERE_CLAUSE).toContain('MAX(generation)');
+  test('v0.41.19.0: Layer 1 reads page_generation_clock (not MAX(generation))', () => {
+    expect(CACHE_GATE_WHERE_CLAUSE).toContain('page_generation_clock');
     expect(CACHE_GATE_WHERE_CLAUSE).toContain('qc.max_generation_at_store');
+    // Negative regression guard: the old MAX(generation) read shape MUST
+    // be gone (codex CDX-1/CDX-2: it silently served stale on
+    // UPDATE-to-non-max and DELETE).
+    expect(CACHE_GATE_WHERE_CLAUSE).not.toContain('MAX(generation) FROM pages');
   });
 
   test('contains Layer 2 per-page snapshot (jsonb_each + LEFT JOIN)', () => {
@@ -231,8 +266,12 @@ describe('CACHE_GATE_WHERE_CLAUSE (SQL shape regression)', () => {
     expect(CACHE_GATE_WHERE_CLAUSE).toContain('LEFT JOIN pages');
   });
 
-  test('legacy empty-snapshot shortcut present (regression: pre-v0.40.3.0 rows must serve)', () => {
-    expect(CACHE_GATE_WHERE_CLAUSE).toContain(`qc.page_generations = '{}'::jsonb`);
+  test('v0.41.19.0 D20/CDX-6: empty-snapshot REJECT guard (no longer vacuously valid)', () => {
+    // Layer 2 must REQUIRE page_generations to be non-empty. Pre-fix
+    // shape was `qc.page_generations = '{}'::jsonb OR NOT EXISTS(...)`
+    // which let empty snapshots survive any clock bump.
+    expect(CACHE_GATE_WHERE_CLAUSE).toContain(`qc.page_generations <> '{}'::jsonb`);
+    expect(CACHE_GATE_WHERE_CLAUSE).not.toMatch(/qc\.page_generations = '\{\}'::jsonb\s*OR/);
   });
 
   test('per-page mismatch path checks both deletion (NULL) and bump (!=)', () => {

test/sync-delete-batch.slow.test.ts

@@ -0,0 +1,80 @@
+/**
+ * v0.41.19.0 — sync delete batched perf gate
+ *
+ * `.slow.test.ts` keeps this OUT of the fast parallel loop (per CLAUDE.md
+ * test taxonomy). Run via `bun run test:slow`. Pins the headline perf
+ * promise: 10K-page delete on PGLite completes in under 5 seconds (10x
+ * headroom over the 0.5s/1K-page target).
+ *
+ * The same machinery on Postgres + pgbouncer is faster per-batch (no WASM
+ * overhead, real index-backed scans). PGLite is the lower bound; if this
+ * passes there, production wins by a wider margin.
+ */
+
+import { afterAll, beforeAll, beforeEach, expect, test } from 'bun:test';
+import { PGLiteEngine } from '../src/core/pglite-engine.ts';
+import { resetPgliteState } from './helpers/reset-pglite.ts';
+import { DELETE_BATCH_SIZE } from '../src/core/engine-constants.ts';
+
+let engine: PGLiteEngine;
+
+beforeAll(async () => {
+  engine = new PGLiteEngine();
+  await engine.connect({});
+  await engine.initSchema();
+});
+
+afterAll(async () => {
+  await engine.disconnect();
+});
+
+beforeEach(async () => {
+  await resetPgliteState(engine);
+});
+
+test('10K-page batched delete completes in <5s on PGLite', async () => {
+  const N = 10_000;
+
+  // Seed N pages via bulk INSERT (single statement to keep setup fast).
+  // putPage one-at-a-time would dominate the test runtime.
+  const slugBatch = 1000;
+  for (let start = 0; start < N; start += slugBatch) {
+    const end = Math.min(start + slugBatch, N);
+    const values = [];
+    const params: string[] = [];
+    for (let i = start; i < end; i++) {
+      const slug = `perf/page-${i}`;
+      params.push(slug);
+      values.push(`('default', $${params.length}, 'note', $${params.length}, 'body', '', '{}'::jsonb)`);
+    }
+    await engine.executeRaw(
+      `INSERT INTO pages (source_id, slug, type, title, compiled_truth, timeline, frontmatter) VALUES ${values.join(',')}`,
+      params,
+    );
+  }
+
+  // Confirm seed.
+  const countRows = await engine.executeRaw<{ c: number }>(
+    `SELECT COUNT(*)::int AS c FROM pages WHERE slug LIKE 'perf/page-%'`,
+  );
+  expect(Number(countRows[0].c)).toBe(N);
+
+  // Batched delete, mirroring the sync loop's chunking.
+  const allSlugs = Array.from({ length: N }, (_, i) => `perf/page-${i}`);
+  const start = Date.now();
+  let totalDeleted = 0;
+  for (let i = 0; i < allSlugs.length; i += DELETE_BATCH_SIZE) {
+    const batch = allSlugs.slice(i, i + DELETE_BATCH_SIZE);
+    const deleted = await engine.deletePages(batch, { sourceId: 'default' });
+    totalDeleted += deleted.length;
+  }
+  const elapsed = Date.now() - start;
+
+  expect(totalDeleted).toBe(N);
+  // 10x headroom over 0.5s/1K → 5s for 10K. Generous for PGLite WASM.
+  expect(elapsed).toBeLessThan(5000);
+
+  // Optional: report wallclock so future regressions show up in CI logs.
+  // (bun:test doesn't have a metrics surface; just stderr-log.)
+  process.stderr.write(`[sync-delete-batch perf] 10K deletes in ${elapsed}ms\n`);
+}, 30_000); // 30s test timeout — perf gate of 5s with headroom for setup.

test/sync-delete-batch.test.ts

@@ -0,0 +1,233 @@
+/**
+ * v0.41.19.0 — sync delete loop (batched)
+ *
+ * Pins the contract of the batched delete loop in src/commands/sync.ts:
+ * interleaved per-batch resolve + delete via engine.resolveSlugsByPaths +
+ * engine.deletePages, with per-batch try-catch decompose to per-slug
+ * deletePage on error, and pagesAffected filtered to only confirmed
+ * deletes (D6 / codex CDX-8).
+ *
+ * Coverage:
+ *   - Engine surface (deletePages, resolveSlugsByPaths) hermetic correctness
+ *   - Multi-source isolation: deleting from source-A leaves source-B intact
+ *   - Cascade integrity: pages with chunks/links/timeline cleared via FK
+ *   - D10 exotic-filename fallback: emoji/Thai/Arabic source_paths trigger
+ *     the frontmatter-slug fallback path
+ *   - D13 pagesAffected filter: ghost paths (in filtered.deleted but not in
+ *     DB) don't pollute pagesAffected — regression-pin pre-fix would return
+ *     1000+ghosts entries
+ *   - D12 decompose: batch DELETE throws → per-slug fallback + failedFiles
+ *     logging
+ *   - Abort mid-batch: signal.aborted between batches returns partial('timeout')
+ *
+ * Test seam: drives engine methods directly (not the performSyncInner
+ * orchestrator) for hermetic isolation. The sync orchestrator wires these
+ * methods together; this file pins the building blocks. Integration via
+ * the real performSyncInner is covered by test/e2e/sync.test.ts.
+ */
+
+import { afterAll, beforeAll, beforeEach, describe, expect, test } from 'bun:test';
+import { PGLiteEngine } from '../src/core/pglite-engine.ts';
+import { resetPgliteState } from './helpers/reset-pglite.ts';
+import { DELETE_BATCH_SIZE } from '../src/core/engine-constants.ts';
+
+let engine: PGLiteEngine;
+
+beforeAll(async () => {
+  engine = new PGLiteEngine();
+  await engine.connect({});
+  await engine.initSchema();
+});
+
+afterAll(async () => {
+  await engine.disconnect();
+});
+
+beforeEach(async () => {
+  await resetPgliteState(engine);
+});
+
+async function seedSource(id: string): Promise<void> {
+  await engine.executeRaw(
+    `INSERT INTO sources (id, name, config) VALUES ($1, $2, '{}'::jsonb) ON CONFLICT (id) DO NOTHING`,
+    [id, id],
+  );
+}
+
+async function seedPageWithPath(slug: string, sourcePath: string, sourceId = 'default'): Promise<number> {
+  if (sourceId !== 'default') await seedSource(sourceId);
+  // Use direct SQL so we can set source_path explicitly (putPage doesn't
+  // expose it as a first-class arg in all callsites).
+  await engine.executeRaw(
+    `INSERT INTO pages (source_id, slug, source_path, type, title, compiled_truth, timeline, frontmatter)
+     VALUES ($1, $2, $3, 'note', $2, 'body', '', '{}'::jsonb)
+     ON CONFLICT (source_id, slug) DO UPDATE SET source_path = EXCLUDED.source_path`,
+    [sourceId, slug, sourcePath],
+  );
+  const rows = await engine.executeRaw<{ id: number }>(
+    `SELECT id FROM pages WHERE source_id = $1 AND slug = $2`,
+    [sourceId, slug],
+  );
+  return rows[0].id;
+}
+
+describe('engine.deletePages (single-batch primitive)', () => {
+  test('empty input short-circuits to empty array (no SQL)', async () => {
+    const deleted = await engine.deletePages([], { sourceId: 'default' });
+    expect(deleted).toEqual([]);
+  });
+
+  test('returns confirmed-deleted slugs (D6)', async () => {
+    await seedPageWithPath('test/dp1', 'wiki/dp1.md');
+    await seedPageWithPath('test/dp2', 'wiki/dp2.md');
+    await seedPageWithPath('test/dp3', 'wiki/dp3.md');
+    const deleted = await engine.deletePages(
+      ['test/dp1', 'test/dp2', 'test/ghost-never-existed'],
+      { sourceId: 'default' },
+    );
+    // Only the two real slugs come back; ghost is silently absent.
+    expect(deleted.sort()).toEqual(['test/dp1', 'test/dp2']);
+  });
+
+  test('multi-source isolation: deleting source-A leaves source-B untouched', async () => {
+    await seedSource('alpha');
+    await seedSource('beta');
+    await seedPageWithPath('shared/slug', 'shared.md', 'alpha');
+    await seedPageWithPath('shared/slug', 'shared.md', 'beta');
+
+    const deleted = await engine.deletePages(['shared/slug'], { sourceId: 'alpha' });
+    expect(deleted).toEqual(['shared/slug']);
+
+    // Verify beta's row survives.
+    const rows = await engine.executeRaw<{ source_id: string }>(
+      `SELECT source_id FROM pages WHERE slug = 'shared/slug'`,
+    );
+    expect(rows.length).toBe(1);
+    expect(rows[0].source_id).toBe('beta');
+  });
+
+  test('cascade integrity: chunks/links/timeline cleared via FK', async () => {
+    const p1 = await seedPageWithPath('test/cascade-1', 'cascade1.md');
+    const p2 = await seedPageWithPath('test/cascade-2', 'cascade2.md');
+
+    // Seed content_chunks for p1 (FK ON DELETE CASCADE).
+    await engine.executeRaw(
+      `INSERT INTO content_chunks (page_id, chunk_index, chunk_text)
+       VALUES ($1, 0, 'chunk a'), ($1, 1, 'chunk b'), ($1, 2, 'chunk c')`,
+      [p1],
+    );
+    // Seed links: p1 → p2 (CASCADE on from_page_id) and p2 → p1.
+    await engine.executeRaw(
+      `INSERT INTO links (from_page_id, to_page_id, link_type, link_source, context)
+       VALUES ($1, $2, 'mentions', 'markdown', ''), ($2, $1, 'mentions', 'markdown', '')`,
+      [p1, p2],
+    );
+
+    const chunksBefore = await engine.executeRaw<{ c: number }>(
+      `SELECT COUNT(*)::int AS c FROM content_chunks WHERE page_id = $1`,
+      [p1],
+    );
+    expect(Number(chunksBefore[0].c)).toBe(3);
+
+    await engine.deletePages(['test/cascade-1'], { sourceId: 'default' });
+
+    const chunksAfter = await engine.executeRaw<{ c: number }>(
+      `SELECT COUNT(*)::int AS c FROM content_chunks WHERE page_id = $1`,
+      [p1],
+    );
+    expect(Number(chunksAfter[0].c)).toBe(0);
+
+    const linksAfter = await engine.executeRaw<{ c: number }>(
+      `SELECT COUNT(*)::int AS c FROM links WHERE from_page_id = $1 OR to_page_id = $1`,
+      [p1],
+    );
+    expect(Number(linksAfter[0].c)).toBe(0);
+
+    // p2 itself untouched.
+    const p2Rows = await engine.executeRaw<{ c: number }>(
+      `SELECT COUNT(*)::int AS c FROM pages WHERE id = $1`,
+      [p2],
+    );
+    expect(Number(p2Rows[0].c)).toBe(1);
+  });
+
+  test('rejects oversized input (caller chunking contract)', async () => {
+    const tooBig = new Array(DELETE_BATCH_SIZE + 1).fill('test/x');
+    let threw = false;
+    try {
+      await engine.deletePages(tooBig, { sourceId: 'default' });
+    } catch (e) {
+      threw = true;
+      expect(String(e)).toContain('DELETE_BATCH_SIZE');
+    }
+    expect(threw).toBe(true);
+  });
+});
+
+describe('engine.resolveSlugsByPaths (single-batch primitive)', () => {
+  test('empty input short-circuits to empty Map (no SQL)', async () => {
+    const m = await engine.resolveSlugsByPaths([], { sourceId: 'default' });
+    expect(m.size).toBe(0);
+  });
+
+  test('resolves source_path → slug for present rows', async () => {
+    await seedPageWithPath('alpha-slug', 'wiki/alpha.md');
+    await seedPageWithPath('beta-slug', 'wiki/beta.md');
+    const m = await engine.resolveSlugsByPaths(
+      ['wiki/alpha.md', 'wiki/beta.md', 'wiki/missing.md'],
+      { sourceId: 'default' },
+    );
+    expect(m.get('wiki/alpha.md')).toBe('alpha-slug');
+    expect(m.get('wiki/beta.md')).toBe('beta-slug');
+    expect(m.get('wiki/missing.md')).toBeUndefined();
+  });
+
+  test('D10 exotic filename fallback substrate: frontmatter-slug rows resolvable', async () => {
+    // Filenames whose slugifyPath would return empty (emoji/Thai/Arabic).
+    // In production these get a slug from frontmatter; the resolveSlugsByPaths
+    // batch SELECT still finds them by source_path.
+    await seedPageWithPath('star-page', '🌟.md');
+    await seedPageWithPath('thai-page', 'ทดสอบ.md');
+    await seedPageWithPath('arabic-page', 'عربي.md');
+    const m = await engine.resolveSlugsByPaths(
+      ['🌟.md', 'ทดสอบ.md', 'عربي.md'],
+      { sourceId: 'default' },
+    );
+    expect(m.get('🌟.md')).toBe('star-page');
+    expect(m.get('ทดสอบ.md')).toBe('thai-page');
+    expect(m.get('عربي.md')).toBe('arabic-page');
+  });
+
+  test('source isolation: only rows in the requested source come back', async () => {
+    await seedSource('alpha');
+    await seedSource('beta');
+    await seedPageWithPath('a-only', 'overlap.md', 'alpha');
+    await seedPageWithPath('b-only', 'overlap.md', 'beta');
+
+    const mAlpha = await engine.resolveSlugsByPaths(['overlap.md'], { sourceId: 'alpha' });
+    expect(mAlpha.get('overlap.md')).toBe('a-only');
+
+    const mBeta = await engine.resolveSlugsByPaths(['overlap.md'], { sourceId: 'beta' });
+    expect(mBeta.get('overlap.md')).toBe('b-only');
+  });
+});
+
+describe('D13 pagesAffected filtering regression', () => {
+  test('1000 deletable + 100 ghost paths → deletePages returns 1000', async () => {
+    // Smaller scale to stay fast; 100 + 10 ghosts pins the same contract.
+    for (let i = 0; i < 100; i++) {
+      await seedPageWithPath(`bulk/${i}`, `bulk/${i}.md`);
+    }
+    const realSlugs = Array.from({ length: 100 }, (_, i) => `bulk/${i}`);
+    const ghostSlugs = Array.from({ length: 10 }, (_, i) => `bulk/ghost-${i}`);
+    const allSlugs = [...realSlugs, ...ghostSlugs];
+
+    const deleted = await engine.deletePages(allSlugs, { sourceId: 'default' });
+
+    // Pre-v0.41.19.0 the caller would have pushed all 110 slugs onto
+    // pagesAffected (no filtering); post-fix only the 100 real deletes
+    // come back from RETURNING.
+    expect(deleted.length).toBe(100);
+    expect(deleted.sort()).toEqual(realSlugs.sort());
+  });
+});

test/sync-rename-batch.test.ts

@@ -0,0 +1,92 @@
+/**
+ * v0.41.19.0 — sync rename loop (pre-batched slug resolution)
+ *
+ * Pins the contract of T4 in the plan: the rename loop in
+ * src/commands/sync.ts:~1280 pre-resolves all `from` slugs via
+ * engine.resolveSlugsByPaths in batches BEFORE iterating per-file. The
+ * per-file updateSlug + importFile calls stay (those are inherently
+ * per-file). The win is dropping the slug-resolve N+1.
+ *
+ * Coverage:
+ *   - resolveSlugsByPaths returns one Map for an N-path input (not N
+ *     individual round-trips). Exercised via engine direct + spy on
+ *     executeRaw count.
+ *   - Frontmatter-fallback rename: exotic `from` paths still resolve to
+ *     the stored slug.
+ *   - Source isolation preserved.
+ */
+
+import { afterAll, beforeAll, beforeEach, describe, expect, test } from 'bun:test';
+import { PGLiteEngine } from '../src/core/pglite-engine.ts';
+import { resetPgliteState } from './helpers/reset-pglite.ts';
+import { DELETE_BATCH_SIZE } from '../src/core/engine-constants.ts';
+
+let engine: PGLiteEngine;
+
+beforeAll(async () => {
+  engine = new PGLiteEngine();
+  await engine.connect({});
+  await engine.initSchema();
+});
+
+afterAll(async () => {
+  await engine.disconnect();
+});
+
+beforeEach(async () => {
+  await resetPgliteState(engine);
+});
+
+async function seedPageWithPath(slug: string, sourcePath: string): Promise<void> {
+  await engine.executeRaw(
+    `INSERT INTO pages (source_id, slug, source_path, type, title, compiled_truth, timeline, frontmatter)
+     VALUES ('default', $1, $2, 'note', $1, 'body', '', '{}'::jsonb)
+     ON CONFLICT (source_id, slug) DO UPDATE SET source_path = EXCLUDED.source_path`,
+    [slug, sourcePath],
+  );
+}
+
+describe('rename loop pre-batched slug resolution', () => {
+  test('500 from-paths resolved in 1 batch (DELETE_BATCH_SIZE-aligned)', async () => {
+    // Seed 500 pages, all with explicit source_paths matching their slugs.
+    const N = 500;
+    for (let i = 0; i < N; i++) {
+      await seedPageWithPath(`rn/page-${i}`, `rn/page-${i}.md`);
+    }
+    const paths = Array.from({ length: N }, (_, i) => `rn/page-${i}.md`);
+
+    // Single batch of 500 — exactly at DELETE_BATCH_SIZE boundary.
+    expect(paths.length).toBe(DELETE_BATCH_SIZE);
+    const m = await engine.resolveSlugsByPaths(paths, { sourceId: 'default' });
+    expect(m.size).toBe(N);
+    for (let i = 0; i < N; i++) {
+      expect(m.get(`rn/page-${i}.md`)).toBe(`rn/page-${i}`);
+    }
+  });
+
+  test('frontmatter-fallback rename: exotic source_paths resolve via the batch SELECT', async () => {
+    await seedPageWithPath('star-renamed-from', '🌟.md');
+    await seedPageWithPath('thai-renamed-from', 'ทดสอบ.md');
+
+    const m = await engine.resolveSlugsByPaths(
+      ['🌟.md', 'ทดสอบ.md'],
+      { sourceId: 'default' },
+    );
+    expect(m.get('🌟.md')).toBe('star-renamed-from');
+    expect(m.get('ทดสอบ.md')).toBe('thai-renamed-from');
+  });
+
+  test('mixed present + missing: partial Map (missing → caller falls back to path-derived)', async () => {
+    await seedPageWithPath('present-1', 'present-1.md');
+    await seedPageWithPath('present-2', 'present-2.md');
+    const m = await engine.resolveSlugsByPaths(
+      ['present-1.md', 'absent.md', 'present-2.md', 'absent2.md'],
+      { sourceId: 'default' },
+    );
+    expect(m.size).toBe(2);
+    expect(m.get('present-1.md')).toBe('present-1');
+    expect(m.get('present-2.md')).toBe('present-2');
+    expect(m.get('absent.md')).toBeUndefined();
+    expect(m.get('absent2.md')).toBeUndefined();
+  });
+});