v0.42.51.0 fix(sync): contention-free clock + checkpoint integrity + honest sync freshness (#2255)

* fix(sync): contention-free page-generation clock — sequence swap The page-generation clock backed the query-cache Layer-1 bookmark via a FOR EACH STATEMENT trigger running `UPDATE page_generation_clock SET value=value+1 WHERE id=1`. That took a transaction-length RowExclusiveLock on one tuple, so every concurrent page writer serialized on the prior writer's COMMIT — sync ran at ~0.8 cores regardless of worker count. Swap to a SEQUENCE bumped by nextval() (a microsecond LWLock, never a row lock). The clock's only contract is monotonic advancement on any page INSERT/UPDATE/DELETE; last_value is non-transactional, so rolled-back or concurrent-uncommitted writers only OVER-invalidate the cache (lose a hit), never serve stale. - migration v118: CREATE SEQUENCE + load-bearing 2-arg setval (is_called= true, floor 1, seeded >= old clock and MAX(generation)) + repoint the trigger function body + DELETE query_cache so no old-clock bookmark survives the swap. v107 left immutable. - query-cache-gate.ts: 3 readers -> SELECT last_value FROM page_generation_clock_seq. - schema.sql + pglite-schema.ts (+ regenerated schema-embedded.ts) ship the sequence on fresh install; table + trigger names retained. - tests: clockValue reads last_value; mechanism proof (trigger fn uses nextval not the row UPDATE); rollback-advances-clock safety pin; real PGLite sequence round-trip (is_called gotcha); shape test requires _seq. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sync): op_checkpoints array-shape guard — CHECK + repair + defensive loader completed_keys is JSONB and the checkpoint loader runs jsonb_array_elements_text over it. A non-array (scalar) value makes that throw "cannot extract elements from a scalar", which takes down the whole UNION load — including the valid op_checkpoint_paths child rows — and loses all checkpoint progress for that key. No current writer produces a scalar, but an older binary / external script / future bug could. Make the corruption class structurally impossible and self-healing: - migration v119: LOCK TABLE (so an out-of-band scalar can't land between repair and constrain; no-op on single-connection PGLite), repair any pre-existing scalar to '[]' (op_checkpoint_paths child rows are the append-only source of truth, so the reset loses nothing), then add the named CHECK (jsonb_typeof(completed_keys) = 'array') via a pg_constraint IF NOT EXISTS guard. A DB-enforced always-on guard — the correct pattern vs a migration verify-hook, which never runs on already-stamped brains. - schema.sql + pglite-schema.ts (+ regenerated schema-embedded.ts) ship the same NAMED inline CHECK so fresh installs match migrated brains and v119 skips the duplicate. - op-checkpoint.ts loader: gate the legacy arm on jsonb_typeof = 'array' so a scalar parent is skipped (children still load) instead of throwing the whole union, and log a specific corruption warning when one is seen. - tests: CHECK rejects a scalar (exactly one constraint, no blob+migration dupe); loader survives a scalar parent and returns the children; v119 repair converts a scalar to '[]'. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(doctor): report actively-running sync via live lock, not stale freshness A slow source that makes partial progress every cycle but never fully completes used to read as permanently "stale" / "never synced" because last_sync_at only advances on a full successful sync. The naive fix (treat recent checkpoint banking as "in progress") is unsafe: a blocked sync banks the good files then writes no anchor, so banking can't tell in-progress from wedged. Use the only honest signal: a LIVE, non-expired per-source sync lock (inspectLock + syncLockId against gbrain_cycle_locks). Every non-skipLock sync holds it and refreshes it; a blocked/failed sync's process has exited (no lock row) and a wedged holder stops refreshing (TTL lapses), so either correctly falls through to the stale path and is NEVER masked. An actively-syncing source (including a never-synced source doing its first sync) counts as synced_recently, preserving the pinned 3-bucket invariant. The lock lookup reuses doctor's existing dynamic db-lock import and swallows any throw (stub engine, pre-lock-table brain) to false, so it can only ADD an in-progress verdict, never suppress a real stale one. Tests (real PGLiteEngine + real lock rows): stale+no-lock -> fail; stale+live-lock -> ok; never-synced+live-lock -> ok; never-synced+no-lock -> fail; expired-TTL lock -> fail (wedged not masked); blocked source with banked checkpoint rows but no lock -> still fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sync): honest --force-break-lock diagnostic when no lock is held --force-break-lock used to emit the same terse "Lock ... is not held (nothing to break)" line and exit 0 even when a sync was genuinely wedged, sending the operator down a dead end — the wedge was not a held lock. Keep rc=0 (breaking a non-existent lock is idempotently successful; flipping the exit code would break automation), but under --force say plainly that nothing was broken and point at the real next step (gbrain sync / gbrain doctor) plus a `wedge_hint` field in --json output. The non-force path is byte-for-byte unchanged. runBreakLock is exported for the test. Tests: force+no-lock -> wedge_hint JSON + human hint, rc 0; non-force+no-lock -> unchanged terse line, no hint. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(doctor): surface the in-progress sync holder in the freshness message Plan-completion follow-up to the BUG 4 live-lock signal: when a source is actively syncing, name the holder (pid + host) in the check message instead of silently folding it into synced_recently. The note is appended only when something is in progress, so steady-state messages stay byte-for-byte unchanged (the pinned exact-message + 3-bucket-invariant tests still pass). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sync): pre-landing review fixes — monotonic clock seed, scoped CHECK guard Adversarial (codex) review of the implementation diff caught three: - P1 (correctness): the fresh-schema setval was not monotonic. initSchema replays the schema blob, and the unconditional setval(MAX(generation)) could move page_generation_clock_seq.last_value BACKWARD on an already-upgraded brain, letting a stored query_cache bookmark serve stale rows. Seed via GREATEST over the sequence's OWN last_value (+ old table value + MAX(generation)) in all 3 fresh schemas and migration v118, so a replay is idempotent — mirrors the old table's ON CONFLICT DO NOTHING. Pinned by a new monotonic regression test. - P2: v119's CHECK-exists guard keyed on conname only (not globally unique). Scope it to conrelid = 'op_checkpoints'::regclass. - P3: in-progress note ran into the prior sentence in fail/warn doctor messages; separate it with '. '. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: make Anthropic/ZE no-key tests hermetic against a dev config key These "no key" tests cleared only ANTHROPIC_API_KEY / ZEROENTROPY_API_KEY from the env, but hasAnthropicKey() and checkZeEmbeddingHealth() also read the key from ~/.gbrain/config.json. On a dev machine whose real config holds a key, the no-key assertions flipped and the tests failed locally (they passed only in key-less CI). Add a shared with-env emptyHome() helper and point GBRAIN_HOME at an empty dir in every no-key path so loadConfig finds nothing — matching the already-hermetic anthropic-key / gateway-probe tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.44.1.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(key-files): sync doctor + op-checkpoint entries to v0.44.1.0 truth checkSyncFreshness now reports an actively-running sync via the live per-source lock (names holder pid+host, counts as synced_recently) instead of flagging it stale; loadOpCheckpoint gates the legacy union arm on jsonb_typeof = 'array' so a scalar parent can't take down the whole load, and migration v119's CHECK constraint makes the corruption class structurally impossible. Reference docs describe current behavior only — both entries updated in place, no release-clause appends. Guard + llms freshness test green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: re-version to v0.42.51.0 (natural next-off-master) Maintainer override of the queue allocator's leap to 0.44.1.0 (it jumped past in-flight sibling PR claims at 0.42.50/0.43.0/0.44.0). Take the natural next slot in the 0.42.x line above the immediate sibling claim (0.42.50.0); a merge re-bump resolves any collision if a cathedral PR lands first. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci(e2e): bound + retry the OpenClaw install so a transient npm hang can't burn the Tier 2 budget The Tier 2 (LLM Skills) job failed at 30m16s — the `npm install -g openclaw@2026.4.9` step hung on a transient npm/registry stall (orphan `npm install openclaw` was still running at cancel time) and consumed the entire 30m job budget that v0.42.50.0 (#2254) introduced. The install normally finishes in under a minute (Tier 2 is ~4m end to end on master), so this is flaky-install infra, not a test failure. Wrap the install in `timeout 120` + a 3-attempt retry loop with an 8-minute step backstop: a hung attempt is killed in 2 min and retried instead of eating the whole job. Same bound-the-hang philosophy as #2254's job timeouts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 22:15:33 +00:00 · 2026-06-17 14:02:47 -07:00
parent 70d5f36db6
commit 9bf96db807
23 changed files with 701 additions and 57 deletions
@@ -12,7 +12,7 @@
 */

 import { describe, expect, test, beforeEach } from 'bun:test';
-import { withEnv } from '../helpers/with-env.ts';
+import { withEnv, emptyHome } from '../helpers/with-env.ts';
 import {
  runLlmCall,
  parseLlmJson,
@@ -29,7 +29,7 @@ beforeEach(() => {
 describe('probeLlmAvailability', () => {
  test('returns null when ANTHROPIC_API_KEY is unset', async () => {
    await withEnv(
-      { ANTHROPIC_API_KEY: undefined as unknown as string },
+      { ANTHROPIC_API_KEY: undefined as unknown as string, GBRAIN_HOME: emptyHome() },
      async () => {
        expect(probeLlmAvailability('claude-haiku-4-5')).toBeNull();
        expect(probeLlmAvailability('anthropic:claude-haiku-4-5')).toBeNull();
@@ -94,7 +94,7 @@ describe('runLlmCall — happy path', () => {
 describe('runLlmCall — fail-open paths', () => {
  test('provider unavailable returns null without calling transport', async () => {
    await withEnv(
-      { ANTHROPIC_API_KEY: undefined as unknown as string },
+      { ANTHROPIC_API_KEY: undefined as unknown as string, GBRAIN_HOME: emptyHome() },
      async () => {
        let calls = 0;
        const result = await runLlmCall<unknown>({
@@ -12,7 +12,7 @@
 */

 import { describe, expect, test, beforeEach } from 'bun:test';
-import { withEnv } from '../helpers/with-env.ts';
+import { withEnv, emptyHome } from '../helpers/with-env.ts';
 import { runLlmFallback } from '../../src/core/conversation-parser/llm-fallback.ts';
 import { _resetLlmCacheForTests } from '../../src/core/conversation-parser/llm-base.ts';
 import { makeChatResult } from './helpers.ts';
@@ -71,7 +71,7 @@ describe('runLlmFallback', () => {

  test('provider unavailable: returns null without calling transport', async () => {
    await withEnv(
-      { ANTHROPIC_API_KEY: undefined as unknown as string },
+      { ANTHROPIC_API_KEY: undefined as unknown as string, GBRAIN_HOME: emptyHome() },
      async () => {
        let calls = 0;
        const result = await runLlmFallback({
@@ -11,7 +11,7 @@
 import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
 import { PGLiteEngine } from '../src/core/pglite-engine.ts';
 import { resetPgliteState } from './helpers/reset-pglite.ts';
-import { withEnv } from './helpers/with-env.ts';
+import { withEnv, emptyHome } from './helpers/with-env.ts';
 import {
  checkZeEmbeddingHealth,
  checkEmbeddingWidthConsistency,
@@ -58,8 +58,10 @@ describe('checkZeEmbeddingHealth', () => {
      embedding_dimensions: 1280,
      env: { ...process.env, ZEROENTROPY_API_KEY: undefined as any },
    });
-    // Clear the env var for the no-key path (user's real env may have it set).
-    await withEnv({ ZEROENTROPY_API_KEY: undefined }, async () => {
+    // Clear the env var AND isolate GBRAIN_HOME for the no-key path: the check
+    // reads ZEROENTROPY_API_KEY from env OR the gbrain config file, so a dev
+    // machine whose real ~/.gbrain/config.json holds the key needs both cleared.
+    await withEnv({ ZEROENTROPY_API_KEY: undefined, GBRAIN_HOME: emptyHome() }, async () => {
      const check = await checkZeEmbeddingHealth(engine);
      expect(check.status).toBe('warn');
      expect(check.message).toContain('ZEROENTROPY_API_KEY');
@@ -1330,3 +1330,114 @@ describe('v0.42 (#1699) — quarantined_pages + flagged_pages checks', () => {
    expect(source).toMatch(/name: 'flagged_pages'/);
  });
 });
+
+// ============================================================================
+// BUG 4 (v0.42.x) — doctor reports an actively-running sync via the live lock,
+// not stale freshness. Uses a REAL PGLiteEngine so inspectLock/syncLockId run
+// against actual gbrain_cycle_locks rows (the stub engine can't model a lock).
+// ============================================================================
+describe('BUG 4 — in-progress sync via live lock, not stale freshness', () => {
+  let engine: any;
+  let syncLockId: (s: string) => string;
+
+  beforeAll(async () => {
+    const { PGLiteEngine } = await import('../src/core/pglite-engine.ts');
+    ({ syncLockId } = await import('../src/core/db-lock.ts'));
+    engine = new PGLiteEngine();
+    await engine.connect({});
+    await engine.initSchema();
+  });
+
+  afterAll(async () => {
+    await engine.disconnect();
+  });
+
+  beforeEach(async () => {
+    const { resetPgliteState } = await import('./helpers/reset-pglite.ts');
+    await resetPgliteState(engine);
+    await engine.executeRaw(`DELETE FROM gbrain_cycle_locks`);
+  });
+
+  const staleDate = () => new Date(Date.now() - 5 * 24 * 60 * 60 * 1000); // 5d ago
+
+  async function addSource(id: string, lastSyncAt: Date | null) {
+    await engine.executeRaw(
+      `INSERT INTO sources (id, name, local_path, last_sync_at, config)
+       VALUES ($1, $1, $2, $3, '{"federated":true}'::jsonb)
+       ON CONFLICT (id) DO UPDATE SET last_sync_at = EXCLUDED.last_sync_at`,
+      [id, `/tmp/${id}`, lastSyncAt],
+    );
+  }
+
+  // ttlMinutes > 0 → live lock; <= 0 → already-expired (wedged) holder.
+  async function holdLock(sourceId: string, ttlMinutes: number) {
+    await engine.executeRaw(
+      `INSERT INTO gbrain_cycle_locks (id, holder_pid, holder_host, acquired_at, ttl_expires_at, last_refreshed_at)
+       VALUES ($1, 4242, 'testhost', now(), now() + ($2 || ' minutes')::interval, now())`,
+      [syncLockId(sourceId), String(ttlMinutes)],
+    );
+  }
+
+  test('stale source with NO live lock → fail (blocked/wedged is not masked)', async () => {
+    const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
+    await addSource('wiki', staleDate());
+    const result = await checkSyncFreshness(engine);
+    expect(result.status).toBe('fail');
+    expect(result.message).toContain(`'wiki'`);
+  });
+
+  test('stale source WITH a live (non-expired) lock → ok (sync in progress)', async () => {
+    const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
+    await addSource('wiki', staleDate());
+    await holdLock('wiki', 30);
+    const result = await checkSyncFreshness(engine);
+    expect(result.status).toBe('ok');
+    expect(result.details?.synced_recently_count).toBe(1);
+    expect(result.details?.stale_count).toBe(0);
+    // BUG 4: operator sees the in-progress holder, not silence.
+    expect(result.message).toContain('sync in progress');
+    expect(result.message).toContain('pid 4242');
+  });
+
+  test('never-synced source WITH a live lock → ok (initial sync in progress)', async () => {
+    const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
+    await addSource('wiki', null);
+    await holdLock('wiki', 30);
+    const result = await checkSyncFreshness(engine);
+    expect(result.status).toBe('ok');
+    expect(result.details?.synced_recently_count).toBe(1);
+    expect(result.message).toContain('sync in progress');
+  });
+
+  test('never-synced source with NO lock → fail (unchanged behavior)', async () => {
+    const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
+    await addSource('wiki', null);
+    const result = await checkSyncFreshness(engine);
+    expect(result.status).toBe('fail');
+    expect(result.message).toContain('never been synced');
+  });
+
+  test('expired-TTL lock does NOT mask staleness (wedged-but-not-refreshing holder)', async () => {
+    const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
+    await addSource('wiki', staleDate());
+    await holdLock('wiki', -5); // ttl_expires_at 5 min in the past
+    const result = await checkSyncFreshness(engine);
+    expect(result.status).toBe('fail');
+  });
+
+  test('blocked source with banked checkpoint rows but NO live lock → still fail', async () => {
+    const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
+    await addSource('wiki', staleDate());
+    // A blocked sync banks the good files then exits without an anchor and
+    // without a held lock. Banking must NOT be read as "in progress".
+    await engine.executeRaw(
+      `INSERT INTO op_checkpoints (op, fingerprint, completed_keys, updated_at)
+       VALUES ('sync', 'fp-blocked', '[]'::jsonb, now())`,
+    );
+    await engine.executeRaw(
+      `INSERT INTO op_checkpoint_paths (op, fingerprint, path) VALUES ('sync', 'fp-blocked', 'banked.md')`,
+    );
+    const result = await checkSyncFreshness(engine);
+    expect(result.status).toBe('fail');
+  });
+});
@@ -1,3 +1,7 @@
+import { mkdtempSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+
 /**
 * Run a callback with `process.env` mutations applied, then restore the prior
 * values via try/finally. The canonical pattern for env-touching tests in this
@@ -69,3 +73,17 @@ export async function withEnv<T>(
    }
  }
 }
+
+/**
+ * A fresh empty temp dir for `GBRAIN_HOME`, so `loadConfig()` / `configDir()`
+ * resolve to a directory with no config.json. Pair with a `withEnv` override
+ * (`GBRAIN_HOME: emptyHome()`) on any "no key" assertion: `hasAnthropicKey()`
+ * and the ZE/embedding key probes read BOTH the env var AND the gbrain config
+ * file, so clearing only the env var is NOT hermetic on a dev machine whose
+ * real `~/.gbrain/config.json` holds a key — the assertion flips and the test
+ * fails locally while passing in key-less CI. The dir is tiny and intentionally
+ * leaked (test process is short-lived); the OS reaps tmp.
+ */
+export function emptyHome(): string {
+  return mkdtempSync(join(tmpdir(), 'gbrain-nokey-home-'));
+}
@@ -243,6 +243,82 @@ describe('resumeFilter (pure)', () => {
  });
 });

+describe('BUG 3: completed_keys array-shape guard (v119 CHECK + defensive loader)', () => {
+  const CONSTRAINT = 'op_checkpoints_completed_keys_array';
+
+  test('CHECK rejects a scalar completed_keys write — exactly one constraint (no blob+migration dupe)', async () => {
+    // Fresh PGLite install: the schema blob ships the NAMED inline CHECK and
+    // migration v119's IF NOT EXISTS skips re-adding it. Exactly one constraint.
+    const c = await engine.executeRaw<{ n: number }>(
+      `SELECT count(*)::int AS n FROM pg_constraint WHERE conname = $1`,
+      [CONSTRAINT],
+    );
+    expect(Number(c[0].n)).toBe(1);
+
+    let threw = false;
+    try {
+      await engine.executeRaw(
+        `INSERT INTO op_checkpoints (op, fingerprint, completed_keys, updated_at)
+         VALUES ('embed', 'fp-reject', '"not-an-array"'::jsonb, now())`,
+      );
+    } catch {
+      threw = true;
+    }
+    expect(threw).toBe(true);
+  });
+
+  test('loader survives a scalar parent: returns child rows (not []), does not throw', async () => {
+    const key = { op: 'sync', fingerprint: 'fp-scalar-survive' };
+    // Bypass the CHECK to simulate pre-migration / out-of-band corruption.
+    await engine.executeRaw(`ALTER TABLE op_checkpoints DROP CONSTRAINT ${CONSTRAINT}`);
+    try {
+      await engine.executeRaw(
+        `INSERT INTO op_checkpoints (op, fingerprint, completed_keys, updated_at)
+         VALUES ('sync', 'fp-scalar-survive', '"corrupt-scalar"'::jsonb, now())`,
+      );
+      await engine.executeRaw(
+        `INSERT INTO op_checkpoint_paths (op, fingerprint, path)
+         VALUES ('sync', 'fp-scalar-survive', 'child-a.md')`,
+      );
+      // Pre-guard, jsonb_array_elements_text on the scalar threw and the catch
+      // returned [] — losing child-a.md. The typeof guard skips the scalar so
+      // the valid child survives.
+      const loaded = await loadOpCheckpoint(engine, key);
+      expect(loaded).toEqual(['child-a.md']);
+    } finally {
+      await engine.executeRaw(
+        `UPDATE op_checkpoints SET completed_keys = '[]'::jsonb WHERE jsonb_typeof(completed_keys) <> 'array'`,
+      );
+      await engine.executeRaw(
+        `ALTER TABLE op_checkpoints ADD CONSTRAINT ${CONSTRAINT} CHECK (jsonb_typeof(completed_keys) = 'array')`,
+      );
+    }
+  });
+
+  test('v119 repair converts a scalar parent to an empty array', async () => {
+    await engine.executeRaw(`ALTER TABLE op_checkpoints DROP CONSTRAINT ${CONSTRAINT}`);
+    try {
+      await engine.executeRaw(
+        `INSERT INTO op_checkpoints (op, fingerprint, completed_keys, updated_at)
+         VALUES ('embed', 'fp-repair', '"scalar"'::jsonb, now())`,
+      );
+      // Migration v119's repair statement.
+      await engine.executeRaw(
+        `UPDATE op_checkpoints SET completed_keys = '[]'::jsonb, updated_at = now()
+         WHERE jsonb_typeof(completed_keys) <> 'array'`,
+      );
+      const typ = await engine.executeRaw<{ t: string }>(
+        `SELECT jsonb_typeof(completed_keys) AS t FROM op_checkpoints WHERE op = 'embed' AND fingerprint = 'fp-repair'`,
+      );
+      expect(typ[0].t).toBe('array');
+    } finally {
+      await engine.executeRaw(
+        `ALTER TABLE op_checkpoints ADD CONSTRAINT ${CONSTRAINT} CHECK (jsonb_typeof(completed_keys) = 'array')`,
+      );
+    }
+  });
+});
+
 describe('purgeStaleCheckpoints', () => {
  test('no stale rows: returns 0', async () => {
    await recordCompleted(engine, { op: 'embed', fingerprint: 'fresh' }, ['x']);
@@ -44,8 +44,11 @@ beforeEach(async () => {
 });

 async function clockValue(): Promise<number> {
+  // v0.42.x: the Layer-1 bookmark moved from the locked single-row
+  // page_generation_clock table to a contention-free SEQUENCE bumped by
+  // nextval() in the statement trigger. Read last_value.
  const rows = await engine.executeRaw<{ value: number }>(
-    `SELECT value FROM page_generation_clock WHERE id = 1`,
+    `SELECT last_value AS value FROM page_generation_clock_seq`,
  );
  return Number(rows[0]?.value ?? -1);
 }
@@ -69,13 +72,20 @@ describe('page_generation_clock table + statement-level trigger', () => {
    expect(threw).toBe(true);
  });

-  test('seed: clock starts at COALESCE(MAX(pages.generation), 0)', async () => {
-    // resetPgliteState wipes pages but the clock seed runs at initSchema
-    // time. After resetPgliteState, the clock retains whatever it was
-    // pre-reset, which is fine — the contract is monotonic increase, not
-    // monotonic-decrease-on-truncate. (Production resets don't happen.)
+  test('seed: clock sequence starts at >= 1 with is_called=true', async () => {
+    // v0.42.x: the sequence is seeded via setval(GREATEST(1, MAX(generation)))
+    // at initSchema with is_called=true (2-arg setval), so the FIRST write's
+    // nextval strictly exceeds the seed. Without is_called=true a fresh
+    // sequence's first nextval returns the start value and last_value would not
+    // visibly advance — that would let a fresh install serve a stale cache row.
+    // resetPgliteState does NOT reset the sequence (sequences aren't pg_tables),
+    // so last_value only ever increases — monotonic, never decrease-on-truncate.
    const v = await clockValue();
-    expect(v).toBeGreaterThanOrEqual(0);
+    expect(v).toBeGreaterThanOrEqual(1);
+    const meta = await engine.executeRaw<{ is_called: boolean }>(
+      `SELECT is_called FROM page_generation_clock_seq`,
+    );
+    expect(meta[0].is_called).toBe(true);
  });

  test('INSERT bumps clock by exactly 1 (single-row insert via raw SQL)', async () => {
@@ -225,3 +235,81 @@ describe('query-cache integration (D14 + CDX-6 + CDX-7 end-to-end)', () => {
    expect(afterClock).toBe(beforeClock + 1);
  });
 });
+
+describe('v0.42.x sequence-backed clock (BUG 1: contention removal)', () => {
+  test('mechanism: trigger function uses nextval, NOT a locked row UPDATE', async () => {
+    // The contention source was `UPDATE page_generation_clock SET value=value+1
+    // WHERE id=1` (a transaction-length RowExclusiveLock on one tuple). Prove at
+    // the schema level that it is gone and replaced by nextval (a microsecond
+    // LWLock). This is the deterministic, PGLite-runnable contention proof.
+    const rows = await engine.executeRaw<{ src: string }>(
+      `SELECT prosrc AS src FROM pg_proc WHERE proname = 'bump_page_generation_clock_fn'`,
+    );
+    expect(rows.length).toBe(1);
+    expect(rows[0].src).toContain("nextval('page_generation_clock_seq')");
+    expect(rows[0].src).not.toContain('UPDATE page_generation_clock');
+  });
+
+  test('rollback still advances the sequence (over-invalidation is the SAFE direction)', async () => {
+    const before = await clockValue();
+    // Aborted import: the statement trigger fires nextval (sequences are
+    // non-transactional), so last_value advances even though the page never
+    // commits. A cache row stamped before this now fails Layer 1 and
+    // re-validates — a LOST HIT, never a stale serve.
+    try {
+      await engine.transaction(async (tx) => {
+        await tx.executeRaw(
+          `INSERT INTO pages (source_id, slug, type, title, compiled_truth, timeline, frontmatter)
+           VALUES ('default', 'test/rollback-page', 'note', 't', 'body', '', '{}'::jsonb)`,
+        );
+        throw new Error('abort');
+      });
+    } catch {
+      /* expected */
+    }
+    const after = await clockValue();
+    expect(after).toBeGreaterThan(before);
+    // The page itself did NOT persist — rollback worked.
+    const pages = await engine.executeRaw<{ n: number }>(
+      `SELECT COUNT(*)::int AS n FROM pages WHERE slug = 'test/rollback-page' AND source_id = 'default'`,
+    );
+    expect(Number(pages[0].n)).toBe(0);
+  });
+
+  test('PGLite supports sequences: CREATE / nextval / setval / last_value round-trip', async () => {
+    // codex flagged: no existing sequence usage in the repo — prove (not assert)
+    // that PGLite's WASM Postgres supports the constructs migration v118 relies
+    // on, including the is_called gotcha the load-bearing setval guards against.
+    await engine.executeRaw(`CREATE SEQUENCE IF NOT EXISTS test_probe_seq`);
+    // Fresh sequence (is_called=false): first nextval returns the START value 1,
+    // and last_value does NOT visibly advance past it — the exact trap.
+    const n1 = await engine.executeRaw<{ v: number }>(`SELECT nextval('test_probe_seq') AS v`);
+    expect(Number(n1[0].v)).toBe(1);
+    const n2 = await engine.executeRaw<{ v: number }>(`SELECT nextval('test_probe_seq') AS v`);
+    expect(Number(n2[0].v)).toBe(2);
+    // 2-arg setval → last_value=N, is_called=true; the next nextval = N+1.
+    await engine.executeRaw(`SELECT setval('test_probe_seq', 100)`);
+    const lv = await engine.executeRaw<{ v: number }>(`SELECT last_value AS v FROM test_probe_seq`);
+    expect(Number(lv[0].v)).toBe(100);
+    const n3 = await engine.executeRaw<{ v: number }>(`SELECT nextval('test_probe_seq') AS v`);
+    expect(Number(n3[0].v)).toBe(101);
+    await engine.executeRaw(`DROP SEQUENCE test_probe_seq`);
+  });
+
+  test('re-seeding is monotonic: the GREATEST guard never moves last_value backward', async () => {
+    // Regression for the codex P1: initSchema replays the schema blob, whose
+    // setval must NOT reset the clock below its current value — a backward move
+    // would let a stored query_cache bookmark serve stale rows. Push the
+    // sequence high, then run the EXACT monotonic seed the blob + v118 use.
+    await engine.executeRaw(`SELECT setval('page_generation_clock_seq', 999999)`);
+    await engine.executeRaw(
+      `SELECT setval('page_generation_clock_seq', GREATEST(
+         1,
+         COALESCE((SELECT last_value FROM page_generation_clock_seq), 0),
+         COALESCE((SELECT value FROM page_generation_clock WHERE id = 1), 0),
+         COALESCE((SELECT MAX(generation) FROM pages), 0)
+       ))`,
+    );
+    expect(await clockValue()).toBeGreaterThanOrEqual(999999);
+  });
+});
@@ -252,13 +252,16 @@ describe('buildPageGenerationsSnapshot (PGLite-backed)', () => {
 });

 describe('CACHE_GATE_WHERE_CLAUSE (SQL shape regression)', () => {
-  test('v0.41.19.0: Layer 1 reads page_generation_clock (not MAX(generation))', () => {
-    expect(CACHE_GATE_WHERE_CLAUSE).toContain('page_generation_clock');
+  test('v0.42.x: Layer 1 reads page_generation_clock_seq.last_value (not the locked row, not MAX)', () => {
+    expect(CACHE_GATE_WHERE_CLAUSE).toContain('page_generation_clock_seq');
+    expect(CACHE_GATE_WHERE_CLAUSE).toContain('last_value');
    expect(CACHE_GATE_WHERE_CLAUSE).toContain('qc.max_generation_at_store');
    // Negative regression guard: the old MAX(generation) read shape MUST
    // be gone (codex CDX-1/CDX-2: it silently served stale on
    // UPDATE-to-non-max and DELETE).
    expect(CACHE_GATE_WHERE_CLAUSE).not.toContain('MAX(generation) FROM pages');
+    // The locked single-row read (the BUG 1 contention source) MUST be gone.
+    expect(CACHE_GATE_WHERE_CLAUSE).not.toContain('value FROM page_generation_clock WHERE id');
  });

  test('contains Layer 2 per-page snapshot (jsonb_each + LEFT JOIN)', () => {
@@ -246,3 +246,67 @@ describe('R6 regression: schema bootstrap includes last_refreshed_at column', ()
    expect(rows[0].is_nullable).toBe('YES');
  });
 });
+
+// ============================================================================
+// BUG 5 (v0.42.x) — honest --force-break-lock diagnostic when no lock is held.
+// Previously --force-break-lock emitted the same terse "not held (nothing to
+// break)" line and exited 0, sending operators down a dead end when a sync was
+// wedged for a reason other than a held lock.
+// ============================================================================
+describe('BUG 5 — --force-break-lock honest no-lock diagnostic', () => {
+  const LOCK = 'gbrain-sync:wiki';
+
+  test('force + no lock → wedge_hint in JSON, status absent, rc 0', async () => {
+    const { runBreakLock } = await import('../src/commands/sync.ts');
+    const logs: string[] = [];
+    const orig = console.log;
+    console.log = (...a: unknown[]) => { logs.push(a.map(String).join(' ')); };
+    let rc: number;
+    try {
+      rc = await runBreakLock(engine, LOCK, 'wiki', { force: true, json: true });
+    } finally {
+      console.log = orig;
+    }
+    expect(rc).toBe(0);
+    const parsed = JSON.parse(logs[0]);
+    expect(parsed.status).toBe('absent');
+    expect(parsed.lock).toBe(LOCK);
+    expect(typeof parsed.wedge_hint).toBe('string');
+    expect(parsed.wedge_hint).toContain('not a held lock');
+  });
+
+  test('force + no lock → human output carries the wedge hint, not the terse line', async () => {
+    const { runBreakLock } = await import('../src/commands/sync.ts');
+    const logs: string[] = [];
+    const orig = console.log;
+    console.log = (...a: unknown[]) => { logs.push(a.map(String).join(' ')); };
+    let rc: number;
+    try {
+      rc = await runBreakLock(engine, LOCK, 'wiki', { force: true, json: false });
+    } finally {
+      console.log = orig;
+    }
+    expect(rc).toBe(0);
+    const out = logs.join('\n');
+    expect(out).toContain('nothing to break');
+    expect(out).toContain('gbrain doctor');
+    expect(out).not.toBe(`Lock ${LOCK} is not held (nothing to break).`);
+  });
+
+  test('non-force + no lock → unchanged terse line, no wedge_hint', async () => {
+    const { runBreakLock } = await import('../src/commands/sync.ts');
+    const logs: string[] = [];
+    const orig = console.log;
+    console.log = (...a: unknown[]) => { logs.push(a.map(String).join(' ')); };
+    let rc: number;
+    try {
+      rc = await runBreakLock(engine, LOCK, 'wiki', { force: false, json: true });
+    } finally {
+      console.log = orig;
+    }
+    expect(rc).toBe(0);
+    const parsed = JSON.parse(logs[0]);
+    expect(parsed.status).toBe('absent');
+    expect(parsed.wedge_hint).toBeUndefined();
+  });
+});
@@ -18,7 +18,7 @@
 import { describe, test, expect } from 'bun:test';
 import { __thinkAdapter } from '../src/core/think/index.ts';
 import { resetGateway } from '../src/core/ai/gateway.ts';
-import { withEnv } from './helpers/with-env.ts';
+import { withEnv, emptyHome } from './helpers/with-env.ts';

 describe('think gateway adapter — response shape conversion', () => {
  test('chatResultToMessage maps ChatResult.text to Anthropic.Message content[0].text', () => {
@@ -76,7 +76,7 @@ describe('think gateway adapter — model-id normalization', () => {
  });

  test('tryBuildGatewayClient returns null when ANTHROPIC_API_KEY is absent (preserves legacy NO_ANTHROPIC_API_KEY signal)', async () => {
-    await withEnv({ ANTHROPIC_API_KEY: undefined }, async () => {
+    await withEnv({ ANTHROPIC_API_KEY: undefined, GBRAIN_HOME: emptyHome() }, async () => {
      const client = await __thinkAdapter.tryBuildGatewayClient('claude-opus-4-7');
      expect(client).toBeNull();
    });
@@ -86,7 +86,7 @@ describe('think gateway adapter — model-id normalization', () => {
    await withEnv({ ANTHROPIC_API_KEY: 'sk-test-key' }, async () => {
      expect(__thinkAdapter.hasAnthropicKey()).toBe(true);
    });
-    await withEnv({ ANTHROPIC_API_KEY: undefined }, async () => {
+    await withEnv({ ANTHROPIC_API_KEY: undefined, GBRAIN_HOME: emptyHome() }, async () => {
      expect(__thinkAdapter.hasAnthropicKey()).toBe(false);
    });
  });
@@ -115,7 +115,7 @@ describe('think gateway adapter — #1698 slash form + explicit-model fork', ()
  });

  test('explicit anthropic model with no key THROWS (unavailable)', async () => {
-    await withEnv({ ANTHROPIC_API_KEY: undefined }, async () => {
+    await withEnv({ ANTHROPIC_API_KEY: undefined, GBRAIN_HOME: emptyHome() }, async () => {
      await expect(
        __thinkAdapter.tryBuildGatewayClient('anthropic:claude-sonnet-4-6', { explicitModel: true }),
      ).rejects.toThrow(/not usable.*unavailable/);
@@ -164,7 +164,7 @@ describe('think gateway adapter — #1698 slash form + explicit-model fork', ()
  // 'no LLM available' stub). A future refactor that turns this into a graceful path fails here.
  test('D1 backstop: explicit non-anthropic model, no key → BUILDS then create() THROWS (never a stub)', async () => {
    await withEnv(
-      { ANTHROPIC_API_KEY: undefined, DEEPSEEK_API_KEY: undefined, OPENAI_API_KEY: undefined },
+      { ANTHROPIC_API_KEY: undefined, DEEPSEEK_API_KEY: undefined, OPENAI_API_KEY: undefined, GBRAIN_HOME: emptyHome() },
      async () => {
        resetGateway();  // unconfigured → gateway.chat() throws AIConfigError at create()
        // deepseek:deepseek-chat passes validateModelId (real recipe + chat touchpoint) — the