From 2326cd4a2c5e91f8b9e3035cc70a4eae16018520 Mon Sep 17 00:00:00 2001
From: Garry Tan <garrytan@gmail.com>
Date: Tue, 2 Jun 2026 22:42:42 -0700
Subject: [PATCH] docs(claude): annotate process-watchdog + #1768/#1633 fixes

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 CLAUDE.md     | 5 +++--
 llms-full.txt | 5 +++--
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/CLAUDE.md b/CLAUDE.md
index 213ec1227..b8837dad8 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -172,6 +172,7 @@ strict behavior when unset.
 - `src/core/dry-fix.ts` — `gbrain doctor --fix` engine. `autoFixDryViolations(fixes, {dryRun})` rewrites inlined rules to `> **Convention:** see [path](path).` callouts via three shape-aware expanders (bullet / blockquote / paragraph). Five guards: working-tree-dirty (`getWorkingTreeStatus()` returns 3-state `'clean' | 'dirty' | 'not_a_repo'`), no-git-backup, inside-code-fence, already-delegated (40-line proximity, consistent with detector), ambiguous-multi-match, block-is-callout. `execFileSync` array args (no shell — no injection surface). EOF newline preserved. **v0.37.1.0:** safety primitives extracted to `src/core/skill-fix-gates.ts` (back-compat re-exports preserved). New `MISSING_RULE_PATTERNS` INSERT pattern type lives alongside the existing REPLACE patterns — same auto-fix entry point, same git-safety gates, but instead of rewriting an existing block, INSERT patterns place a canonical callout at a target offset (today: `after-h1-paragraph` only; designed to extend). The first INSERT pattern is `brain_first`, which auto-inserts `> **Convention:** see [conventions/brain-first.md](../conventions/brain-first.md) for the lookup chain (search → query → get_page → external).` on any flagged SKILL.md whose analyzer verdict is `missing_brain_first`. Idempotent — re-runs detect the existing callout and skip.
 - `src/core/backoff.ts` — Adaptive load-aware throttling: CPU/memory checks, exponential backoff, active hours multiplier
 - `src/core/retry.ts` (v0.41.19.0, extended v0.41.27.0) — canonical retry primitive for transient connection errors. Exports `withRetry<T>(fn, opts)` execution wrapper + `BULK_RETRY_OPTS` constant (`{maxRetries:3, delayMs:1000, delayMaxMs:10000, jitter:'decorrelated'}`) tuned for Supabase Supavisor's 5-10s circuit-breaker recovery window + `BATCH_AUDIT_SITES` typed const (closed enum of every audit-emission site) + `resolveBulkRetryOpts(env)` (reads `GBRAIN_BULK_MAX_RETRIES` / `GBRAIN_BULK_RETRY_BASE_MS` / `GBRAIN_BULK_RETRY_MAX_MS` with `>=0` validation, throws on bad input with paste-ready hint) + `abortableSleep(ms, signal?)` (AbortSignal-aware setTimeout) + `RetryAbortError` (tagged error for clean shutdown) + `computeNextDelay()` (pure-fn for 3 jitter modes: `'none'`, `'full'`, `'decorrelated'`). The execution wrapper is consumed by `postgres-engine.ts` + `pglite-engine.ts` batch primitives (`addLinksBatch` / `addTimelineEntriesBatch` / `upsertChunks`) so every caller — current AND future — inherits retry as part of the data-primitive's contract. CI guard `scripts/check-no-double-retry.sh` fails the build on `withRetry(...engine.batch...)` patterns (prevents 3×3=9 retry amplification). CI guard `scripts/check-batch-audit-site.sh` validates every string-literal `auditSite: '...'` against the closed `BATCH_AUDIT_SITES` enum. **Closes the v0.41.17 production incident** where ~3,000 rows were silently lost per dream cycle on a 16K-page brain because the prior single-500ms-retry shape couldn't survive Supavisor's 5-10s circuit-breaker recovery. Decorrelated jitter (AWS-style: `uniform(base, prevDelay*3)` capped at `delayMaxMs`) replaces naive `'full'` jitter which allowed near-zero retries that re-hit the still-recovering breaker. Pinned by `test/core/retry.test.ts` (37 cases) + `test/core/retry-stress.slow.test.ts` (5 cases simulating 100 batches × 30% blip rate, asserts zero row loss). **v0.41.27.0 (#1570):** `WithRetryOpts` gains optional `reconnect?: () => Promise<void>` callback awaited in the catch branch AFTER `isRetryableConnError` classification but BEFORE the inter-attempt sleep — lets engine-level callers rebuild a dead pool/singleton between attempts. `PostgresEngine.batchRetry` injects `() => this.reconnect()` so the v0.22.1 race-safe `_reconnecting` guard kicks in transparently. Fail-loud posture (per codex outside-voice finding 3): a reconnect throw PROPAGATES as the new error, replacing the symptomatic "No database connection" so operators see the real cause (auth failure, EHOSTUNREACH). `onRetry` callbacks now also awaited (back-compat-safe: existing sync arrows work identically; async callbacks now correctly delay the sleep). Pinned by `test/core/retry-reconnect.test.ts` (5 cases: ordering, back-compat, fail-loud propagation, signal-abort short-circuit, awaited onRetry timing) + `test/e2e/db-singleton-shared-recovery.test.ts` (3 DB-gated cases pinning shared-singleton survival via retry reconnect + diagnostic audit fires on disconnect + instance-pool disconnect doesn't touch module singleton).
+- `src/core/process-watchdog.ts` (v0.42.13.0, #1633) — out-of-band hard-deadline killer for `gbrain sync`. A spinning sync (synchronous catastrophic-regex / ReDoS in pack link-inference) STARVES the main event loop, so the existing SIGTERM handler (`process-cleanup.ts`), `--timeout` `setTimeout`, and abort-flag checks can't fire — the process becomes unkillable-by-SIGTERM and, under cron, orphans pile up for 24h+ (the reported incident). `installProcessWatchdog({deadlineMs, graceMs?, label?, heartbeatMs?, onWarn?}): WatchdogHandle` spawns a Bun `worker_threads` Worker via `new Worker(code, {eval: true, workerData})` — its own OS thread + event loop fires even while main is in an unyielding sync loop. At `deadlineMs` it `process.kill(process.pid, 'SIGTERM')` (clean-shutdown chance if responsive); at `deadlineMs+graceMs` `process.kill(process.pid, 'SIGKILL')` (uncatchable — guaranteed death under starvation). Signaling SELF has NO PID-reuse footgun (current PID never reused while alive — the reason the detached-child-watches-parent design was rejected). `eval: true` bakes the worker body into the `bun build --compile` binary with no separate-file embedding. Empirically validated on Bun 1.3.13 (worker timer + SIGKILL killed a `while(true){}`-starved process; Codex outside-voice + a repo spike both confirmed). `handle.dispose()` (clean-exit `finally`) `worker.terminate()`s it; `unref()`'d so it never keeps the process alive. Pure `watchdogDecision(elapsedMs, deadlineMs, graceMs) → 'wait'|'sigterm'|'sigkill'` extracted for unit tests. Optional `heartbeatMs` emits periodic `[<label>] parent alive Ns, hard-kill in ~Ms` lines (visible in cron logs even under starvation — the diagnosis surface). Fallback: if `new Worker` throws, degrades to an in-process timer with a loud warning that it can't fire under starvation. Reusable beyond sync (autopilot/cycle are follow-up adopters). Pinned by `test/process-watchdog.test.ts` (pure decision matrix + handle contract) + `test/process-watchdog.serial.test.ts` (Bun-pinned spawn integration: a starved harness process IS killed ~deadline+grace, a no-watchdog control does NOT self-exit, clean dispose never kills). Wired into `src/cli.ts` sync dispatch BEFORE `connectEngine` (so a connect-phase hang is bounded too — `timeout-layer-vs-connectengine` learning); deadline resolved by `resolveSyncHardDeadline` in `sync.ts` (precedence: `--no-hard-deadline` > `--hard-deadline` > `--timeout`(non-`--all`) > `GBRAIN_SYNC_MAX_RUNTIME_SECONDS` env > non-TTY default 3600s > none).
 - `src/core/audit/batch-retry-audit.ts` (v0.41.19.0, extended v0.41.26.1) — JSONL audit primitive for batch-retry events. Built on `audit-writer.ts` cathedral. Schema: `{ts, site, batch_size, attempt, outcome: 'success' | 'exhausted', delay_ms, error_message_summary, error_code?}`. Privacy posture: NEVER logs slugs / page IDs / content (mirrors `shell-audit.ts` from v0.20+). `logBatchRetry` fires per successful retry recovery; `logBatchExhausted` fires when retries exhaust and rows are lost. `readRecentBatchRetryEvents(hours=24)` returns `{events, corrupted_lines, files_scanned, files_unreadable}` — corruption + permission errors surface to doctor, not silently swallowed (codex H-9). `pruneOldBatchRetryAuditFiles(daysToKeep=30)` deletes old files (codex H-8 — implements the v0.41 plan's "30-day pruning convention" for real). Called from `gbrain dream --phase purge`. File: `~/.gbrain/audit/batch-retry-YYYY-Www.jsonl` (honors `GBRAIN_AUDIT_DIR`). **v0.41.26.1 privacy backfill:** `summarizeError` now routes error messages through the shared `redactConnectionInfo` helper from `src/core/audit/redact-connection-info.ts` BEFORE truncation, so DSNs / hostnames / credentials / IPv4 octets can't leak from a Postgres connection-failure error into operator-shared JSONL dumps. Same risk class as lock-renewal-audit; closed in the same wave. Pinned by `test/audit/batch-retry-audit.test.ts` (12 cases) + new `test/audit/batch-retry-redaction.test.ts` (3 privacy regressions).
 - `src/core/audit/lock-renewal-audit.ts` (v0.41.26.1) — JSONL audit primitive for per-job lock-renewal faults. Sibling of `batch-retry-audit.ts`; built on `audit-writer.ts`. Four outcomes: `failure` (single renewLock throw, counter incremented), `success_after_failure` (recovery; emits the recovery count), `gave_up` (time-based deadline exceeded; abort fired), `executeJob_rejected` (the SECOND unhandledRejection vector from D7 — the stored `executeJob(...).finally(...)` promise itself rejected, e.g. failJob threw during the same DB outage). Schema: `{ts, job_id, job_name, attempt?, outcome, error_message_summary?, error_code?}`. Privacy: NEVER logs `lock_token` or `job.data`. Error summaries route through `redactConnectionInfo` BEFORE truncation. Defense-in-depth: every audit call inside the lock-renewal tick's catch block is wrapped in its own inner try/catch so a misbehaving audit-writer can't re-introduce the unhandledRejection bug class via a new surface. `readRecentLockRenewalEvents(hours=24)` walks current + previous ISO week with corrupted-line tolerance. `pruneOldLockRenewalAuditFiles(daysToKeep=30)` is ready for future dream-cycle purge wiring (filed as TODO-LR-3). File: `~/.gbrain/audit/lock-renewal-YYYY-Www.jsonl`. Operator UX is `tail -F` until the doctor check lands (TODO-LR-2). Pinned by `test/audit/lock-renewal-audit.test.ts` (11 cases).
 - `src/core/audit/redact-connection-info.ts` (v0.41.26.1) — Shared pure helper. `redactConnectionInfo(text: string): string` strips Postgres connection info before any audit JSONL write: `postgres://`/`postgresql://` URLs, `host=foo`, `user=foo`, `password=foo`, `pwd=foo`, IPv4 octets. Each match becomes `<REDACTED:kind>`. Negative-lookbehind/lookahead `[\w.@-]` on the IPv4 pattern defeats version-string false positives like `v3.1.4.0` or `tree-sitter@0.26.3.1` while still matching real IPs in PG errors (`(192.168.1.42)`). Order-sensitive pattern set: URL forms first so substrings inside URLs don't get double-redacted. Idempotent (running twice produces same output), pure (no I/O), hot-path-safe (regex compiled at module load). Wired into BOTH `lock-renewal-audit.ts` (new) AND `batch-retry-audit.ts` (privacy backfill — same risk class). Risk model documented in module header: Postgres errors during connection failures embed DSNs / credentials / hostnames into error messages; operators routinely paste audit dumps into GitHub issues / Slack to debug; the audit channel must be safe to share by construction. Known limitations: bare-quoted hostnames (`at "db.example.com"`) and bare-quoted usernames (`for user "postgres.foo"`) are NOT caught — the IP in those PG error shapes is the highest-value leak and IS caught. Quoted-string patterns filed as TODO-LR-5. Pinned by `test/audit/redact-connection-info.test.ts` (15 cases covering all 5 patterns + real Supabase fixture + ENOTFOUND fixture + version-string false-positive defense).
@@ -193,7 +194,7 @@ strict behavior when unset.
 - `src/core/conversation-parser/` (v0.41.16.0, v0.41.24.0, v0.41.29.0) — 14-pattern built-in chat-format registry + opt-in LLM polish/fallback. Replaces the single inline regex that PR #1461 was trying to extend with one more shape. **v0.41.29.0** added the 14th pattern `bold-name-no-time` (regex `/^\*\*(?!\[)(.+?):\*\*\s*(.*)$/`, declared at index 3 after `bold-paren-time`): parses `**Speaker:** text` with NO per-line timestamp — the shape Circleback / Granola / Zoom emit. Anchors every message at `T00:00:00Z` of the frontmatter date (no fabricated wall-clock; line order preserves sequence), same no-time convention as `irc-classic`. The non-shadow guarantee is the colon-INSIDE-bold regex (NOT declaration order — `parse.ts` scores every candidate independently, order is only the tie-break); the `(?!\[)` lookahead rejects telegram-bracket `**[18:37] Name:**` so disabling telegram-bracket yields an honest `no_match` instead of `speaker="[18:37] Name"`. Because `**Label:** text` is a common prose idiom, the pattern sets the new optional `PatternEntry.score_full_body: true`: `parse.ts` recomputes the winner's acceptance score over the FULL body (not just the 10-line head window) before the `SCORING_MIN_ACCEPTANCE` floor, so a notes page with bold labels clustered in its first 10 lines (head score 0.3, which is NOT `< SCORING_HEAD_TRIGGER_THRESHOLD` so the head fallback never fires) drops to its true low density and stays `no_match` instead of mis-parsing as a conversation. The same wave scrubbed the pre-existing real names in `bold-paren-time`'s `test_positive`. Pinned by the `bold-name-no-time` + clustered-head `no_match` cases in `test/conversation-parser/parse.test.ts` + `bold-name-no-time.jsonl` / `adversarial.jsonl` fixtures. **v0.41.24.0 (closes #1533)** added the 13th pattern `bold-paren-time` (`**Speaker** (HH:MM): text` and `**Speaker** (HH:MM:SS): text` — the time-only shape an OpenClaw meeting-ingestion pipeline produces from Circleback transcripts; date_source: frontmatter) AND tightened the parser's fallback gates in `parse.ts`: `SCORING_HEAD_TRIGGER_THRESHOLD = 0.3` triggers a full-body re-score when the head pass scores below that (closes the bug class where a stray head match at 0.1 — e.g. blockquote that accidentally matches an unrelated pattern — suppressed the fallback entirely); `SCORING_MIN_ACCEPTANCE = 0.05` blocks essay false-positives (a 300-line essay with one stray chat-shape line scores ~0.003, below the floor, stays `no_match` instead of returning a 1-message regex_match). Both gates pinned by 11 new unit cases in `test/conversation-parser/parse.test.ts`. New exported `scorePatternFull(body, entry)` for direct full-body scoring; private `getNonBlankLines(body, headCap?)` + `scoreFromLines(lines, entry)` helpers DRY the quick_reject+regex loop. Empirical against all Circleback meetings in `~/git/brain/meetings/`: 113 of 367 parse correctly post-fix (was 0); 20,167 messages flow through to the fact extractor (was 0). The other 254 are notes-only meetings whose transcripts live in separate files. Plan + Codex absorption at `~/.claude/plans/system-instruction-you-are-working-starry-frost.md`. Modules: `types.ts` (PatternEntry + ParseResult + DateContext + CaptureMap + TimezonePolicy), `builtins.ts` (13 hand-vetted patterns sourced from public format docs — iMessage/Slack, Telegram bracket + text-export, bold-paren-time, Discord classic + export, WhatsApp ISO + US, Signal, Matrix/Element, IRC classic + weechat, Teams export; module-load validation runs every `test_positive[]` + `test_negative[]` sample at startup, so a typo in any built-in regex makes gbrain refuse to start; `DEFAULT_SPEAKER_CLEAN` exported as the PR #1461 helper promoted to a module-level default), `parse.ts` (orchestrator with D18 pattern-priority scoring across the first 10 lines + D8 date derivation chain `explicit > frontmatter.date > effective_date > '1970-01-01'` + D5 multi-line continuation + D19 timezone warning), `llm-base.ts` (shared `runLlmCall<T>` with content-hash cache in-process + DB-persistent via migration v97 + 4-strategy JSON repair + Anthropic-key probe; both polish and fallback are thin wrappers per D6 DRY), `llm-polish.ts` (per D15 opt-IN; headroom guard skips when tracker within $0.10 of cap; pure `applyPolish` for merge/drop/edit ops), `llm-fallback.ts` (per D15 opt-IN; per D17 NO regex inference + NO persistence — silent corruption avoided), `eval.ts` (`scoreFixture` + `aggregateScores` + `parseFixtureJsonl` for the fixture-corpus CI gate), `nightly-probe.ts` (DI-stubbed; mode-gated default per D10 — tokenmax=ON, conservative/balanced opt-in; adversarial false-positive detection). CLI surfaces at `src/commands/eval-conversation-parser.ts` (`gbrain eval conversation-parser <fixture.jsonl>` exit 0/1/2 — wired into `bun run verify` via `check:conversation-parser`) and `src/commands/conversation-parser.ts` (`scan <slug>` for debug, `list-builtins` for operator discoverability, `validate <file>` emits "deferred to v0.42+" notice). Doctor checks: `conversation_format_coverage` (per-pattern hit count + unmatched %), `progressive_batch_audit_health` (abort-verdict counts from last 7d), `conversation_parser_probe_health` (opt-in nightly probe state). Closes PR #1461 — contributor's `BRACKET_TIME_RX` + `cleanSpeaker` survive verbatim as the `telegram-bracket` built-in + `DEFAULT_SPEAKER_CLEAN` export, Co-Authored-By preserved. Plan + 23 decisions + codex outside-voice absorption at `~/.claude/plans/system-instruction-you-are-working-cuddly-hollerith.md`. Pinned by 74 unit cases across `test/conversation-parser/{parse,llm-base,llm-fallback,llm-polish,nightly-probe}.test.ts` + the 27-case PR #1461 baseline at `test/extract-conversation-facts.test.ts` (back-compat invariant). Migration v97 (`conversation_parser_llm_cache_table`). Fixtures at `test/fixtures/conversation-formats/{imessage,telegram-bracket,whatsapp-iso,whatsapp-us,signal-export,irc-classic,irc-weechat,matrix-element,teams-export,all,adversarial}.jsonl` with `scripts/check-fixture-privacy.sh` banning real-name leaks.
 - `src/core/progressive-batch/` (v0.41.16.0) — shared ramp-up + cost-cap + verification primitive extracted after rule-of-three was comfortably past (12+ ad-hoc cost-prompt patterns scattered across reindex, reindex-multimodal, book-mirror, brainstorm, eval-suspected-contradictions, post-upgrade-reembed, eval-contradictions/cost-prompt). Wintermute-inspired ramp shape (trial 10 → ramp 100 → ramp 500 → full with verification at each stage), productionized as a runtime primitive with verifier+policy injection (callers describe HOW TO MEASURE SUCCESS, not WHEN TO WAIT FOR CTRL-C). Modules: `types.ts` (Stage, StageVerdict, AbortReason, discriminated `Verifier` union per D20: `OutputCountVerifier | IdempotentMutationVerifier | NoopVerifier`, Policy, StageReport), `orchestrator.ts` (`runProgressiveBatch(items, verifier, policy, runner)` — reads `getCurrentBudgetTracker()` ahead of `Policy.maxCostUsd` per D3 fail-closed; null both ways triggers `abort_cost_cap reason='no_budget_safety_net'`), `audit.ts` (ISO-week JSONL at `~/.gbrain/audit/progressive-batch-YYYY-Www.jsonl` via the shared `audit-writer` primitive), `stage-report.ts` (ASCII stage-report formatter for the default `Policy.onStageReport` handler). Env knobs: `GBRAIN_PROGRESSIVE_BATCH_DISABLED=1` (skip ramp + stderr warn), `GBRAIN_PROGRESSIVE_BATCH_AUTO=1` (skip Ctrl-C grace), `GBRAIN_PROGRESSIVE_BATCH_STAGES=10,100,500` (override). v0.41.16.0 ships with ONE proven consumer (the parser cathedral via extract-conversation-facts retrofit); the 9-site retrofit of `reindex.ts`, `reindex-multimodal.ts`, `reindex-code.ts`, `post-upgrade-reembed.ts`, `book-mirror.ts`, `brainstorm/orchestrator.ts`, `eval-suspected-contradictions.ts`, `eval-contradictions/cost-prompt.ts`, `contextual-reindex-per-chunk.ts` lands as the v0.41.14.0+ follow-up wave per D2's bisectability requirement (each retrofit gets its own commit; behavior parity tested before/after migration). Per D21: sites that previously "jumped straight to full" keep doing so by default; ramp is opt-in per-site via `Policy.interactiveAbortMs > 0`. Pinned by 35 unit cases in `test/progressive-batch/orchestrator.test.ts` covering every verdict path.
 - `src/commands/extract-conversation-facts.ts` + `src/core/cycle/conversation-facts-backfill.ts` (v0.41.11.0) — bulk fact extraction for long-form conversation pages. The recall-miss class on long iMessage/Slack imports (60K+ msg history; a chunk that reads only "Locker 93 code 9494" has no topical anchor because "cabin" was established 50K messages earlier) gets fixed by walking conversation/meeting/slack/email pages, splitting into time-windowed segments (30-min gap or 30-msg cap), prepending a topical/temporal header ("Conversation between A and B from <iso1> to <iso2>"), and running through the existing `extractFactsFromTurn()` so the resulting anchor-rich facts surface in `gbrain search` (which already blends facts). Architecture decisions absorbed from 2 Codex outside-voice rounds + 2 eng review passes: **strict per-source core** (`runExtractConversationFactsCore({sourceId, ...})` always takes one sourceId; CLI + cycle phase each do their own multi-source iteration because PHASE_SCOPE='source' is taxonomy-only per cycle.ts:131); **two-phase enumeration** (paginated `listPages({type, sourceId, limit:10})` so worst case is 10×25MB=250MB resident vs PR #1406's unbounded; per-page body cap MAX_PAGE_BODY_BYTES=25MB with `pages_skipped_too_large` counter surfaced in doctor); **page-global row_num accumulator** (facts unique index is `(source_id, source_markdown_slug, row_num)` per migration v51 — per-segment row_num would collide on segment 2); **page-level TERMINAL audit row** written to facts table after all segments commit (source='cli:extract-conversation-facts:terminal'; doctor's NOT EXISTS query matches the terminal row, NOT any-fact-for-slug, so partial-extraction pages stay in backlog); **optional `opts.budgetTracker?`** (when present, core uses it as-is — nested `withBudgetTracker` REPLACES per gateway.ts:2011; when absent, core auto-wraps with `BudgetTracker({maxCostUsd})`); **body read covers compiled_truth + timeline** (iMessage importers put history in timeline half; PR's `compiled_truth ?? ''` would silently drop it); **honors `facts.extraction_enabled` kill-switch** with `--override-disabled` escape; **--types LIST allowlist** (`conversation,meeting,slack,email`) with CLI default reading `cycle.conversation_facts_backfill.types` config as single source of truth; **fingerprint on sourceId only** so widening types config doesn't invalidate completed-page state; **string-encoded op-checkpoint** entries `"<sourceId>|<slug>|<endIso>"` for resume only (op_checkpoints stores string[] and is 7-day GC'd — durable audit is the facts table itself via terminal row); **`--background` via maybeBackground** (Minion handler `extract-conversation-facts` re-creates BudgetTracker from `data.max_cost_usd`; on `BudgetExhausted` mid-job, catches + persists + marks `completed` with `result.budget_exhausted=true` — NOT a failure). The companion cycle phase `conversation_facts_backfill` (default OFF; `gbrain config set cycle.conversation_facts_backfill.enabled true` opts in) iterates `listSources(engine)` directly, creates ONE brain-wide tracker per tick + wraps the loop in `withBudgetTracker` + passes the tracker into every per-source invocation via `opts.budgetTracker` so the core doesn't nest-replace. Two-layer cost AND walltime protection: per-source caps (`max_cost_usd=$1`, `max_walltime_min=20`) AND brain-wide caps (`max_total_cost_usd=$5`, `max_total_walltime_min=30`). Pinned by 27 unit cases in `test/extract-conversation-facts.test.ts` (parse/segment/render/checkpoint/fingerprint/terminal-audit/row_num/F2-kill-switch). Schema migration v94 in `src/core/migrate.ts` adds a partial index `idx_facts_extract_conversation_session ON facts(source_id, source_session) WHERE source LIKE 'cli:extract-conversation-facts%'` (v14 precedent: `transaction:false` + invalid-index pre-drop on Postgres; plain CREATE INDEX on PGLite) so the doctor backlog query stays fast on million-fact brains. `src/commands/doctor.ts:computeConversationFactsBacklogCheck` is 3-state (SKIPPED when `cycle.conversation_facts_backfill.enabled=false` — no opt-out noise debt per Eng-v2 C9; OK when caught up; WARN when >10 pages lack the terminal audit row, with paste-ready remediation step composable via `gbrain doctor --remediate`). `src/commands/sources.ts:runAudit` extended with `facts_backfill_estimate: {pages, est_segments, est_cost_usd, types}` so `gbrain sources audit <id>` previews cost before any run. Schema-pack (v0.41.11.0): `gbrain-base.yaml` promotes `conversation` (temporal, extractable) + `atom` (annotation, NOT extractable — atoms ARE the extracted form) into the base seed; `concept.extractable` flips true semantically (cosmetic on backstop path because backstop uses hardcoded `ELIGIBLE_TYPES` in `src/core/facts/eligibility.ts:51`, not pack extractable — the original concept-grandfather migration was solving a phantom, dropped per Codex T3). `ALL_PAGE_TYPES` in `src/core/types.ts` extended with the two new types. Plan + all decisions persisted at `~/.claude/plans/system-instruction-you-are-working-linear-unicorn.md` and the CEO plan at `~/.gstack/projects/garrytan-gbrain/ceo-plans/2026-05-25-conversation-retrieval-upgrade.md`.
-- `src/commands/extract.ts` — `gbrain extract links|timeline|all [--source fs|db] [--source-id <id>]`: batch link/timeline extraction. fs walks markdown files, db walks pages from the engine (mutation-immune snapshot iteration; use this for live brains with no local checkout). As of v0.12.1 there is no in-memory dedup pre-load — candidates are buffered 100 at a time and flushed via `addLinksBatch` / `addTimelineEntriesBatch`; `ON CONFLICT DO NOTHING` enforces uniqueness at the DB layer, and the `created` counter returns real rows inserted (truthful on re-runs). v0.22.1 (#417): `ExtractOpts.slugs?: string[]` enables incremental extract — when set, `extractForSlugs()` reads ONLY those slugs' files (single combined links+timeline pass) instead of the full directory walk. CLI `gbrain extract` keeps full-walk behavior; the cycle path threads sync's `pagesAffected` through. `walkMarkdownFiles(brainDir)` still runs at line 455 to build `allSlugs` for link resolution — see `TODOS.md` for replacing it with `engine.getAllSlugs()`. **v0.37.7.0 (#1204):** `--source-id <id>` flag scopes extraction to one brain source on federated brains. Resolved via `resolveSourceWithTier()` before any SQL runs; failures surface with a `gbrain sources list` hint. Closes the silent-collapse-to-`default` bug class for extract. **v0.42.7.0 (#1696) — link-extraction freshness watermark + `extract --stale`:** the "imported ≠ curated" wave. New `gbrain extract --stale [--source-id <id>] [--catch-up] [--dry-run] [--json]` branch (`extractStaleFromDB`) — incremental DB-source link+timeline sweep over pages whose `pages.links_extracted_at` watermark is stale. Stale predicate (shared by both engines + the doctor check): `links_extracted_at IS NULL OR links_extracted_at < LINK_EXTRACTOR_VERSION_TS::timestamptz OR updated_at > links_extracted_at` — the `updated_at` arm catches the MCP `put_page` / `sync --no-extract` edited-since-extract path. Three new `BrainEngine` methods (parity in postgres-engine.ts + pglite-engine.ts + bootstrap probes): `countStalePagesForExtraction(opts?)`, `listStalePagesForExtraction({batchSize, afterPageId?, sourceId?, versionTs?})` (returns page CONTENT so the sweep avoids an N+1 `getPage`; `rowToStalePage` in utils.ts maps the row, `StalePageRow` in types.ts), and `markPagesExtractedBatch(refs, defaultExtractedAt)` (3-array unnest `slug[],source_id[],ts[]`; each ref may carry its own `extractedAt`). `STALE_BATCH_SIZE` default 25 (`GBRAIN_EXTRACT_STALE_BATCH`) — small because page bodies are unbounded (the LIMIT is the only fetch-time memory bound); `STALE_TIME_BUDGET_MS` 30min wall-clock (`--catch-up` removes the cap). CDX-4 non-swallowing flush: link/timeline flush throws propagate and abort the batch; stamp LAST so a crash leaves pages unstamped and they re-extract idempotently (`addLinksBatch` ON CONFLICT DO NOTHING + timeline dedup). D4 race fix: `extractStaleFromDB` stamps with each row's READ `updated_at` (not `now()`), so a concurrent edit during the sweep advances `updated_at` past the stamp → page stays stale → re-extracted next run rather than marked fresh-with-old-content. Source-correct stamping at the DB-extract sites via `stampExtracted` (best-effort wrapper, never throws); `extractLinksFromDB` only stamps the combined watermark when `subcommand === 'all'` (both links AND timeline ran — a links-only run must not hide timeline staleness, CDX C3). `LINK_EXTRACTOR_VERSION_TS` lives in `src/core/link-extraction.ts` (bump like `CHUNKER_VERSION` when the extractor shape changes → invalidates all prior stamps). Migration v112 (`pages_links_extracted_at`) adds the nullable `TIMESTAMPTZ` column + composite `(source_id, links_extracted_at)` index (CONCURRENTLY + invalid-remnant pre-drop on Postgres, plain on PGLite), NO backfill so the real backlog surfaces on first `gbrain doctor`. Schema parity in schema.sql + regenerated pglite-schema.ts + schema-embedded.ts + `REQUIRED_BOOTSTRAP_COVERAGE`. `src/commands/doctor.ts:checkLinksExtractionLag` (the `links_extraction_lag` check, also in `doctorReportRemote`) is warn-only by default (>`GBRAIN_EXTRACTION_LAG_WARN_PCT`, default 20%; shared `EXTRACTION_LAG_WARN_PCT_DEFAULT` + `EXTRACTION_LAG_MIN_PAGES=100` + exported `_resolveEnvNumber`), hard-fails only when `GBRAIN_EXTRACTION_LAG_FAIL_PCT` is set; vacuous-skips <100 pages (no `--source`); pre-v112 brains graceful-skip via `isUndefinedColumnError`; strictly a SQL COUNT (safe on the remote/thin-client path). `src/commands/sync.ts` gains `--no-extract` (threaded through single-source + `--all` + `syncOneSource`), stamps `links_extracted_at` for `pagesAffected` at the inline-extract call site (after `extractLinksForSlugs`/`extractTimelineForSlugs`, CDX-6), and `maybeExtractionNudge` prints a one-line stderr nudge after a `synced | first_sync | up_to_date` sync that leaves a backlog (the `first_sync` case is the initial-import scenario, D5; `shouldNudgeAfterSync` pure predicate; `GBRAIN_SYNC_NO_EXTRACT_NUDGE` suppresses; shares the doctor vacuous-skip predicate). `src/core/retry.ts` adds `'extract.stale'` to `BATCH_AUDIT_SITES`; `src/core/doctor-categories.ts` adds `links_extraction_lag` to `BRAIN_CHECK_NAMES`. Pinned by `test/extract-stale.test.ts` (incl. CDX-1 edited-after-stamp regression + crash-contract), `test/sync-inline-extract-stamps.serial.test.ts` (IRON-RULE), `test/sync-nudge-status-gate.test.ts`, `test/doctor-links-extraction-lag.test.ts`, engine-parity (Postgres↔PGLite) for the 3 methods + v112 round-trip. Plan: `~/.claude/plans/system-instruction-you-are-working-squishy-crayon.md`.
+- `src/commands/extract.ts` — `gbrain extract links|timeline|all [--source fs|db] [--source-id <id>]`: batch link/timeline extraction. fs walks markdown files, db walks pages from the engine (mutation-immune snapshot iteration; use this for live brains with no local checkout). As of v0.12.1 there is no in-memory dedup pre-load — candidates are buffered 100 at a time and flushed via `addLinksBatch` / `addTimelineEntriesBatch`; `ON CONFLICT DO NOTHING` enforces uniqueness at the DB layer, and the `created` counter returns real rows inserted (truthful on re-runs). v0.22.1 (#417): `ExtractOpts.slugs?: string[]` enables incremental extract — when set, `extractForSlugs()` reads ONLY those slugs' files (single combined links+timeline pass) instead of the full directory walk. CLI `gbrain extract` keeps full-walk behavior; the cycle path threads sync's `pagesAffected` through. `walkMarkdownFiles(brainDir)` still runs at line 455 to build `allSlugs` for link resolution — see `TODOS.md` for replacing it with `engine.getAllSlugs()`. **v0.37.7.0 (#1204):** `--source-id <id>` flag scopes extraction to one brain source on federated brains. Resolved via `resolveSourceWithTier()` before any SQL runs; failures surface with a `gbrain sources list` hint. Closes the silent-collapse-to-`default` bug class for extract. **v0.42.7.0 (#1696) — link-extraction freshness watermark + `extract --stale`:** the "imported ≠ curated" wave. New `gbrain extract --stale [--source-id <id>] [--catch-up] [--dry-run] [--json]` branch (`extractStaleFromDB`) — incremental DB-source link+timeline sweep over pages whose `pages.links_extracted_at` watermark is stale. Stale predicate (shared by both engines + the doctor check): `links_extracted_at IS NULL OR links_extracted_at < LINK_EXTRACTOR_VERSION_TS::timestamptz OR updated_at > links_extracted_at` — the `updated_at` arm catches the MCP `put_page` / `sync --no-extract` edited-since-extract path. Three new `BrainEngine` methods (parity in postgres-engine.ts + pglite-engine.ts + bootstrap probes): `countStalePagesForExtraction(opts?)`, `listStalePagesForExtraction({batchSize, afterPageId?, sourceId?, versionTs?})` (returns page CONTENT so the sweep avoids an N+1 `getPage`; `rowToStalePage` in utils.ts maps the row, `StalePageRow` in types.ts), and `markPagesExtractedBatch(refs, defaultExtractedAt)` (3-array unnest `slug[],source_id[],ts[]`; each ref may carry its own `extractedAt`). `STALE_BATCH_SIZE` default 25 (`GBRAIN_EXTRACT_STALE_BATCH`) — small because page bodies are unbounded (the LIMIT is the only fetch-time memory bound); `STALE_TIME_BUDGET_MS` 30min wall-clock (`--catch-up` removes the cap). CDX-4 non-swallowing flush: link/timeline flush throws propagate and abort the batch; stamp LAST so a crash leaves pages unstamped and they re-extract idempotently (`addLinksBatch` ON CONFLICT DO NOTHING + timeline dedup). D4 race fix: `extractStaleFromDB` stamps with each row's READ `updated_at` (not `now()`), so a concurrent edit during the sweep advances `updated_at` past the stamp → page stays stale → re-extracted next run rather than marked fresh-with-old-content. Source-correct stamping at the DB-extract sites via `stampExtracted` (best-effort wrapper, never throws); `extractLinksFromDB` only stamps the combined watermark when `subcommand === 'all'` (both links AND timeline ran — a links-only run must not hide timeline staleness, CDX C3). `LINK_EXTRACTOR_VERSION_TS` lives in `src/core/link-extraction.ts` (bump like `CHUNKER_VERSION` when the extractor shape changes → invalidates all prior stamps). Migration v112 (`pages_links_extracted_at`) adds the nullable `TIMESTAMPTZ` column + composite `(source_id, links_extracted_at)` index (CONCURRENTLY + invalid-remnant pre-drop on Postgres, plain on PGLite), NO backfill so the real backlog surfaces on first `gbrain doctor`. Schema parity in schema.sql + regenerated pglite-schema.ts + schema-embedded.ts + `REQUIRED_BOOTSTRAP_COVERAGE`. `src/commands/doctor.ts:checkLinksExtractionLag` (the `links_extraction_lag` check, also in `doctorReportRemote`) is warn-only by default (>`GBRAIN_EXTRACTION_LAG_WARN_PCT`, default 20%; shared `EXTRACTION_LAG_WARN_PCT_DEFAULT` + `EXTRACTION_LAG_MIN_PAGES=100` + exported `_resolveEnvNumber`), hard-fails only when `GBRAIN_EXTRACTION_LAG_FAIL_PCT` is set; vacuous-skips <100 pages (no `--source`); pre-v112 brains graceful-skip via `isUndefinedColumnError`; strictly a SQL COUNT (safe on the remote/thin-client path). `src/commands/sync.ts` gains `--no-extract` (threaded through single-source + `--all` + `syncOneSource`), stamps `links_extracted_at` for `pagesAffected` at the inline-extract call site (after `extractLinksForSlugs`/`extractTimelineForSlugs`, CDX-6), and `maybeExtractionNudge` prints a one-line stderr nudge after a `synced | first_sync | up_to_date` sync that leaves a backlog (the `first_sync` case is the initial-import scenario, D5; `shouldNudgeAfterSync` pure predicate; `GBRAIN_SYNC_NO_EXTRACT_NUDGE` suppresses; shares the doctor vacuous-skip predicate). `src/core/retry.ts` adds `'extract.stale'` to `BATCH_AUDIT_SITES`; `src/core/doctor-categories.ts` adds `links_extraction_lag` to `BRAIN_CHECK_NAMES`. Pinned by `test/extract-stale.test.ts` (incl. CDX-1 edited-after-stamp regression + crash-contract), `test/sync-inline-extract-stamps.serial.test.ts` (IRON-RULE), `test/sync-nudge-status-gate.test.ts`, `test/doctor-links-extraction-lag.test.ts`, engine-parity (Postgres↔PGLite) for the 3 methods + v112 round-trip. Plan: `~/.claude/plans/system-instruction-you-are-working-squishy-crayon.md`. **v0.42.13.0 (#1768) — `links_extraction_lag` never cleared on Postgres:** the D4 race-fix stamped `page.updated_at.toISOString()`, but `StalePageRow.updated_at` is a JS `Date` (millisecond-truncated) while the DB `updated_at` keeps microseconds — so `updated_at > links_extracted_at` stayed permanently true and the lag was stuck at 100% (the remediation the check recommends, `extract --stale`, could never satisfy its own check). NOT a trigger (there's no `BEFORE UPDATE` trigger on `pages` that sets `updated_at`). Fix preserves the stamp contract + D4: both engines' `listStalePagesForExtraction` SELECT now projects a deterministic full-µs UTC string `to_char(updated_at AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS.US"Z"') AS updated_at_iso` (NOT raw `::text`, which is `DateStyle`-fragile — Codex outside-voice catch); `StalePageRow` gains `updated_at_iso: string`; `rowToStalePage` (utils.ts) populates it (ISO-only fallback, never `String(Date)`); `extractStaleFromDB` stamps `page.updated_at_iso` instead of the truncated Date. `markPagesExtractedBatch` SQL is UNCHANGED, so backdated stamps (the version-arm test) still work and the CDX-1 edited-since arm is strengthened to exact equality. Pinned by a new deterministic PGLite regression in `test/extract-stale.test.ts` (inject a µs `updated_at`, run `--stale`, assert lag → 0 and stays 0) + engine-parity.
 - `src/core/extract/receipt-writer.ts` + `src/core/extract/rollup-writer.ts` + `src/commands/extract-status.ts` + `src/commands/extract-explain.ts` + `src/commands/extract-benchmark.ts` + `src/core/schema-pack/scaffold-extractable.ts` (v0.41.23.0) — unified extract operator-surface wave. Every shipped extractor (deterministic `facts.conversation` in `src/commands/extract-conversation-facts.ts` + three LLM-backed cycle phases at `src/core/cycle/{extract-atoms,synthesize-concepts,propose-takes,extract-facts}.ts`) now writes ONE receipt page per run (`writeReceipt`) + UPSERTs a row to `extract_rollup_7d` (`upsertExtractRollup`). Receipt slug per D-EXTRACT-17: `extracts/{date}/{kind}/{source_id}/{run_id_short}/round-{N}.md`; frontmatter stamps BOTH `type: extract_receipt` AND `dream_generated: true` per D-EXTRACT-19 (belt+suspenders against extraction-loop guard drift). `extract_receipt` joins `ALL_PAGE_TYPES` in `src/core/types.ts`; `extracts/` prefix gets a 0.3x source-boost demote in `src/core/search/source-boost.ts` so receipts surface in search but never dominate user content. Migration v104 adds `extract_rollup_7d (kind, source_id, day, cost_usd, halt_count, eval_pass_count, eval_fail_count, round_completed_count, rollup_write_failures, updated_at)` with PK `(kind, source_id, day)` + `idx_extract_rollup_7d_day`. Rollup writes are best-effort with process-scoped error-dedup so transient DB failures bump `rollup_write_failures` instead of crashing the cycle. New `extract_health` doctor check reads the rollup for last 7 days, warns at halt-rate > 10% AND when rollup_write_failures > 0; pre-v104 brains report `ok` silently. Operator CLI surface: `gbrain extract status [--source-id ID] [--kind X] [--verbose] [--json]` (7-day rollup, sorted by halt_rate desc + cost desc, kubectl-style top-5 + "more rows" hint, stable `schema_version: 1` JSON envelope); `gbrain extract --explain <kind>` (resolution chain: pack-declared vs built-in cycle phase, prompt_template + fixture_corpus paths with `✓`/`(missing)` markers, last 7d rollup); `gbrain extract benchmark --pack X --kind Y` (loads pack fixture corpus through strict D-EXTRACT-21 path validation — rejects absolute paths, `..` traversal, null bytes, AND symlinks resolving outside pack root; v0.41.23 ships as a stub-reporter — LLM dispatch deferred to a follow-up release). Pack-author authoring loop: `src/core/schema-pack/manifest-v1.ts` widens `extractable` from `z.boolean()` to `z.union([z.boolean(), ExtractableSpecSchema])` (carries `prompt_template`, `fixture_corpus`, `eval_dimensions`, `benchmark_min_recall`, plus reserved `verifier_path` per D-EXTRACT-37 — parses but refuses at runtime in v0.41.23); new `extractableSpecsFromPack` + `getExtractableSpec` + `refuseVerifierPathInV042` helpers in `src/core/schema-pack/extractable.ts`; `gbrain schema scaffold-extractable <type> --pack <pack>` declares the type extractable in one verb, generates 5 placeholder fixtures + a prompt template stub under `packs/<pack>/{fixtures,prompts}/extract/`, refuses to overwrite without `--force`. Tests: 22 cases in `test/extractable-spec-widening.test.ts` (back-compat boolean shape + struct shape + D-EXTRACT-37 verifier_path refuse), 12 cases in `test/extract/receipt-writer.test.ts` (uses canonical PGLite block per CLAUDE.md R3+R4 — one engine per file with `resetPgliteState` in `beforeEach`), 17 cases in `test/extract/benchmark.test.ts` (path validation rejections + JSONL contract), 15 cases in `test/extract/status.test.ts` (pure aggregation + formatting), 15 cases in `test/schema-pack/scaffold-extractable.test.ts` (scaffold mechanics + explicit privacy-rule guards against real-name leakage), 8 cases in `test/doctor-extract-health.test.ts`. Plan persisted at `~/.claude/plans/system-instruction-you-are-working-stateless-dragonfly.md`. Deferred to a follow-up release: replay versioning + `gbrain extract replay --since v<sha>`; unified LLM dispatcher (deliberately respects v0.41.13 head comment that extraction is regex + cost-cap value-add lives at embed step).
 - `src/commands/import.ts` — `gbrain import <path> [--source-id <id>]`: page import with the v0.34.2.0 path-set checkpoint described above. **v0.37.7.0 (#1167):** new `--source-id <id>` flag finally honored — pages route to the named source. Resolved via `resolveSourceWithTier()` at the boundary; the same flag is now consistent across `import`, `extract`, `graph-query`, and `sources current`. Pinned by `test/import-source-id.test.ts`.
 - `src/commands/graph-query.ts` — `gbrain graph-query <slug> [--type T] [--depth N] [--direction in|out|both] [--include-foreign]`: typed-edge relationship traversal (renders indented tree). **v0.37.7.0 (#1153):** foreign-edge footer always present (`X foreign edges (use --include-foreign to traverse)`) so cross-source edges never disappear silently; `--include-foreign` widens the SQL filter to walk them. Pinned by `test/graph-query.test.ts`.
@@ -300,7 +301,7 @@ strict behavior when unset.
 - `src/core/embedding-dim-check.ts` extension (v0.41.17.0, T5+T6) — facts.embedding dim drift surface. New `readFactsEmbeddingDim(engine): Promise<FactsColumnDimResult>` covers both `vector(N)` and `halfvec(N)` shapes (codex #19 — migration v40 falls back to `vector` on pgvector < 0.7). Regex ordering halfvec-before-vector pinned by tests (substring "vec" appears in "halfvec"; naive `/vector/i` would shadow). New `buildFactsAlterRecipe(dims, configured, type)` emits the paste-ready `DROP INDEX IF EXISTS idx_facts_embedding_hnsw; ALTER TABLE facts ALTER COLUMN embedding TYPE halfvec(N) USING embedding::halfvec(N); CREATE INDEX idx_facts_embedding_hnsw ON facts USING hnsw (embedding halfvec_cosine_ops) WHERE ...` flow (codex #18 — NOT bare REINDEX which doesn't rewrite the index after a column-type change). New `assertFactsEmbeddingDimMatchesConfig(engine)` is the D15 preflight — throws `FactsEmbeddingDimMismatchError` (tagged class with `tag: 'FACTS_EMBEDDING_DIM_MISMATCH'` for parity with the worker-pool's MUST_ABORT semantics) when configured dim doesn't match the column width. Result cached per-engine via `WeakMap`. PGLite engines silently skip. New doctor check `facts_embedding_width_consistency` (registered in `runDoctor` after `embedding_width_consistency`) reuses the same helpers — surfaces drift with paste-ready ALTER recipe identical to the preflight error. Pinned by 18 cases in `test/embedding-dim-check-facts.test.ts`.
 - `src/core/postgres-engine.ts` extension (v0.41.17.0, T6, codex #20) — `insertFact` + `insertFacts` no longer hardcode `tx.unsafe(\`'${embedLit}'::vector\`)` for the embedding column. New `resolveFactsEmbeddingCast()` private method probes `pg_attribute` once per engine instance (cached in `_factsEmbeddingCastSuffix`) and returns `'::halfvec'` when migration v40 created the column as halfvec, else `'::vector'`. Both insert paths use the cached suffix so the cast matches the actual column type. Pre-fix all three insert sites hardcoded `::vector`; works on pgvector >= 0.7 via implicit auto-cast but fails on older pgvector. Test seam `__resetFactsEmbeddingCastCacheForTest()` clears per-engine cache.
 - `src/core/cycle.ts` + `src/core/cycle/extract-atoms.ts` + `src/core/cycle/synthesize-concepts.ts` + `src/commands/extract.ts` + `src/commands/doctor.ts` + `src/core/op-checkpoint.ts` extensions (v0.41.20.0 ops-fix-wave) — six daily-driver ops pains in one bisectable wave. (1) **Batch idempotency for `extract_atoms`:** new `atomsExistingForHashes(engine, sourceId, hashes[])` exported from `src/core/cycle/extract-atoms.ts` replaces the per-hash loop that did 7K individual queries at the start of every cycle on brains with conversation-transcript corpora (5-10 min silent overhead). One batched SQL roundtrip returns the set of `content_hash16` values already extracted as atoms for this source. Fail-open: SQL error logs to stderr and returns an empty set so extraction proceeds. Powered by migration v104 `pages_atom_source_hash_idx` (partial expression index on `frontmatter->>'source_hash'` for atom rows where `deleted_at IS NULL`; Postgres uses `CREATE INDEX CONCURRENTLY` with invalid-remnant pre-drop mirroring v97 `pages_dedup_partial_index`, PGLite uses plain `CREATE INDEX`). Without this index the batch idempotency check would seq-scan pages on big brains and defeat the perf win. (2) **Shorter cycle lock TTL + active in-phase refresh:** `LOCK_TTL_MINUTES` dropped from 30 → 5 in `src/core/cycle.ts`. New exported `buildYieldDuringPhase(lock, outer)` closure calls `lock.refresh()` AND any external hook on every fire; throttled to 30s inside each phase via `maybeYield`. Fires both inside the main work loop AND immediately after every `await chat(...)` LLM call so long Haiku/Sonnet calls don't sit past TTL. `LockHandle` and `buildYieldDuringPhase` exported for test seam access. `synthesize_concepts` no longer fires `yieldDuringPhase` per-concept-group — same hook, throttled via the new shared `maybeYield` helper. A crashed cycle releases its lock 6x faster while a healthy long-running cycle keeps the lock alive. Known residual under shorter TTL: a single `await chat()` past 5 min wallclock can expire the lock mid-await without the original phase noticing — `DbLockHandle.refresh()` throwing on 0 rows affected is filed as P2 follow-up TODO-OPS-2. (3) **Progress wiring through long phases:** new `progress?: ProgressReporter` opt on `ExtractAtomsOpts` and `SynthesizeConceptsOpts`. Cycle.ts passes its phase-level reporter down (NOT a child reporter — that would produce a path collision `cycle.extract_atoms.extract_atoms.work`). Phases only call `tick()` and `heartbeat()`; cycle.ts owns `start()` and `finish()`. You see `[cycle.extract_atoms] N (atoms_created)` ticks every ~1s during both long phases instead of "start" then silence for 10+ min. (4) **`by-mention` resume:** new `mentionsFingerprint({source, type, since, gazetteerHash})` in `src/core/op-checkpoint.ts`. The gazetteer hash is the load-bearing field — adding new entity pages mid-pause shifts the hash, gets a new fingerprint, and triggers a fresh scan against the new gazetteer instead of silently skipping previously-scanned pages. `gbrain extract links --by-mention` now resumes from where it died via the existing `op_checkpoints` framework with a `flushAndCheckpoint` ordering — links flush to the DB FIRST, page keys commit to the checkpoint SECOND, persist THIRD. A crash between `batch.push()` and flush leaves the page un-checkpointed so resume re-scans it. Persist cadence: every 1000 items OR every 30s, whichever first. Clean exit clears the checkpoint. `--dry-run` deliberately skips both load and write so it stays an inspection mode. (5) **`sync_consolidation` doctor check:** multi-source brains see a paste-ready `gbrain sync --all --parallel 4 --workers 4 --skip-failed` recommendation. Single-source brains get "not applicable." SQL errors return `warn` via the check's own try/catch — outer doctor catch isn't a safe assumption. Companion "Multi-source brains" recipe block in `skills/cron-scheduler/SKILL.md` documents the `sync --all` pattern as preferred over per-source entries. (6) **Test-isolation fixes:** `test/cycle-last-full-cycle-at.test.ts` + `test/schema-cli.test.ts` migrated to per-test `GBRAIN_HOME=tempdir` isolation. Pinned by 44 new unit/PGLite cases across 9 files: `test/cycle/extract-atoms-batch.test.ts` (5 — batch idempotency), `test/cycle/cycle-lock-ttl.test.ts` (1 — regression pin on `LOCK_TTL_MINUTES === 5`), `test/op-checkpoint-mentions-fingerprint.test.ts` (7 — fingerprint sensitivity including gazetteer-hash regression guard), `test/cycle/extract-atoms-progress.test.ts` (4 — phase doesn't call start/finish, ticks fire per item), `test/cycle/synthesize-concepts-progress.test.ts` (3 — same shape), `test/cycle/yield-during-phase-refresh.test.ts` (7 — buildYieldDuringPhase actually calls lock.refresh() + outer hook, throws non-fatal), `test/cycle/yield-during-phase-throttle.test.ts` (3 — 30s throttle gate), `test/extract-by-mention-resume.test.ts` (5 — checkpoint persistence ordering, dry-run skips persist, gazetteer change invalidates, filtered pages get checkpointed), `test/doctor-sync-consolidation.test.ts` (6 — edge case matrix for source counts + archived filtering + SQL error path). Two follow-up TODOs filed in TODOS.md under "v0.41.19.0 ops-fix-wave follow-ups": `gbrain sync print-cron` subcommand (TODO-OPS-1) and lock-loss detection via `DbLockHandle.refresh()` throwing on 0 rows affected (TODO-OPS-2).
-- `src/commands/sync.ts` — `gbrain sync` CLI + the `performSync` / `performFullSync` library entrypoints (consumed by the autopilot cycle and the Minion sync handler). v0.22.13 (PR #490): `performSync` wraps its body in a `gbrain-sync` writer lock so two concurrent syncs (manual + autopilot, two terminals, two Conductor workspaces) cannot both write `last_commit` and let the last writer win. Head-drift gate after the import phase re-checks `git rev-parse HEAD`; if HEAD moved (someone ran `git checkout` / `git pull` mid-sync), the bookmark refuses to advance. Vanished files now record a failedFiles entry instead of silent-skip — the silent-skip-then-advance pathology that survived prior hardening passes is dead. Worker engines wrap in try/finally so disconnect always fires (panic-path leak fix). Both PGLite-detection sites use `engine.kind === 'pglite'`. CLI accepts `--workers N` (alias `--concurrency N`), validated via `parseWorkers`. Explicit `--workers` bypasses the auto-path file-count floor; auto path defers to `autoConcurrency()`. Banner moved to stderr. **v0.34.2.0:** the inline `.sort()` over add/mod paths is replaced with `sortNewestFirst(addsAndMods)` from `src/core/sort-newest-first.ts`, so the newest-first descending-lex policy lives in one helper shared with `gbrain import` instead of drifting across two files. **v0.40.3.0 (productionized from PR #1314 by @garrytan-agents):** the load-bearing change is the lock-identity invariant — `performSync` now defaults to a per-source lock id (`gbrain-sync:<sourceId>`) whenever `opts.sourceId` is set, AND wraps the writer window in `withRefreshingLock` from `src/core/db-lock.ts` so long-running sources (the PR motivator: media-corpus / 250K+ chunks) don't lose their lock at the 30-min TTL mid-run. Legacy single-default-source brains keep the bare `tryAcquireDbLock(SYNC_LOCK_ID)` path for back-compat. `SyncOpts.lockId?: string` is the explicit override (escape hatch; production callers don't set it). The fix closes the bug class Codex's outside-voice review caught in the original PR — a `sync --all` worker on per-source lock racing against `sync --source foo` on global lock would have silently corrupted the same source. `gbrain sync --all` got a continuous worker pool: `parseWorkers`-validated `--parallel N` (default `min(sourceCount, --workers, DEFAULT_PARALLEL_SOURCES=4)` from `src/core/sync-concurrency.ts`), `parallel` long-lived async workers pulling from a shared FIFO queue (no head-of-line blocking from wave dispatch), per-source `withSourcePrefix(src.id, ...)` wrap from `src/core/console-prefix.ts` so every `slog`/`serr` line carries `[<source-id>] ` for kubectl-style greppable parallel output. `--skip-failed` and `--retry-failed` reject with a paste-ready error when combined with `--parallel > 1` (the brain-global `sync-failures.jsonl` has no per-source scope; v0.41+ TODO source-scopes it). Connection-budget stderr warning fires when `parallel × workers × 2 > 16` (the `× 2 per-file pool` factor: each per-file worker opens its own `PostgresEngine` with `poolSize=2`, so `--parallel 4 --workers 4` is actually 32 connections, not 16 — Codex caught the 2× understatement in the original plan). New exports `resolveParallelism`, `syncOneSource`, `buildSyncStatusReport`, `printSyncStatusReport`, `SyncStatusReport` back the `gbrain sources status` dashboard surface (D3 → `sources` subcommand, not a `sync` flag, so reads and writes don't share a verb). Stable `--json` envelope `{schema_version: 1, sources, parallel, ok_count, error_count}` on stdout under `--json`; human banners route to stderr via the `humanSink` helper so `jq` parses cleanly. Exit matrix: 0 = all ok, 1 = any error, 2 = cost-prompt-not-confirmed (unchanged). The dashboard SQL is the canonical `content_chunks ch JOIN pages pg ON pg.id = ch.page_id WHERE pg.deleted_at IS NULL` shape with `archived = false` filter at the caller — the original PR shipped `chunks ch JOIN ON page_slug`, which would have crashed on PGLite parse and silently zeroed on Postgres via the swallow-catch. Embedding column resolved via `resolveEmbeddingColumn(undefined, cfg)` from `src/core/search/embedding-column.ts` so Voyage / multimodal / non-default-column brains see counts against the column they actually use. Errors propagate from the dashboard SQL — no swallow-catch (Q2 sub-fix). The IRON RULE regression lives in `test/e2e/sync-status-pglite.test.ts`: real PGLite seeds 2 sources × pages × chunks, soft-deletes 1 page, archives 1 source, validates the SQL excludes both AND the active embedding column is the one used. 38 in-file `console.log`/`console.error` call sites inside `performSync*` migrated to `slog`/`serr` (top-level `runSync` orchestrator console calls intentionally stay outside the prefix scope). **v0.41.25.0 (supersedes PR #1538):** sync delete loop rewritten as interleaved per-batch resolve+delete using `engine.resolveSlugsByPaths` + `engine.deletePages` from `src/core/engine.ts`. 73K-delete commit drops from ~146K SQL round-trips (~5h on Postgres+pgbouncer) to ~292 round-trips (~2min) — closes the cascade-staleness bug class where one big-delete commit jammed every other source's sync for hours. Per-batch try-catch decomposes batch DELETE failures to per-slug `deletePage` fallback so a transient blip on batch 73 doesn't lose 500 deletes; unrecoverable per-slug failures land in `failedFiles` (matches the existing import-loop pattern). `pagesAffected` filters to D6 confirmed-deleted slugs (downstream extract/embed stop wasting work on phantoms). Same batched slug-resolve treatment for the rename loop (`gbrain sync` renames — Phase 3 in the plan; the per-file `importFile` stays per-file, only the slug-resolve N+1 gets batched). The `resolveSlugByPathOrSourcePath` single-call helper at `sync.ts:267` now delegates to `engine.resolveSlugsByPaths` when `sourceId` is set (D8 DRY — one owner of the SQL + fallback semantics), keeping legacy `executeRaw` fallback for the no-sourceId path which is functionally dead post-v0.34.1's source-resolution wiring. Legacy no-sourceId branch in the delete loop preserves correctness on a slow path; the engine batch methods require sourceId so the new fast path only fires when production threading is intact. `failedFiles` declaration hoisted to top of `performSyncInner` so both delete decompose and import loops feed the same sync-bookmark gate. **v0.41.31.0 (mode-aware cost gate):** the v0.20.0 unconditional `sync --all` ConfirmationRequired gate is superseded by a mode-aware gate driven by `willEmbedSynchronously` + `shouldBlockSync` from `src/core/embedding.ts`. federated_v2 is resolved ONCE up front (shared with the fan-out below). On the DEFERRED path (v2 on, parallel) embedding goes to per-source `embed-backfill` jobs that carry their own `$X/source/24h` cap (default $25, `SPEND_CAP_CONFIG_KEY` from `embed-backfill-submit.ts`), so the gate is INFORMATIONAL — it prints an FYI naming the backfill cap + current backlog (`sumStaleChunkChars({signature})` priced via `estimateCostFromChars` at `currentEmbeddingPricePerMTok()`) + the count of already-queued `embed-backfill` jobs (TODO-2 visibility), and NEVER exits 2. On the INLINE path (v2 off, or `--serial` without `--no-embed`) the BLOCKING gate fires only when the NEW-CONTENT estimate exceeds the `sync.cost_gate_min_usd` floor (default $0.50). The new-content estimate (`estimateInlineNewTokens`) contributes ZERO for sources provably unchanged since last sync (`isSourceUnchangedSinceSync` HEAD==last_commit + clean tree, AND `chunker_version === CURRENT`) — mirrors doctor's `sync_freshness` + sync's own do-work gate — and the full-tree ceiling for changed sources. F2 fix: the pre-existing stale backlog (NULL embeddings + signature drift) is shown informationally but NEVER gated on, because sync doesn't sweep it inline (`gbrain embed --stale` does); gating on it would block every inline cron after a model swap. New helpers `resolveCostGateFloorUsd(engine)` (reads `sync.cost_gate_min_usd`, fail-open $0.50, accepts 0) + `resolveBackfillCapUsd(engine)`. The `sync --all` SELECT widens to carry `last_commit + chunker_version` (both columns predate v0.41; no migration). JSON envelopes gain `mode` + `gate` discriminators (`dry_run | deferred_notice | below_floor | confirmation_required`). `SyncStatusReportSource` gains `backfill_queued` / `backfill_active` / `backfill_last_completed_at` (best-effort; 0/null on pre-minions brains). All embedding-cost preview math reads the configured model name via `getEmbeddingModelName()` (no hardcoded OpenAI). Pinned by `test/sync-cost-gate.serial.test.ts` + `test/sync-cost-preview.test.ts`. **v0.41.37.0 (#1569):** new `SyncOpts.noSchemaPack` (CLI `--no-schema-pack`, threaded through `performSync` AND `syncOneSource` so `sync --all` honors it) — skips `loadActivePack` entirely so no user-supplied pack page-type regex runs during import; pages fall back to legacy prefix typing. Escape hatch for completing a sync when a suspect pack regex is the suspected cause of a wedge (re-run extraction later). Same release adds a per-file BEGIN heartbeat: `if (process.env.GBRAIN_SYNC_TRACE) serr('[sync] begin import: <path>')` emitted BEFORE `importFile` (the existing `progress.tick` fires only AFTER `importFile` returns — useless when one file wedges). Off by default to avoid a line per file on huge brains; `serr` is source-prefix-aware so under `--workers >1` / `--all` the stuck file is the begin-line with no matching completion. Triage doc: `docs/architecture/serve-sync-concurrency.md` (PGLite single-writer serve↔sync contention — stop `gbrain serve` before a large PGLite sync — plus the `GBRAIN_SYNC_TRACE` + `--no-schema-pack` hang-triage recipes).
+- `src/commands/sync.ts` — `gbrain sync` CLI + the `performSync` / `performFullSync` library entrypoints (consumed by the autopilot cycle and the Minion sync handler). v0.22.13 (PR #490): `performSync` wraps its body in a `gbrain-sync` writer lock so two concurrent syncs (manual + autopilot, two terminals, two Conductor workspaces) cannot both write `last_commit` and let the last writer win. Head-drift gate after the import phase re-checks `git rev-parse HEAD`; if HEAD moved (someone ran `git checkout` / `git pull` mid-sync), the bookmark refuses to advance. Vanished files now record a failedFiles entry instead of silent-skip — the silent-skip-then-advance pathology that survived prior hardening passes is dead. Worker engines wrap in try/finally so disconnect always fires (panic-path leak fix). Both PGLite-detection sites use `engine.kind === 'pglite'`. CLI accepts `--workers N` (alias `--concurrency N`), validated via `parseWorkers`. Explicit `--workers` bypasses the auto-path file-count floor; auto path defers to `autoConcurrency()`. Banner moved to stderr. **v0.34.2.0:** the inline `.sort()` over add/mod paths is replaced with `sortNewestFirst(addsAndMods)` from `src/core/sort-newest-first.ts`, so the newest-first descending-lex policy lives in one helper shared with `gbrain import` instead of drifting across two files. **v0.40.3.0 (productionized from PR #1314 by @garrytan-agents):** the load-bearing change is the lock-identity invariant — `performSync` now defaults to a per-source lock id (`gbrain-sync:<sourceId>`) whenever `opts.sourceId` is set, AND wraps the writer window in `withRefreshingLock` from `src/core/db-lock.ts` so long-running sources (the PR motivator: media-corpus / 250K+ chunks) don't lose their lock at the 30-min TTL mid-run. Legacy single-default-source brains keep the bare `tryAcquireDbLock(SYNC_LOCK_ID)` path for back-compat. `SyncOpts.lockId?: string` is the explicit override (escape hatch; production callers don't set it). The fix closes the bug class Codex's outside-voice review caught in the original PR — a `sync --all` worker on per-source lock racing against `sync --source foo` on global lock would have silently corrupted the same source. `gbrain sync --all` got a continuous worker pool: `parseWorkers`-validated `--parallel N` (default `min(sourceCount, --workers, DEFAULT_PARALLEL_SOURCES=4)` from `src/core/sync-concurrency.ts`), `parallel` long-lived async workers pulling from a shared FIFO queue (no head-of-line blocking from wave dispatch), per-source `withSourcePrefix(src.id, ...)` wrap from `src/core/console-prefix.ts` so every `slog`/`serr` line carries `[<source-id>] ` for kubectl-style greppable parallel output. `--skip-failed` and `--retry-failed` reject with a paste-ready error when combined with `--parallel > 1` (the brain-global `sync-failures.jsonl` has no per-source scope; v0.41+ TODO source-scopes it). Connection-budget stderr warning fires when `parallel × workers × 2 > 16` (the `× 2 per-file pool` factor: each per-file worker opens its own `PostgresEngine` with `poolSize=2`, so `--parallel 4 --workers 4` is actually 32 connections, not 16 — Codex caught the 2× understatement in the original plan). New exports `resolveParallelism`, `syncOneSource`, `buildSyncStatusReport`, `printSyncStatusReport`, `SyncStatusReport` back the `gbrain sources status` dashboard surface (D3 → `sources` subcommand, not a `sync` flag, so reads and writes don't share a verb). Stable `--json` envelope `{schema_version: 1, sources, parallel, ok_count, error_count}` on stdout under `--json`; human banners route to stderr via the `humanSink` helper so `jq` parses cleanly. Exit matrix: 0 = all ok, 1 = any error, 2 = cost-prompt-not-confirmed (unchanged). The dashboard SQL is the canonical `content_chunks ch JOIN pages pg ON pg.id = ch.page_id WHERE pg.deleted_at IS NULL` shape with `archived = false` filter at the caller — the original PR shipped `chunks ch JOIN ON page_slug`, which would have crashed on PGLite parse and silently zeroed on Postgres via the swallow-catch. Embedding column resolved via `resolveEmbeddingColumn(undefined, cfg)` from `src/core/search/embedding-column.ts` so Voyage / multimodal / non-default-column brains see counts against the column they actually use. Errors propagate from the dashboard SQL — no swallow-catch (Q2 sub-fix). The IRON RULE regression lives in `test/e2e/sync-status-pglite.test.ts`: real PGLite seeds 2 sources × pages × chunks, soft-deletes 1 page, archives 1 source, validates the SQL excludes both AND the active embedding column is the one used. 38 in-file `console.log`/`console.error` call sites inside `performSync*` migrated to `slog`/`serr` (top-level `runSync` orchestrator console calls intentionally stay outside the prefix scope). **v0.41.25.0 (supersedes PR #1538):** sync delete loop rewritten as interleaved per-batch resolve+delete using `engine.resolveSlugsByPaths` + `engine.deletePages` from `src/core/engine.ts`. 73K-delete commit drops from ~146K SQL round-trips (~5h on Postgres+pgbouncer) to ~292 round-trips (~2min) — closes the cascade-staleness bug class where one big-delete commit jammed every other source's sync for hours. Per-batch try-catch decomposes batch DELETE failures to per-slug `deletePage` fallback so a transient blip on batch 73 doesn't lose 500 deletes; unrecoverable per-slug failures land in `failedFiles` (matches the existing import-loop pattern). `pagesAffected` filters to D6 confirmed-deleted slugs (downstream extract/embed stop wasting work on phantoms). Same batched slug-resolve treatment for the rename loop (`gbrain sync` renames — Phase 3 in the plan; the per-file `importFile` stays per-file, only the slug-resolve N+1 gets batched). The `resolveSlugByPathOrSourcePath` single-call helper at `sync.ts:267` now delegates to `engine.resolveSlugsByPaths` when `sourceId` is set (D8 DRY — one owner of the SQL + fallback semantics), keeping legacy `executeRaw` fallback for the no-sourceId path which is functionally dead post-v0.34.1's source-resolution wiring. Legacy no-sourceId branch in the delete loop preserves correctness on a slow path; the engine batch methods require sourceId so the new fast path only fires when production threading is intact. `failedFiles` declaration hoisted to top of `performSyncInner` so both delete decompose and import loops feed the same sync-bookmark gate. **v0.41.31.0 (mode-aware cost gate):** the v0.20.0 unconditional `sync --all` ConfirmationRequired gate is superseded by a mode-aware gate driven by `willEmbedSynchronously` + `shouldBlockSync` from `src/core/embedding.ts`. federated_v2 is resolved ONCE up front (shared with the fan-out below). On the DEFERRED path (v2 on, parallel) embedding goes to per-source `embed-backfill` jobs that carry their own `$X/source/24h` cap (default $25, `SPEND_CAP_CONFIG_KEY` from `embed-backfill-submit.ts`), so the gate is INFORMATIONAL — it prints an FYI naming the backfill cap + current backlog (`sumStaleChunkChars({signature})` priced via `estimateCostFromChars` at `currentEmbeddingPricePerMTok()`) + the count of already-queued `embed-backfill` jobs (TODO-2 visibility), and NEVER exits 2. On the INLINE path (v2 off, or `--serial` without `--no-embed`) the BLOCKING gate fires only when the NEW-CONTENT estimate exceeds the `sync.cost_gate_min_usd` floor (default $0.50). The new-content estimate (`estimateInlineNewTokens`) contributes ZERO for sources provably unchanged since last sync (`isSourceUnchangedSinceSync` HEAD==last_commit + clean tree, AND `chunker_version === CURRENT`) — mirrors doctor's `sync_freshness` + sync's own do-work gate — and the full-tree ceiling for changed sources. F2 fix: the pre-existing stale backlog (NULL embeddings + signature drift) is shown informationally but NEVER gated on, because sync doesn't sweep it inline (`gbrain embed --stale` does); gating on it would block every inline cron after a model swap. New helpers `resolveCostGateFloorUsd(engine)` (reads `sync.cost_gate_min_usd`, fail-open $0.50, accepts 0) + `resolveBackfillCapUsd(engine)`. The `sync --all` SELECT widens to carry `last_commit + chunker_version` (both columns predate v0.41; no migration). JSON envelopes gain `mode` + `gate` discriminators (`dry_run | deferred_notice | below_floor | confirmation_required`). `SyncStatusReportSource` gains `backfill_queued` / `backfill_active` / `backfill_last_completed_at` (best-effort; 0/null on pre-minions brains). All embedding-cost preview math reads the configured model name via `getEmbeddingModelName()` (no hardcoded OpenAI). Pinned by `test/sync-cost-gate.serial.test.ts` + `test/sync-cost-preview.test.ts`. **v0.41.37.0 (#1569):** new `SyncOpts.noSchemaPack` (CLI `--no-schema-pack`, threaded through `performSync` AND `syncOneSource` so `sync --all` honors it) — skips `loadActivePack` entirely so no user-supplied pack page-type regex runs during import; pages fall back to legacy prefix typing. Escape hatch for completing a sync when a suspect pack regex is the suspected cause of a wedge (re-run extraction later). Same release adds a per-file BEGIN heartbeat: `if (process.env.GBRAIN_SYNC_TRACE) serr('[sync] begin import: <path>')` emitted BEFORE `importFile` (the existing `progress.tick` fires only AFTER `importFile` returns — useless when one file wedges). Off by default to avoid a line per file on huge brains; `serr` is source-prefix-aware so under `--workers >1` / `--all` the stuck file is the begin-line with no matching completion. Triage doc: `docs/architecture/serve-sync-concurrency.md` (PGLite single-writer serve↔sync contention — stop `gbrain serve` before a large PGLite sync — plus the `GBRAIN_SYNC_TRACE` + `--no-schema-pack` hang-triage recipes). **v0.42.13.0 (#1633) — orphan-pileup / unkillable sync:** exports `resolveSyncHardDeadline(args, {isTty, env, defaultNonTtySec?})` (pure; precedence `--no-hard-deadline` > `--hard-deadline <s>` > `--timeout <s>`(non-`--all`, auto-arms the hard backstop) > `GBRAIN_SYNC_MAX_RUNTIME_SECONDS` env > non-TTY default 3600s > none; `HARD_DEADLINE_GRACE_SEC=30`) + `composeAbortSignals(...signals)` (CQ2 — `AbortSignal.any` over the defined signals; single→passthrough, none→undefined). `src/cli.ts` installs `installProcessWatchdog` (see `src/core/process-watchdog.ts`) for `command === 'sync'` BEFORE `connectEngine` using that resolution, disposing it in the dispatch `finally` — so a runaway sync (incl. a connect-phase hang) is SIGKILLed by the deadline even when its event loop is starved, making the cron orphan-pileup structurally impossible. Part B graceful-cancel: `runSync` registers a SIGINT handler on both the single-source and `--all` paths that aborts an interrupt `AbortController` composed (via `composeAbortSignals`) with the per-source `--timeout` signal — Ctrl-C returns a clean `partial` + releases the lock via the normal finally instead of a hard cut that leaks the lock (process-cleanup.ts already owns SIGTERM lock-release + exit; the watchdog owns the hard kill). `src/core/db-lock.ts:withRefreshingLock` now `unref()`s its refresh `setInterval`. NOT root-causing the spin (no repro; #1633 stays open for it — leading lead is ReDoS in pack link-inference, partially mitigated by v0.41.37.0's redos-guard); the watchdog heartbeat + existing `[gbrain phase]` breadcrumbs are the diagnosis surface. Per-file cooperative yielding was evaluated and skipped (the import loop already `await`s per file → already a yield point; only true mid-file synchronous starvation matters, which the watchdog covers). Pinned by `test/sync-hard-deadline.test.ts` (resolution precedence + `composeAbortSignals`).
 - `src/commands/import.ts` — `gbrain import` CLI + `runImport` library entrypoint. v0.34.2.0 replaces the prior positional-index checkpoint (`processedIndex: N` into a sorted file list) with a path-set checkpoint via `src/core/import-checkpoint.ts`. The walk still applies `sortNewestFirst()` for embed-cost ordering, but checkpoint correctness no longer depends on sort order. A file enters `completed: Set<relativePath>` only when its `processFile` returns success (including content-hash short-circuit no-ops); failed files never enter the set, so the next run retries them automatically with no manual `~/.gbrain/import-checkpoint.json` delete. Three bug classes died: parallel-import-with-slow-worker drops the slow file on crash-resume (closed — the slow file isn't in `completed` until its own `processFile` resolves), failed-file-bumps-counter-past-itself (closed — failures don't add to `completed`), and v0.33.x sort-flip-drops-newest-N-on-cross-version-resume (closed — order is no longer part of the checkpoint). Old positional checkpoints are detected and discarded with a stderr line on first resume; re-walking is cheap because `content_hash` short-circuits unchanged files. Checkpoint persists every 100 successful adds, not every 100 processed files, so a long failure tail doesn't churn the JSON. Pinned by `test/import-checkpoint.test.ts` (18 unit cases over the helpers) + `test/import-resume.test.ts` (5 integration cases under PGLite, including the SLUG_MISMATCH retry regression codex caught during plan-eng-review).
 - `src/core/import-checkpoint.ts` (v0.34.2.0) — `loadCheckpoint(brainDir)`, `saveCheckpoint(brainDir, completed)`, `resumeFilter(files, completed, brainDir)`, `clearCheckpoint()`, plus the `ImportCheckpoint` type. Path-set checkpoint format (`{schema_version, brainDir, completed: string[]}`) replaces the v0.33.x positional `{processedIndex: N}` format. Atomic write via `.tmp` + `rename()` so a mid-write crash never leaves a partial JSON. `loadCheckpoint` returns `null` on: missing file, malformed JSON, brainDir mismatch (you ran import against a different brain), and the old positional format (logged to stderr before being discarded). `resumeFilter` returns `{toProcess, skippedCount}` — pure, no I/O, deterministic. `clearCheckpoint` is no-op-on-missing for clean-exit cleanup. Honors `GBRAIN_HOME` via `gbrainPath()` so test isolation via `withEnv({GBRAIN_HOME: tmpdir})` works without monkey-patching the fs layer. Best-effort persistence — `saveCheckpoint` logs warnings on write errors but never throws, so import keeps making progress even if disk is full.
 - `src/core/sort-newest-first.ts` (v0.34.2.0) — single source of truth for the descending-lex sort that `gbrain import` and `gbrain sync` both apply. Mutates in place (Array.prototype.sort semantics), returns the same array reference for fluent chaining. Empty/single-element inputs short-circuit. Future ordering changes flip one line in this helper instead of touching two CLI commands. Pinned by `test/sort-newest-first.test.ts` (5 hermetic cases: descending order, mixed prefixes, empty input, single-element input, in-place-mutation contract).
diff --git a/llms-full.txt b/llms-full.txt
index 19dfcb4b4..ee43f40ba 100644
--- a/llms-full.txt
+++ b/llms-full.txt
@@ -314,6 +314,7 @@ strict behavior when unset.
 - `src/core/dry-fix.ts` — `gbrain doctor --fix` engine. `autoFixDryViolations(fixes, {dryRun})` rewrites inlined rules to `> **Convention:** see [path](path).` callouts via three shape-aware expanders (bullet / blockquote / paragraph). Five guards: working-tree-dirty (`getWorkingTreeStatus()` returns 3-state `'clean' | 'dirty' | 'not_a_repo'`), no-git-backup, inside-code-fence, already-delegated (40-line proximity, consistent with detector), ambiguous-multi-match, block-is-callout. `execFileSync` array args (no shell — no injection surface). EOF newline preserved. **v0.37.1.0:** safety primitives extracted to `src/core/skill-fix-gates.ts` (back-compat re-exports preserved). New `MISSING_RULE_PATTERNS` INSERT pattern type lives alongside the existing REPLACE patterns — same auto-fix entry point, same git-safety gates, but instead of rewriting an existing block, INSERT patterns place a canonical callout at a target offset (today: `after-h1-paragraph` only; designed to extend). The first INSERT pattern is `brain_first`, which auto-inserts `> **Convention:** see [conventions/brain-first.md](../conventions/brain-first.md) for the lookup chain (search → query → get_page → external).` on any flagged SKILL.md whose analyzer verdict is `missing_brain_first`. Idempotent — re-runs detect the existing callout and skip.
 - `src/core/backoff.ts` — Adaptive load-aware throttling: CPU/memory checks, exponential backoff, active hours multiplier
 - `src/core/retry.ts` (v0.41.19.0, extended v0.41.27.0) — canonical retry primitive for transient connection errors. Exports `withRetry<T>(fn, opts)` execution wrapper + `BULK_RETRY_OPTS` constant (`{maxRetries:3, delayMs:1000, delayMaxMs:10000, jitter:'decorrelated'}`) tuned for Supabase Supavisor's 5-10s circuit-breaker recovery window + `BATCH_AUDIT_SITES` typed const (closed enum of every audit-emission site) + `resolveBulkRetryOpts(env)` (reads `GBRAIN_BULK_MAX_RETRIES` / `GBRAIN_BULK_RETRY_BASE_MS` / `GBRAIN_BULK_RETRY_MAX_MS` with `>=0` validation, throws on bad input with paste-ready hint) + `abortableSleep(ms, signal?)` (AbortSignal-aware setTimeout) + `RetryAbortError` (tagged error for clean shutdown) + `computeNextDelay()` (pure-fn for 3 jitter modes: `'none'`, `'full'`, `'decorrelated'`). The execution wrapper is consumed by `postgres-engine.ts` + `pglite-engine.ts` batch primitives (`addLinksBatch` / `addTimelineEntriesBatch` / `upsertChunks`) so every caller — current AND future — inherits retry as part of the data-primitive's contract. CI guard `scripts/check-no-double-retry.sh` fails the build on `withRetry(...engine.batch...)` patterns (prevents 3×3=9 retry amplification). CI guard `scripts/check-batch-audit-site.sh` validates every string-literal `auditSite: '...'` against the closed `BATCH_AUDIT_SITES` enum. **Closes the v0.41.17 production incident** where ~3,000 rows were silently lost per dream cycle on a 16K-page brain because the prior single-500ms-retry shape couldn't survive Supavisor's 5-10s circuit-breaker recovery. Decorrelated jitter (AWS-style: `uniform(base, prevDelay*3)` capped at `delayMaxMs`) replaces naive `'full'` jitter which allowed near-zero retries that re-hit the still-recovering breaker. Pinned by `test/core/retry.test.ts` (37 cases) + `test/core/retry-stress.slow.test.ts` (5 cases simulating 100 batches × 30% blip rate, asserts zero row loss). **v0.41.27.0 (#1570):** `WithRetryOpts` gains optional `reconnect?: () => Promise<void>` callback awaited in the catch branch AFTER `isRetryableConnError` classification but BEFORE the inter-attempt sleep — lets engine-level callers rebuild a dead pool/singleton between attempts. `PostgresEngine.batchRetry` injects `() => this.reconnect()` so the v0.22.1 race-safe `_reconnecting` guard kicks in transparently. Fail-loud posture (per codex outside-voice finding 3): a reconnect throw PROPAGATES as the new error, replacing the symptomatic "No database connection" so operators see the real cause (auth failure, EHOSTUNREACH). `onRetry` callbacks now also awaited (back-compat-safe: existing sync arrows work identically; async callbacks now correctly delay the sleep). Pinned by `test/core/retry-reconnect.test.ts` (5 cases: ordering, back-compat, fail-loud propagation, signal-abort short-circuit, awaited onRetry timing) + `test/e2e/db-singleton-shared-recovery.test.ts` (3 DB-gated cases pinning shared-singleton survival via retry reconnect + diagnostic audit fires on disconnect + instance-pool disconnect doesn't touch module singleton).
+- `src/core/process-watchdog.ts` (v0.42.11.0, #1633) — out-of-band hard-deadline killer for `gbrain sync`. A spinning sync (synchronous catastrophic-regex / ReDoS in pack link-inference) STARVES the main event loop, so the existing SIGTERM handler (`process-cleanup.ts`), `--timeout` `setTimeout`, and abort-flag checks can't fire — the process becomes unkillable-by-SIGTERM and, under cron, orphans pile up for 24h+ (the reported incident). `installProcessWatchdog({deadlineMs, graceMs?, label?, heartbeatMs?, onWarn?}): WatchdogHandle` spawns a Bun `worker_threads` Worker via `new Worker(code, {eval: true, workerData})` — its own OS thread + event loop fires even while main is in an unyielding sync loop. At `deadlineMs` it `process.kill(process.pid, 'SIGTERM')` (clean-shutdown chance if responsive); at `deadlineMs+graceMs` `process.kill(process.pid, 'SIGKILL')` (uncatchable — guaranteed death under starvation). Signaling SELF has NO PID-reuse footgun (current PID never reused while alive — the reason the detached-child-watches-parent design was rejected). `eval: true` bakes the worker body into the `bun build --compile` binary with no separate-file embedding. Empirically validated on Bun 1.3.13 (worker timer + SIGKILL killed a `while(true){}`-starved process; Codex outside-voice + a repo spike both confirmed). `handle.dispose()` (clean-exit `finally`) `worker.terminate()`s it; `unref()`'d so it never keeps the process alive. Pure `watchdogDecision(elapsedMs, deadlineMs, graceMs) → 'wait'|'sigterm'|'sigkill'` extracted for unit tests. Optional `heartbeatMs` emits periodic `[<label>] parent alive Ns, hard-kill in ~Ms` lines (visible in cron logs even under starvation — the diagnosis surface). Fallback: if `new Worker` throws, degrades to an in-process timer with a loud warning that it can't fire under starvation. Reusable beyond sync (autopilot/cycle are follow-up adopters). Pinned by `test/process-watchdog.test.ts` (pure decision matrix + handle contract) + `test/process-watchdog.serial.test.ts` (Bun-pinned spawn integration: a starved harness process IS killed ~deadline+grace, a no-watchdog control does NOT self-exit, clean dispose never kills). Wired into `src/cli.ts` sync dispatch BEFORE `connectEngine` (so a connect-phase hang is bounded too — `timeout-layer-vs-connectengine` learning); deadline resolved by `resolveSyncHardDeadline` in `sync.ts` (precedence: `--no-hard-deadline` > `--hard-deadline` > `--timeout`(non-`--all`) > `GBRAIN_SYNC_MAX_RUNTIME_SECONDS` env > non-TTY default 3600s > none).
 - `src/core/audit/batch-retry-audit.ts` (v0.41.19.0, extended v0.41.26.1) — JSONL audit primitive for batch-retry events. Built on `audit-writer.ts` cathedral. Schema: `{ts, site, batch_size, attempt, outcome: 'success' | 'exhausted', delay_ms, error_message_summary, error_code?}`. Privacy posture: NEVER logs slugs / page IDs / content (mirrors `shell-audit.ts` from v0.20+). `logBatchRetry` fires per successful retry recovery; `logBatchExhausted` fires when retries exhaust and rows are lost. `readRecentBatchRetryEvents(hours=24)` returns `{events, corrupted_lines, files_scanned, files_unreadable}` — corruption + permission errors surface to doctor, not silently swallowed (codex H-9). `pruneOldBatchRetryAuditFiles(daysToKeep=30)` deletes old files (codex H-8 — implements the v0.41 plan's "30-day pruning convention" for real). Called from `gbrain dream --phase purge`. File: `~/.gbrain/audit/batch-retry-YYYY-Www.jsonl` (honors `GBRAIN_AUDIT_DIR`). **v0.41.26.1 privacy backfill:** `summarizeError` now routes error messages through the shared `redactConnectionInfo` helper from `src/core/audit/redact-connection-info.ts` BEFORE truncation, so DSNs / hostnames / credentials / IPv4 octets can't leak from a Postgres connection-failure error into operator-shared JSONL dumps. Same risk class as lock-renewal-audit; closed in the same wave. Pinned by `test/audit/batch-retry-audit.test.ts` (12 cases) + new `test/audit/batch-retry-redaction.test.ts` (3 privacy regressions).
 - `src/core/audit/lock-renewal-audit.ts` (v0.41.26.1) — JSONL audit primitive for per-job lock-renewal faults. Sibling of `batch-retry-audit.ts`; built on `audit-writer.ts`. Four outcomes: `failure` (single renewLock throw, counter incremented), `success_after_failure` (recovery; emits the recovery count), `gave_up` (time-based deadline exceeded; abort fired), `executeJob_rejected` (the SECOND unhandledRejection vector from D7 — the stored `executeJob(...).finally(...)` promise itself rejected, e.g. failJob threw during the same DB outage). Schema: `{ts, job_id, job_name, attempt?, outcome, error_message_summary?, error_code?}`. Privacy: NEVER logs `lock_token` or `job.data`. Error summaries route through `redactConnectionInfo` BEFORE truncation. Defense-in-depth: every audit call inside the lock-renewal tick's catch block is wrapped in its own inner try/catch so a misbehaving audit-writer can't re-introduce the unhandledRejection bug class via a new surface. `readRecentLockRenewalEvents(hours=24)` walks current + previous ISO week with corrupted-line tolerance. `pruneOldLockRenewalAuditFiles(daysToKeep=30)` is ready for future dream-cycle purge wiring (filed as TODO-LR-3). File: `~/.gbrain/audit/lock-renewal-YYYY-Www.jsonl`. Operator UX is `tail -F` until the doctor check lands (TODO-LR-2). Pinned by `test/audit/lock-renewal-audit.test.ts` (11 cases).
 - `src/core/audit/redact-connection-info.ts` (v0.41.26.1) — Shared pure helper. `redactConnectionInfo(text: string): string` strips Postgres connection info before any audit JSONL write: `postgres://`/`postgresql://` URLs, `host=foo`, `user=foo`, `password=foo`, `pwd=foo`, IPv4 octets. Each match becomes `<REDACTED:kind>`. Negative-lookbehind/lookahead `[\w.@-]` on the IPv4 pattern defeats version-string false positives like `v3.1.4.0` or `tree-sitter@0.26.3.1` while still matching real IPs in PG errors (`(192.168.1.42)`). Order-sensitive pattern set: URL forms first so substrings inside URLs don't get double-redacted. Idempotent (running twice produces same output), pure (no I/O), hot-path-safe (regex compiled at module load). Wired into BOTH `lock-renewal-audit.ts` (new) AND `batch-retry-audit.ts` (privacy backfill — same risk class). Risk model documented in module header: Postgres errors during connection failures embed DSNs / credentials / hostnames into error messages; operators routinely paste audit dumps into GitHub issues / Slack to debug; the audit channel must be safe to share by construction. Known limitations: bare-quoted hostnames (`at "db.example.com"`) and bare-quoted usernames (`for user "postgres.foo"`) are NOT caught — the IP in those PG error shapes is the highest-value leak and IS caught. Quoted-string patterns filed as TODO-LR-5. Pinned by `test/audit/redact-connection-info.test.ts` (15 cases covering all 5 patterns + real Supabase fixture + ENOTFOUND fixture + version-string false-positive defense).
@@ -335,7 +336,7 @@ strict behavior when unset.
 - `src/core/conversation-parser/` (v0.41.16.0, v0.41.24.0, v0.41.29.0) — 14-pattern built-in chat-format registry + opt-in LLM polish/fallback. Replaces the single inline regex that PR #1461 was trying to extend with one more shape. **v0.41.29.0** added the 14th pattern `bold-name-no-time` (regex `/^\*\*(?!\[)(.+?):\*\*\s*(.*)$/`, declared at index 3 after `bold-paren-time`): parses `**Speaker:** text` with NO per-line timestamp — the shape Circleback / Granola / Zoom emit. Anchors every message at `T00:00:00Z` of the frontmatter date (no fabricated wall-clock; line order preserves sequence), same no-time convention as `irc-classic`. The non-shadow guarantee is the colon-INSIDE-bold regex (NOT declaration order — `parse.ts` scores every candidate independently, order is only the tie-break); the `(?!\[)` lookahead rejects telegram-bracket `**[18:37] Name:**` so disabling telegram-bracket yields an honest `no_match` instead of `speaker="[18:37] Name"`. Because `**Label:** text` is a common prose idiom, the pattern sets the new optional `PatternEntry.score_full_body: true`: `parse.ts` recomputes the winner's acceptance score over the FULL body (not just the 10-line head window) before the `SCORING_MIN_ACCEPTANCE` floor, so a notes page with bold labels clustered in its first 10 lines (head score 0.3, which is NOT `< SCORING_HEAD_TRIGGER_THRESHOLD` so the head fallback never fires) drops to its true low density and stays `no_match` instead of mis-parsing as a conversation. The same wave scrubbed the pre-existing real names in `bold-paren-time`'s `test_positive`. Pinned by the `bold-name-no-time` + clustered-head `no_match` cases in `test/conversation-parser/parse.test.ts` + `bold-name-no-time.jsonl` / `adversarial.jsonl` fixtures. **v0.41.24.0 (closes #1533)** added the 13th pattern `bold-paren-time` (`**Speaker** (HH:MM): text` and `**Speaker** (HH:MM:SS): text` — the time-only shape an OpenClaw meeting-ingestion pipeline produces from Circleback transcripts; date_source: frontmatter) AND tightened the parser's fallback gates in `parse.ts`: `SCORING_HEAD_TRIGGER_THRESHOLD = 0.3` triggers a full-body re-score when the head pass scores below that (closes the bug class where a stray head match at 0.1 — e.g. blockquote that accidentally matches an unrelated pattern — suppressed the fallback entirely); `SCORING_MIN_ACCEPTANCE = 0.05` blocks essay false-positives (a 300-line essay with one stray chat-shape line scores ~0.003, below the floor, stays `no_match` instead of returning a 1-message regex_match). Both gates pinned by 11 new unit cases in `test/conversation-parser/parse.test.ts`. New exported `scorePatternFull(body, entry)` for direct full-body scoring; private `getNonBlankLines(body, headCap?)` + `scoreFromLines(lines, entry)` helpers DRY the quick_reject+regex loop. Empirical against all Circleback meetings in `~/git/brain/meetings/`: 113 of 367 parse correctly post-fix (was 0); 20,167 messages flow through to the fact extractor (was 0). The other 254 are notes-only meetings whose transcripts live in separate files. Plan + Codex absorption at `~/.claude/plans/system-instruction-you-are-working-starry-frost.md`. Modules: `types.ts` (PatternEntry + ParseResult + DateContext + CaptureMap + TimezonePolicy), `builtins.ts` (13 hand-vetted patterns sourced from public format docs — iMessage/Slack, Telegram bracket + text-export, bold-paren-time, Discord classic + export, WhatsApp ISO + US, Signal, Matrix/Element, IRC classic + weechat, Teams export; module-load validation runs every `test_positive[]` + `test_negative[]` sample at startup, so a typo in any built-in regex makes gbrain refuse to start; `DEFAULT_SPEAKER_CLEAN` exported as the PR #1461 helper promoted to a module-level default), `parse.ts` (orchestrator with D18 pattern-priority scoring across the first 10 lines + D8 date derivation chain `explicit > frontmatter.date > effective_date > '1970-01-01'` + D5 multi-line continuation + D19 timezone warning), `llm-base.ts` (shared `runLlmCall<T>` with content-hash cache in-process + DB-persistent via migration v97 + 4-strategy JSON repair + Anthropic-key probe; both polish and fallback are thin wrappers per D6 DRY), `llm-polish.ts` (per D15 opt-IN; headroom guard skips when tracker within $0.10 of cap; pure `applyPolish` for merge/drop/edit ops), `llm-fallback.ts` (per D15 opt-IN; per D17 NO regex inference + NO persistence — silent corruption avoided), `eval.ts` (`scoreFixture` + `aggregateScores` + `parseFixtureJsonl` for the fixture-corpus CI gate), `nightly-probe.ts` (DI-stubbed; mode-gated default per D10 — tokenmax=ON, conservative/balanced opt-in; adversarial false-positive detection). CLI surfaces at `src/commands/eval-conversation-parser.ts` (`gbrain eval conversation-parser <fixture.jsonl>` exit 0/1/2 — wired into `bun run verify` via `check:conversation-parser`) and `src/commands/conversation-parser.ts` (`scan <slug>` for debug, `list-builtins` for operator discoverability, `validate <file>` emits "deferred to v0.42+" notice). Doctor checks: `conversation_format_coverage` (per-pattern hit count + unmatched %), `progressive_batch_audit_health` (abort-verdict counts from last 7d), `conversation_parser_probe_health` (opt-in nightly probe state). Closes PR #1461 — contributor's `BRACKET_TIME_RX` + `cleanSpeaker` survive verbatim as the `telegram-bracket` built-in + `DEFAULT_SPEAKER_CLEAN` export, Co-Authored-By preserved. Plan + 23 decisions + codex outside-voice absorption at `~/.claude/plans/system-instruction-you-are-working-cuddly-hollerith.md`. Pinned by 74 unit cases across `test/conversation-parser/{parse,llm-base,llm-fallback,llm-polish,nightly-probe}.test.ts` + the 27-case PR #1461 baseline at `test/extract-conversation-facts.test.ts` (back-compat invariant). Migration v97 (`conversation_parser_llm_cache_table`). Fixtures at `test/fixtures/conversation-formats/{imessage,telegram-bracket,whatsapp-iso,whatsapp-us,signal-export,irc-classic,irc-weechat,matrix-element,teams-export,all,adversarial}.jsonl` with `scripts/check-fixture-privacy.sh` banning real-name leaks.
 - `src/core/progressive-batch/` (v0.41.16.0) — shared ramp-up + cost-cap + verification primitive extracted after rule-of-three was comfortably past (12+ ad-hoc cost-prompt patterns scattered across reindex, reindex-multimodal, book-mirror, brainstorm, eval-suspected-contradictions, post-upgrade-reembed, eval-contradictions/cost-prompt). Wintermute-inspired ramp shape (trial 10 → ramp 100 → ramp 500 → full with verification at each stage), productionized as a runtime primitive with verifier+policy injection (callers describe HOW TO MEASURE SUCCESS, not WHEN TO WAIT FOR CTRL-C). Modules: `types.ts` (Stage, StageVerdict, AbortReason, discriminated `Verifier` union per D20: `OutputCountVerifier | IdempotentMutationVerifier | NoopVerifier`, Policy, StageReport), `orchestrator.ts` (`runProgressiveBatch(items, verifier, policy, runner)` — reads `getCurrentBudgetTracker()` ahead of `Policy.maxCostUsd` per D3 fail-closed; null both ways triggers `abort_cost_cap reason='no_budget_safety_net'`), `audit.ts` (ISO-week JSONL at `~/.gbrain/audit/progressive-batch-YYYY-Www.jsonl` via the shared `audit-writer` primitive), `stage-report.ts` (ASCII stage-report formatter for the default `Policy.onStageReport` handler). Env knobs: `GBRAIN_PROGRESSIVE_BATCH_DISABLED=1` (skip ramp + stderr warn), `GBRAIN_PROGRESSIVE_BATCH_AUTO=1` (skip Ctrl-C grace), `GBRAIN_PROGRESSIVE_BATCH_STAGES=10,100,500` (override). v0.41.16.0 ships with ONE proven consumer (the parser cathedral via extract-conversation-facts retrofit); the 9-site retrofit of `reindex.ts`, `reindex-multimodal.ts`, `reindex-code.ts`, `post-upgrade-reembed.ts`, `book-mirror.ts`, `brainstorm/orchestrator.ts`, `eval-suspected-contradictions.ts`, `eval-contradictions/cost-prompt.ts`, `contextual-reindex-per-chunk.ts` lands as the v0.41.14.0+ follow-up wave per D2's bisectability requirement (each retrofit gets its own commit; behavior parity tested before/after migration). Per D21: sites that previously "jumped straight to full" keep doing so by default; ramp is opt-in per-site via `Policy.interactiveAbortMs > 0`. Pinned by 35 unit cases in `test/progressive-batch/orchestrator.test.ts` covering every verdict path.
 - `src/commands/extract-conversation-facts.ts` + `src/core/cycle/conversation-facts-backfill.ts` (v0.41.11.0) — bulk fact extraction for long-form conversation pages. The recall-miss class on long iMessage/Slack imports (60K+ msg history; a chunk that reads only "Locker 93 code 9494" has no topical anchor because "cabin" was established 50K messages earlier) gets fixed by walking conversation/meeting/slack/email pages, splitting into time-windowed segments (30-min gap or 30-msg cap), prepending a topical/temporal header ("Conversation between A and B from <iso1> to <iso2>"), and running through the existing `extractFactsFromTurn()` so the resulting anchor-rich facts surface in `gbrain search` (which already blends facts). Architecture decisions absorbed from 2 Codex outside-voice rounds + 2 eng review passes: **strict per-source core** (`runExtractConversationFactsCore({sourceId, ...})` always takes one sourceId; CLI + cycle phase each do their own multi-source iteration because PHASE_SCOPE='source' is taxonomy-only per cycle.ts:131); **two-phase enumeration** (paginated `listPages({type, sourceId, limit:10})` so worst case is 10×25MB=250MB resident vs PR #1406's unbounded; per-page body cap MAX_PAGE_BODY_BYTES=25MB with `pages_skipped_too_large` counter surfaced in doctor); **page-global row_num accumulator** (facts unique index is `(source_id, source_markdown_slug, row_num)` per migration v51 — per-segment row_num would collide on segment 2); **page-level TERMINAL audit row** written to facts table after all segments commit (source='cli:extract-conversation-facts:terminal'; doctor's NOT EXISTS query matches the terminal row, NOT any-fact-for-slug, so partial-extraction pages stay in backlog); **optional `opts.budgetTracker?`** (when present, core uses it as-is — nested `withBudgetTracker` REPLACES per gateway.ts:2011; when absent, core auto-wraps with `BudgetTracker({maxCostUsd})`); **body read covers compiled_truth + timeline** (iMessage importers put history in timeline half; PR's `compiled_truth ?? ''` would silently drop it); **honors `facts.extraction_enabled` kill-switch** with `--override-disabled` escape; **--types LIST allowlist** (`conversation,meeting,slack,email`) with CLI default reading `cycle.conversation_facts_backfill.types` config as single source of truth; **fingerprint on sourceId only** so widening types config doesn't invalidate completed-page state; **string-encoded op-checkpoint** entries `"<sourceId>|<slug>|<endIso>"` for resume only (op_checkpoints stores string[] and is 7-day GC'd — durable audit is the facts table itself via terminal row); **`--background` via maybeBackground** (Minion handler `extract-conversation-facts` re-creates BudgetTracker from `data.max_cost_usd`; on `BudgetExhausted` mid-job, catches + persists + marks `completed` with `result.budget_exhausted=true` — NOT a failure). The companion cycle phase `conversation_facts_backfill` (default OFF; `gbrain config set cycle.conversation_facts_backfill.enabled true` opts in) iterates `listSources(engine)` directly, creates ONE brain-wide tracker per tick + wraps the loop in `withBudgetTracker` + passes the tracker into every per-source invocation via `opts.budgetTracker` so the core doesn't nest-replace. Two-layer cost AND walltime protection: per-source caps (`max_cost_usd=$1`, `max_walltime_min=20`) AND brain-wide caps (`max_total_cost_usd=$5`, `max_total_walltime_min=30`). Pinned by 27 unit cases in `test/extract-conversation-facts.test.ts` (parse/segment/render/checkpoint/fingerprint/terminal-audit/row_num/F2-kill-switch). Schema migration v94 in `src/core/migrate.ts` adds a partial index `idx_facts_extract_conversation_session ON facts(source_id, source_session) WHERE source LIKE 'cli:extract-conversation-facts%'` (v14 precedent: `transaction:false` + invalid-index pre-drop on Postgres; plain CREATE INDEX on PGLite) so the doctor backlog query stays fast on million-fact brains. `src/commands/doctor.ts:computeConversationFactsBacklogCheck` is 3-state (SKIPPED when `cycle.conversation_facts_backfill.enabled=false` — no opt-out noise debt per Eng-v2 C9; OK when caught up; WARN when >10 pages lack the terminal audit row, with paste-ready remediation step composable via `gbrain doctor --remediate`). `src/commands/sources.ts:runAudit` extended with `facts_backfill_estimate: {pages, est_segments, est_cost_usd, types}` so `gbrain sources audit <id>` previews cost before any run. Schema-pack (v0.41.11.0): `gbrain-base.yaml` promotes `conversation` (temporal, extractable) + `atom` (annotation, NOT extractable — atoms ARE the extracted form) into the base seed; `concept.extractable` flips true semantically (cosmetic on backstop path because backstop uses hardcoded `ELIGIBLE_TYPES` in `src/core/facts/eligibility.ts:51`, not pack extractable — the original concept-grandfather migration was solving a phantom, dropped per Codex T3). `ALL_PAGE_TYPES` in `src/core/types.ts` extended with the two new types. Plan + all decisions persisted at `~/.claude/plans/system-instruction-you-are-working-linear-unicorn.md` and the CEO plan at `~/.gstack/projects/garrytan-gbrain/ceo-plans/2026-05-25-conversation-retrieval-upgrade.md`.
-- `src/commands/extract.ts` — `gbrain extract links|timeline|all [--source fs|db] [--source-id <id>]`: batch link/timeline extraction. fs walks markdown files, db walks pages from the engine (mutation-immune snapshot iteration; use this for live brains with no local checkout). As of v0.12.1 there is no in-memory dedup pre-load — candidates are buffered 100 at a time and flushed via `addLinksBatch` / `addTimelineEntriesBatch`; `ON CONFLICT DO NOTHING` enforces uniqueness at the DB layer, and the `created` counter returns real rows inserted (truthful on re-runs). v0.22.1 (#417): `ExtractOpts.slugs?: string[]` enables incremental extract — when set, `extractForSlugs()` reads ONLY those slugs' files (single combined links+timeline pass) instead of the full directory walk. CLI `gbrain extract` keeps full-walk behavior; the cycle path threads sync's `pagesAffected` through. `walkMarkdownFiles(brainDir)` still runs at line 455 to build `allSlugs` for link resolution — see `TODOS.md` for replacing it with `engine.getAllSlugs()`. **v0.37.7.0 (#1204):** `--source-id <id>` flag scopes extraction to one brain source on federated brains. Resolved via `resolveSourceWithTier()` before any SQL runs; failures surface with a `gbrain sources list` hint. Closes the silent-collapse-to-`default` bug class for extract. **v0.42.7.0 (#1696) — link-extraction freshness watermark + `extract --stale`:** the "imported ≠ curated" wave. New `gbrain extract --stale [--source-id <id>] [--catch-up] [--dry-run] [--json]` branch (`extractStaleFromDB`) — incremental DB-source link+timeline sweep over pages whose `pages.links_extracted_at` watermark is stale. Stale predicate (shared by both engines + the doctor check): `links_extracted_at IS NULL OR links_extracted_at < LINK_EXTRACTOR_VERSION_TS::timestamptz OR updated_at > links_extracted_at` — the `updated_at` arm catches the MCP `put_page` / `sync --no-extract` edited-since-extract path. Three new `BrainEngine` methods (parity in postgres-engine.ts + pglite-engine.ts + bootstrap probes): `countStalePagesForExtraction(opts?)`, `listStalePagesForExtraction({batchSize, afterPageId?, sourceId?, versionTs?})` (returns page CONTENT so the sweep avoids an N+1 `getPage`; `rowToStalePage` in utils.ts maps the row, `StalePageRow` in types.ts), and `markPagesExtractedBatch(refs, defaultExtractedAt)` (3-array unnest `slug[],source_id[],ts[]`; each ref may carry its own `extractedAt`). `STALE_BATCH_SIZE` default 25 (`GBRAIN_EXTRACT_STALE_BATCH`) — small because page bodies are unbounded (the LIMIT is the only fetch-time memory bound); `STALE_TIME_BUDGET_MS` 30min wall-clock (`--catch-up` removes the cap). CDX-4 non-swallowing flush: link/timeline flush throws propagate and abort the batch; stamp LAST so a crash leaves pages unstamped and they re-extract idempotently (`addLinksBatch` ON CONFLICT DO NOTHING + timeline dedup). D4 race fix: `extractStaleFromDB` stamps with each row's READ `updated_at` (not `now()`), so a concurrent edit during the sweep advances `updated_at` past the stamp → page stays stale → re-extracted next run rather than marked fresh-with-old-content. Source-correct stamping at the DB-extract sites via `stampExtracted` (best-effort wrapper, never throws); `extractLinksFromDB` only stamps the combined watermark when `subcommand === 'all'` (both links AND timeline ran — a links-only run must not hide timeline staleness, CDX C3). `LINK_EXTRACTOR_VERSION_TS` lives in `src/core/link-extraction.ts` (bump like `CHUNKER_VERSION` when the extractor shape changes → invalidates all prior stamps). Migration v112 (`pages_links_extracted_at`) adds the nullable `TIMESTAMPTZ` column + composite `(source_id, links_extracted_at)` index (CONCURRENTLY + invalid-remnant pre-drop on Postgres, plain on PGLite), NO backfill so the real backlog surfaces on first `gbrain doctor`. Schema parity in schema.sql + regenerated pglite-schema.ts + schema-embedded.ts + `REQUIRED_BOOTSTRAP_COVERAGE`. `src/commands/doctor.ts:checkLinksExtractionLag` (the `links_extraction_lag` check, also in `doctorReportRemote`) is warn-only by default (>`GBRAIN_EXTRACTION_LAG_WARN_PCT`, default 20%; shared `EXTRACTION_LAG_WARN_PCT_DEFAULT` + `EXTRACTION_LAG_MIN_PAGES=100` + exported `_resolveEnvNumber`), hard-fails only when `GBRAIN_EXTRACTION_LAG_FAIL_PCT` is set; vacuous-skips <100 pages (no `--source`); pre-v112 brains graceful-skip via `isUndefinedColumnError`; strictly a SQL COUNT (safe on the remote/thin-client path). `src/commands/sync.ts` gains `--no-extract` (threaded through single-source + `--all` + `syncOneSource`), stamps `links_extracted_at` for `pagesAffected` at the inline-extract call site (after `extractLinksForSlugs`/`extractTimelineForSlugs`, CDX-6), and `maybeExtractionNudge` prints a one-line stderr nudge after a `synced | first_sync | up_to_date` sync that leaves a backlog (the `first_sync` case is the initial-import scenario, D5; `shouldNudgeAfterSync` pure predicate; `GBRAIN_SYNC_NO_EXTRACT_NUDGE` suppresses; shares the doctor vacuous-skip predicate). `src/core/retry.ts` adds `'extract.stale'` to `BATCH_AUDIT_SITES`; `src/core/doctor-categories.ts` adds `links_extraction_lag` to `BRAIN_CHECK_NAMES`. Pinned by `test/extract-stale.test.ts` (incl. CDX-1 edited-after-stamp regression + crash-contract), `test/sync-inline-extract-stamps.serial.test.ts` (IRON-RULE), `test/sync-nudge-status-gate.test.ts`, `test/doctor-links-extraction-lag.test.ts`, engine-parity (Postgres↔PGLite) for the 3 methods + v112 round-trip. Plan: `~/.claude/plans/system-instruction-you-are-working-squishy-crayon.md`.
+- `src/commands/extract.ts` — `gbrain extract links|timeline|all [--source fs|db] [--source-id <id>]`: batch link/timeline extraction. fs walks markdown files, db walks pages from the engine (mutation-immune snapshot iteration; use this for live brains with no local checkout). As of v0.12.1 there is no in-memory dedup pre-load — candidates are buffered 100 at a time and flushed via `addLinksBatch` / `addTimelineEntriesBatch`; `ON CONFLICT DO NOTHING` enforces uniqueness at the DB layer, and the `created` counter returns real rows inserted (truthful on re-runs). v0.22.1 (#417): `ExtractOpts.slugs?: string[]` enables incremental extract — when set, `extractForSlugs()` reads ONLY those slugs' files (single combined links+timeline pass) instead of the full directory walk. CLI `gbrain extract` keeps full-walk behavior; the cycle path threads sync's `pagesAffected` through. `walkMarkdownFiles(brainDir)` still runs at line 455 to build `allSlugs` for link resolution — see `TODOS.md` for replacing it with `engine.getAllSlugs()`. **v0.37.7.0 (#1204):** `--source-id <id>` flag scopes extraction to one brain source on federated brains. Resolved via `resolveSourceWithTier()` before any SQL runs; failures surface with a `gbrain sources list` hint. Closes the silent-collapse-to-`default` bug class for extract. **v0.42.7.0 (#1696) — link-extraction freshness watermark + `extract --stale`:** the "imported ≠ curated" wave. New `gbrain extract --stale [--source-id <id>] [--catch-up] [--dry-run] [--json]` branch (`extractStaleFromDB`) — incremental DB-source link+timeline sweep over pages whose `pages.links_extracted_at` watermark is stale. Stale predicate (shared by both engines + the doctor check): `links_extracted_at IS NULL OR links_extracted_at < LINK_EXTRACTOR_VERSION_TS::timestamptz OR updated_at > links_extracted_at` — the `updated_at` arm catches the MCP `put_page` / `sync --no-extract` edited-since-extract path. Three new `BrainEngine` methods (parity in postgres-engine.ts + pglite-engine.ts + bootstrap probes): `countStalePagesForExtraction(opts?)`, `listStalePagesForExtraction({batchSize, afterPageId?, sourceId?, versionTs?})` (returns page CONTENT so the sweep avoids an N+1 `getPage`; `rowToStalePage` in utils.ts maps the row, `StalePageRow` in types.ts), and `markPagesExtractedBatch(refs, defaultExtractedAt)` (3-array unnest `slug[],source_id[],ts[]`; each ref may carry its own `extractedAt`). `STALE_BATCH_SIZE` default 25 (`GBRAIN_EXTRACT_STALE_BATCH`) — small because page bodies are unbounded (the LIMIT is the only fetch-time memory bound); `STALE_TIME_BUDGET_MS` 30min wall-clock (`--catch-up` removes the cap). CDX-4 non-swallowing flush: link/timeline flush throws propagate and abort the batch; stamp LAST so a crash leaves pages unstamped and they re-extract idempotently (`addLinksBatch` ON CONFLICT DO NOTHING + timeline dedup). D4 race fix: `extractStaleFromDB` stamps with each row's READ `updated_at` (not `now()`), so a concurrent edit during the sweep advances `updated_at` past the stamp → page stays stale → re-extracted next run rather than marked fresh-with-old-content. Source-correct stamping at the DB-extract sites via `stampExtracted` (best-effort wrapper, never throws); `extractLinksFromDB` only stamps the combined watermark when `subcommand === 'all'` (both links AND timeline ran — a links-only run must not hide timeline staleness, CDX C3). `LINK_EXTRACTOR_VERSION_TS` lives in `src/core/link-extraction.ts` (bump like `CHUNKER_VERSION` when the extractor shape changes → invalidates all prior stamps). Migration v112 (`pages_links_extracted_at`) adds the nullable `TIMESTAMPTZ` column + composite `(source_id, links_extracted_at)` index (CONCURRENTLY + invalid-remnant pre-drop on Postgres, plain on PGLite), NO backfill so the real backlog surfaces on first `gbrain doctor`. Schema parity in schema.sql + regenerated pglite-schema.ts + schema-embedded.ts + `REQUIRED_BOOTSTRAP_COVERAGE`. `src/commands/doctor.ts:checkLinksExtractionLag` (the `links_extraction_lag` check, also in `doctorReportRemote`) is warn-only by default (>`GBRAIN_EXTRACTION_LAG_WARN_PCT`, default 20%; shared `EXTRACTION_LAG_WARN_PCT_DEFAULT` + `EXTRACTION_LAG_MIN_PAGES=100` + exported `_resolveEnvNumber`), hard-fails only when `GBRAIN_EXTRACTION_LAG_FAIL_PCT` is set; vacuous-skips <100 pages (no `--source`); pre-v112 brains graceful-skip via `isUndefinedColumnError`; strictly a SQL COUNT (safe on the remote/thin-client path). `src/commands/sync.ts` gains `--no-extract` (threaded through single-source + `--all` + `syncOneSource`), stamps `links_extracted_at` for `pagesAffected` at the inline-extract call site (after `extractLinksForSlugs`/`extractTimelineForSlugs`, CDX-6), and `maybeExtractionNudge` prints a one-line stderr nudge after a `synced | first_sync | up_to_date` sync that leaves a backlog (the `first_sync` case is the initial-import scenario, D5; `shouldNudgeAfterSync` pure predicate; `GBRAIN_SYNC_NO_EXTRACT_NUDGE` suppresses; shares the doctor vacuous-skip predicate). `src/core/retry.ts` adds `'extract.stale'` to `BATCH_AUDIT_SITES`; `src/core/doctor-categories.ts` adds `links_extraction_lag` to `BRAIN_CHECK_NAMES`. Pinned by `test/extract-stale.test.ts` (incl. CDX-1 edited-after-stamp regression + crash-contract), `test/sync-inline-extract-stamps.serial.test.ts` (IRON-RULE), `test/sync-nudge-status-gate.test.ts`, `test/doctor-links-extraction-lag.test.ts`, engine-parity (Postgres↔PGLite) for the 3 methods + v112 round-trip. Plan: `~/.claude/plans/system-instruction-you-are-working-squishy-crayon.md`. **v0.42.11.0 (#1768) — `links_extraction_lag` never cleared on Postgres:** the D4 race-fix stamped `page.updated_at.toISOString()`, but `StalePageRow.updated_at` is a JS `Date` (millisecond-truncated) while the DB `updated_at` keeps microseconds — so `updated_at > links_extracted_at` stayed permanently true and the lag was stuck at 100% (the remediation the check recommends, `extract --stale`, could never satisfy its own check). NOT a trigger (there's no `BEFORE UPDATE` trigger on `pages` that sets `updated_at`). Fix preserves the stamp contract + D4: both engines' `listStalePagesForExtraction` SELECT now projects a deterministic full-µs UTC string `to_char(updated_at AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS.US"Z"') AS updated_at_iso` (NOT raw `::text`, which is `DateStyle`-fragile — Codex outside-voice catch); `StalePageRow` gains `updated_at_iso: string`; `rowToStalePage` (utils.ts) populates it (ISO-only fallback, never `String(Date)`); `extractStaleFromDB` stamps `page.updated_at_iso` instead of the truncated Date. `markPagesExtractedBatch` SQL is UNCHANGED, so backdated stamps (the version-arm test) still work and the CDX-1 edited-since arm is strengthened to exact equality. Pinned by a new deterministic PGLite regression in `test/extract-stale.test.ts` (inject a µs `updated_at`, run `--stale`, assert lag → 0 and stays 0) + engine-parity.
 - `src/core/extract/receipt-writer.ts` + `src/core/extract/rollup-writer.ts` + `src/commands/extract-status.ts` + `src/commands/extract-explain.ts` + `src/commands/extract-benchmark.ts` + `src/core/schema-pack/scaffold-extractable.ts` (v0.41.23.0) — unified extract operator-surface wave. Every shipped extractor (deterministic `facts.conversation` in `src/commands/extract-conversation-facts.ts` + three LLM-backed cycle phases at `src/core/cycle/{extract-atoms,synthesize-concepts,propose-takes,extract-facts}.ts`) now writes ONE receipt page per run (`writeReceipt`) + UPSERTs a row to `extract_rollup_7d` (`upsertExtractRollup`). Receipt slug per D-EXTRACT-17: `extracts/{date}/{kind}/{source_id}/{run_id_short}/round-{N}.md`; frontmatter stamps BOTH `type: extract_receipt` AND `dream_generated: true` per D-EXTRACT-19 (belt+suspenders against extraction-loop guard drift). `extract_receipt` joins `ALL_PAGE_TYPES` in `src/core/types.ts`; `extracts/` prefix gets a 0.3x source-boost demote in `src/core/search/source-boost.ts` so receipts surface in search but never dominate user content. Migration v104 adds `extract_rollup_7d (kind, source_id, day, cost_usd, halt_count, eval_pass_count, eval_fail_count, round_completed_count, rollup_write_failures, updated_at)` with PK `(kind, source_id, day)` + `idx_extract_rollup_7d_day`. Rollup writes are best-effort with process-scoped error-dedup so transient DB failures bump `rollup_write_failures` instead of crashing the cycle. New `extract_health` doctor check reads the rollup for last 7 days, warns at halt-rate > 10% AND when rollup_write_failures > 0; pre-v104 brains report `ok` silently. Operator CLI surface: `gbrain extract status [--source-id ID] [--kind X] [--verbose] [--json]` (7-day rollup, sorted by halt_rate desc + cost desc, kubectl-style top-5 + "more rows" hint, stable `schema_version: 1` JSON envelope); `gbrain extract --explain <kind>` (resolution chain: pack-declared vs built-in cycle phase, prompt_template + fixture_corpus paths with `✓`/`(missing)` markers, last 7d rollup); `gbrain extract benchmark --pack X --kind Y` (loads pack fixture corpus through strict D-EXTRACT-21 path validation — rejects absolute paths, `..` traversal, null bytes, AND symlinks resolving outside pack root; v0.41.23 ships as a stub-reporter — LLM dispatch deferred to a follow-up release). Pack-author authoring loop: `src/core/schema-pack/manifest-v1.ts` widens `extractable` from `z.boolean()` to `z.union([z.boolean(), ExtractableSpecSchema])` (carries `prompt_template`, `fixture_corpus`, `eval_dimensions`, `benchmark_min_recall`, plus reserved `verifier_path` per D-EXTRACT-37 — parses but refuses at runtime in v0.41.23); new `extractableSpecsFromPack` + `getExtractableSpec` + `refuseVerifierPathInV042` helpers in `src/core/schema-pack/extractable.ts`; `gbrain schema scaffold-extractable <type> --pack <pack>` declares the type extractable in one verb, generates 5 placeholder fixtures + a prompt template stub under `packs/<pack>/{fixtures,prompts}/extract/`, refuses to overwrite without `--force`. Tests: 22 cases in `test/extractable-spec-widening.test.ts` (back-compat boolean shape + struct shape + D-EXTRACT-37 verifier_path refuse), 12 cases in `test/extract/receipt-writer.test.ts` (uses canonical PGLite block per CLAUDE.md R3+R4 — one engine per file with `resetPgliteState` in `beforeEach`), 17 cases in `test/extract/benchmark.test.ts` (path validation rejections + JSONL contract), 15 cases in `test/extract/status.test.ts` (pure aggregation + formatting), 15 cases in `test/schema-pack/scaffold-extractable.test.ts` (scaffold mechanics + explicit privacy-rule guards against real-name leakage), 8 cases in `test/doctor-extract-health.test.ts`. Plan persisted at `~/.claude/plans/system-instruction-you-are-working-stateless-dragonfly.md`. Deferred to a follow-up release: replay versioning + `gbrain extract replay --since v<sha>`; unified LLM dispatcher (deliberately respects v0.41.13 head comment that extraction is regex + cost-cap value-add lives at embed step).
 - `src/commands/import.ts` — `gbrain import <path> [--source-id <id>]`: page import with the v0.34.2.0 path-set checkpoint described above. **v0.37.7.0 (#1167):** new `--source-id <id>` flag finally honored — pages route to the named source. Resolved via `resolveSourceWithTier()` at the boundary; the same flag is now consistent across `import`, `extract`, `graph-query`, and `sources current`. Pinned by `test/import-source-id.test.ts`.
 - `src/commands/graph-query.ts` — `gbrain graph-query <slug> [--type T] [--depth N] [--direction in|out|both] [--include-foreign]`: typed-edge relationship traversal (renders indented tree). **v0.37.7.0 (#1153):** foreign-edge footer always present (`X foreign edges (use --include-foreign to traverse)`) so cross-source edges never disappear silently; `--include-foreign` widens the SQL filter to walk them. Pinned by `test/graph-query.test.ts`.
@@ -442,7 +443,7 @@ strict behavior when unset.
 - `src/core/embedding-dim-check.ts` extension (v0.41.17.0, T5+T6) — facts.embedding dim drift surface. New `readFactsEmbeddingDim(engine): Promise<FactsColumnDimResult>` covers both `vector(N)` and `halfvec(N)` shapes (codex #19 — migration v40 falls back to `vector` on pgvector < 0.7). Regex ordering halfvec-before-vector pinned by tests (substring "vec" appears in "halfvec"; naive `/vector/i` would shadow). New `buildFactsAlterRecipe(dims, configured, type)` emits the paste-ready `DROP INDEX IF EXISTS idx_facts_embedding_hnsw; ALTER TABLE facts ALTER COLUMN embedding TYPE halfvec(N) USING embedding::halfvec(N); CREATE INDEX idx_facts_embedding_hnsw ON facts USING hnsw (embedding halfvec_cosine_ops) WHERE ...` flow (codex #18 — NOT bare REINDEX which doesn't rewrite the index after a column-type change). New `assertFactsEmbeddingDimMatchesConfig(engine)` is the D15 preflight — throws `FactsEmbeddingDimMismatchError` (tagged class with `tag: 'FACTS_EMBEDDING_DIM_MISMATCH'` for parity with the worker-pool's MUST_ABORT semantics) when configured dim doesn't match the column width. Result cached per-engine via `WeakMap`. PGLite engines silently skip. New doctor check `facts_embedding_width_consistency` (registered in `runDoctor` after `embedding_width_consistency`) reuses the same helpers — surfaces drift with paste-ready ALTER recipe identical to the preflight error. Pinned by 18 cases in `test/embedding-dim-check-facts.test.ts`.
 - `src/core/postgres-engine.ts` extension (v0.41.17.0, T6, codex #20) — `insertFact` + `insertFacts` no longer hardcode `tx.unsafe(\`'${embedLit}'::vector\`)` for the embedding column. New `resolveFactsEmbeddingCast()` private method probes `pg_attribute` once per engine instance (cached in `_factsEmbeddingCastSuffix`) and returns `'::halfvec'` when migration v40 created the column as halfvec, else `'::vector'`. Both insert paths use the cached suffix so the cast matches the actual column type. Pre-fix all three insert sites hardcoded `::vector`; works on pgvector >= 0.7 via implicit auto-cast but fails on older pgvector. Test seam `__resetFactsEmbeddingCastCacheForTest()` clears per-engine cache.
 - `src/core/cycle.ts` + `src/core/cycle/extract-atoms.ts` + `src/core/cycle/synthesize-concepts.ts` + `src/commands/extract.ts` + `src/commands/doctor.ts` + `src/core/op-checkpoint.ts` extensions (v0.41.20.0 ops-fix-wave) — six daily-driver ops pains in one bisectable wave. (1) **Batch idempotency for `extract_atoms`:** new `atomsExistingForHashes(engine, sourceId, hashes[])` exported from `src/core/cycle/extract-atoms.ts` replaces the per-hash loop that did 7K individual queries at the start of every cycle on brains with conversation-transcript corpora (5-10 min silent overhead). One batched SQL roundtrip returns the set of `content_hash16` values already extracted as atoms for this source. Fail-open: SQL error logs to stderr and returns an empty set so extraction proceeds. Powered by migration v104 `pages_atom_source_hash_idx` (partial expression index on `frontmatter->>'source_hash'` for atom rows where `deleted_at IS NULL`; Postgres uses `CREATE INDEX CONCURRENTLY` with invalid-remnant pre-drop mirroring v97 `pages_dedup_partial_index`, PGLite uses plain `CREATE INDEX`). Without this index the batch idempotency check would seq-scan pages on big brains and defeat the perf win. (2) **Shorter cycle lock TTL + active in-phase refresh:** `LOCK_TTL_MINUTES` dropped from 30 → 5 in `src/core/cycle.ts`. New exported `buildYieldDuringPhase(lock, outer)` closure calls `lock.refresh()` AND any external hook on every fire; throttled to 30s inside each phase via `maybeYield`. Fires both inside the main work loop AND immediately after every `await chat(...)` LLM call so long Haiku/Sonnet calls don't sit past TTL. `LockHandle` and `buildYieldDuringPhase` exported for test seam access. `synthesize_concepts` no longer fires `yieldDuringPhase` per-concept-group — same hook, throttled via the new shared `maybeYield` helper. A crashed cycle releases its lock 6x faster while a healthy long-running cycle keeps the lock alive. Known residual under shorter TTL: a single `await chat()` past 5 min wallclock can expire the lock mid-await without the original phase noticing — `DbLockHandle.refresh()` throwing on 0 rows affected is filed as P2 follow-up TODO-OPS-2. (3) **Progress wiring through long phases:** new `progress?: ProgressReporter` opt on `ExtractAtomsOpts` and `SynthesizeConceptsOpts`. Cycle.ts passes its phase-level reporter down (NOT a child reporter — that would produce a path collision `cycle.extract_atoms.extract_atoms.work`). Phases only call `tick()` and `heartbeat()`; cycle.ts owns `start()` and `finish()`. You see `[cycle.extract_atoms] N (atoms_created)` ticks every ~1s during both long phases instead of "start" then silence for 10+ min. (4) **`by-mention` resume:** new `mentionsFingerprint({source, type, since, gazetteerHash})` in `src/core/op-checkpoint.ts`. The gazetteer hash is the load-bearing field — adding new entity pages mid-pause shifts the hash, gets a new fingerprint, and triggers a fresh scan against the new gazetteer instead of silently skipping previously-scanned pages. `gbrain extract links --by-mention` now resumes from where it died via the existing `op_checkpoints` framework with a `flushAndCheckpoint` ordering — links flush to the DB FIRST, page keys commit to the checkpoint SECOND, persist THIRD. A crash between `batch.push()` and flush leaves the page un-checkpointed so resume re-scans it. Persist cadence: every 1000 items OR every 30s, whichever first. Clean exit clears the checkpoint. `--dry-run` deliberately skips both load and write so it stays an inspection mode. (5) **`sync_consolidation` doctor check:** multi-source brains see a paste-ready `gbrain sync --all --parallel 4 --workers 4 --skip-failed` recommendation. Single-source brains get "not applicable." SQL errors return `warn` via the check's own try/catch — outer doctor catch isn't a safe assumption. Companion "Multi-source brains" recipe block in `skills/cron-scheduler/SKILL.md` documents the `sync --all` pattern as preferred over per-source entries. (6) **Test-isolation fixes:** `test/cycle-last-full-cycle-at.test.ts` + `test/schema-cli.test.ts` migrated to per-test `GBRAIN_HOME=tempdir` isolation. Pinned by 44 new unit/PGLite cases across 9 files: `test/cycle/extract-atoms-batch.test.ts` (5 — batch idempotency), `test/cycle/cycle-lock-ttl.test.ts` (1 — regression pin on `LOCK_TTL_MINUTES === 5`), `test/op-checkpoint-mentions-fingerprint.test.ts` (7 — fingerprint sensitivity including gazetteer-hash regression guard), `test/cycle/extract-atoms-progress.test.ts` (4 — phase doesn't call start/finish, ticks fire per item), `test/cycle/synthesize-concepts-progress.test.ts` (3 — same shape), `test/cycle/yield-during-phase-refresh.test.ts` (7 — buildYieldDuringPhase actually calls lock.refresh() + outer hook, throws non-fatal), `test/cycle/yield-during-phase-throttle.test.ts` (3 — 30s throttle gate), `test/extract-by-mention-resume.test.ts` (5 — checkpoint persistence ordering, dry-run skips persist, gazetteer change invalidates, filtered pages get checkpointed), `test/doctor-sync-consolidation.test.ts` (6 — edge case matrix for source counts + archived filtering + SQL error path). Two follow-up TODOs filed in TODOS.md under "v0.41.19.0 ops-fix-wave follow-ups": `gbrain sync print-cron` subcommand (TODO-OPS-1) and lock-loss detection via `DbLockHandle.refresh()` throwing on 0 rows affected (TODO-OPS-2).
-- `src/commands/sync.ts` — `gbrain sync` CLI + the `performSync` / `performFullSync` library entrypoints (consumed by the autopilot cycle and the Minion sync handler). v0.22.13 (PR #490): `performSync` wraps its body in a `gbrain-sync` writer lock so two concurrent syncs (manual + autopilot, two terminals, two Conductor workspaces) cannot both write `last_commit` and let the last writer win. Head-drift gate after the import phase re-checks `git rev-parse HEAD`; if HEAD moved (someone ran `git checkout` / `git pull` mid-sync), the bookmark refuses to advance. Vanished files now record a failedFiles entry instead of silent-skip — the silent-skip-then-advance pathology that survived prior hardening passes is dead. Worker engines wrap in try/finally so disconnect always fires (panic-path leak fix). Both PGLite-detection sites use `engine.kind === 'pglite'`. CLI accepts `--workers N` (alias `--concurrency N`), validated via `parseWorkers`. Explicit `--workers` bypasses the auto-path file-count floor; auto path defers to `autoConcurrency()`. Banner moved to stderr. **v0.34.2.0:** the inline `.sort()` over add/mod paths is replaced with `sortNewestFirst(addsAndMods)` from `src/core/sort-newest-first.ts`, so the newest-first descending-lex policy lives in one helper shared with `gbrain import` instead of drifting across two files. **v0.40.3.0 (productionized from PR #1314 by @garrytan-agents):** the load-bearing change is the lock-identity invariant — `performSync` now defaults to a per-source lock id (`gbrain-sync:<sourceId>`) whenever `opts.sourceId` is set, AND wraps the writer window in `withRefreshingLock` from `src/core/db-lock.ts` so long-running sources (the PR motivator: media-corpus / 250K+ chunks) don't lose their lock at the 30-min TTL mid-run. Legacy single-default-source brains keep the bare `tryAcquireDbLock(SYNC_LOCK_ID)` path for back-compat. `SyncOpts.lockId?: string` is the explicit override (escape hatch; production callers don't set it). The fix closes the bug class Codex's outside-voice review caught in the original PR — a `sync --all` worker on per-source lock racing against `sync --source foo` on global lock would have silently corrupted the same source. `gbrain sync --all` got a continuous worker pool: `parseWorkers`-validated `--parallel N` (default `min(sourceCount, --workers, DEFAULT_PARALLEL_SOURCES=4)` from `src/core/sync-concurrency.ts`), `parallel` long-lived async workers pulling from a shared FIFO queue (no head-of-line blocking from wave dispatch), per-source `withSourcePrefix(src.id, ...)` wrap from `src/core/console-prefix.ts` so every `slog`/`serr` line carries `[<source-id>] ` for kubectl-style greppable parallel output. `--skip-failed` and `--retry-failed` reject with a paste-ready error when combined with `--parallel > 1` (the brain-global `sync-failures.jsonl` has no per-source scope; v0.41+ TODO source-scopes it). Connection-budget stderr warning fires when `parallel × workers × 2 > 16` (the `× 2 per-file pool` factor: each per-file worker opens its own `PostgresEngine` with `poolSize=2`, so `--parallel 4 --workers 4` is actually 32 connections, not 16 — Codex caught the 2× understatement in the original plan). New exports `resolveParallelism`, `syncOneSource`, `buildSyncStatusReport`, `printSyncStatusReport`, `SyncStatusReport` back the `gbrain sources status` dashboard surface (D3 → `sources` subcommand, not a `sync` flag, so reads and writes don't share a verb). Stable `--json` envelope `{schema_version: 1, sources, parallel, ok_count, error_count}` on stdout under `--json`; human banners route to stderr via the `humanSink` helper so `jq` parses cleanly. Exit matrix: 0 = all ok, 1 = any error, 2 = cost-prompt-not-confirmed (unchanged). The dashboard SQL is the canonical `content_chunks ch JOIN pages pg ON pg.id = ch.page_id WHERE pg.deleted_at IS NULL` shape with `archived = false` filter at the caller — the original PR shipped `chunks ch JOIN ON page_slug`, which would have crashed on PGLite parse and silently zeroed on Postgres via the swallow-catch. Embedding column resolved via `resolveEmbeddingColumn(undefined, cfg)` from `src/core/search/embedding-column.ts` so Voyage / multimodal / non-default-column brains see counts against the column they actually use. Errors propagate from the dashboard SQL — no swallow-catch (Q2 sub-fix). The IRON RULE regression lives in `test/e2e/sync-status-pglite.test.ts`: real PGLite seeds 2 sources × pages × chunks, soft-deletes 1 page, archives 1 source, validates the SQL excludes both AND the active embedding column is the one used. 38 in-file `console.log`/`console.error` call sites inside `performSync*` migrated to `slog`/`serr` (top-level `runSync` orchestrator console calls intentionally stay outside the prefix scope). **v0.41.25.0 (supersedes PR #1538):** sync delete loop rewritten as interleaved per-batch resolve+delete using `engine.resolveSlugsByPaths` + `engine.deletePages` from `src/core/engine.ts`. 73K-delete commit drops from ~146K SQL round-trips (~5h on Postgres+pgbouncer) to ~292 round-trips (~2min) — closes the cascade-staleness bug class where one big-delete commit jammed every other source's sync for hours. Per-batch try-catch decomposes batch DELETE failures to per-slug `deletePage` fallback so a transient blip on batch 73 doesn't lose 500 deletes; unrecoverable per-slug failures land in `failedFiles` (matches the existing import-loop pattern). `pagesAffected` filters to D6 confirmed-deleted slugs (downstream extract/embed stop wasting work on phantoms). Same batched slug-resolve treatment for the rename loop (`gbrain sync` renames — Phase 3 in the plan; the per-file `importFile` stays per-file, only the slug-resolve N+1 gets batched). The `resolveSlugByPathOrSourcePath` single-call helper at `sync.ts:267` now delegates to `engine.resolveSlugsByPaths` when `sourceId` is set (D8 DRY — one owner of the SQL + fallback semantics), keeping legacy `executeRaw` fallback for the no-sourceId path which is functionally dead post-v0.34.1's source-resolution wiring. Legacy no-sourceId branch in the delete loop preserves correctness on a slow path; the engine batch methods require sourceId so the new fast path only fires when production threading is intact. `failedFiles` declaration hoisted to top of `performSyncInner` so both delete decompose and import loops feed the same sync-bookmark gate. **v0.41.31.0 (mode-aware cost gate):** the v0.20.0 unconditional `sync --all` ConfirmationRequired gate is superseded by a mode-aware gate driven by `willEmbedSynchronously` + `shouldBlockSync` from `src/core/embedding.ts`. federated_v2 is resolved ONCE up front (shared with the fan-out below). On the DEFERRED path (v2 on, parallel) embedding goes to per-source `embed-backfill` jobs that carry their own `$X/source/24h` cap (default $25, `SPEND_CAP_CONFIG_KEY` from `embed-backfill-submit.ts`), so the gate is INFORMATIONAL — it prints an FYI naming the backfill cap + current backlog (`sumStaleChunkChars({signature})` priced via `estimateCostFromChars` at `currentEmbeddingPricePerMTok()`) + the count of already-queued `embed-backfill` jobs (TODO-2 visibility), and NEVER exits 2. On the INLINE path (v2 off, or `--serial` without `--no-embed`) the BLOCKING gate fires only when the NEW-CONTENT estimate exceeds the `sync.cost_gate_min_usd` floor (default $0.50). The new-content estimate (`estimateInlineNewTokens`) contributes ZERO for sources provably unchanged since last sync (`isSourceUnchangedSinceSync` HEAD==last_commit + clean tree, AND `chunker_version === CURRENT`) — mirrors doctor's `sync_freshness` + sync's own do-work gate — and the full-tree ceiling for changed sources. F2 fix: the pre-existing stale backlog (NULL embeddings + signature drift) is shown informationally but NEVER gated on, because sync doesn't sweep it inline (`gbrain embed --stale` does); gating on it would block every inline cron after a model swap. New helpers `resolveCostGateFloorUsd(engine)` (reads `sync.cost_gate_min_usd`, fail-open $0.50, accepts 0) + `resolveBackfillCapUsd(engine)`. The `sync --all` SELECT widens to carry `last_commit + chunker_version` (both columns predate v0.41; no migration). JSON envelopes gain `mode` + `gate` discriminators (`dry_run | deferred_notice | below_floor | confirmation_required`). `SyncStatusReportSource` gains `backfill_queued` / `backfill_active` / `backfill_last_completed_at` (best-effort; 0/null on pre-minions brains). All embedding-cost preview math reads the configured model name via `getEmbeddingModelName()` (no hardcoded OpenAI). Pinned by `test/sync-cost-gate.serial.test.ts` + `test/sync-cost-preview.test.ts`. **v0.41.37.0 (#1569):** new `SyncOpts.noSchemaPack` (CLI `--no-schema-pack`, threaded through `performSync` AND `syncOneSource` so `sync --all` honors it) — skips `loadActivePack` entirely so no user-supplied pack page-type regex runs during import; pages fall back to legacy prefix typing. Escape hatch for completing a sync when a suspect pack regex is the suspected cause of a wedge (re-run extraction later). Same release adds a per-file BEGIN heartbeat: `if (process.env.GBRAIN_SYNC_TRACE) serr('[sync] begin import: <path>')` emitted BEFORE `importFile` (the existing `progress.tick` fires only AFTER `importFile` returns — useless when one file wedges). Off by default to avoid a line per file on huge brains; `serr` is source-prefix-aware so under `--workers >1` / `--all` the stuck file is the begin-line with no matching completion. Triage doc: `docs/architecture/serve-sync-concurrency.md` (PGLite single-writer serve↔sync contention — stop `gbrain serve` before a large PGLite sync — plus the `GBRAIN_SYNC_TRACE` + `--no-schema-pack` hang-triage recipes).
+- `src/commands/sync.ts` — `gbrain sync` CLI + the `performSync` / `performFullSync` library entrypoints (consumed by the autopilot cycle and the Minion sync handler). v0.22.13 (PR #490): `performSync` wraps its body in a `gbrain-sync` writer lock so two concurrent syncs (manual + autopilot, two terminals, two Conductor workspaces) cannot both write `last_commit` and let the last writer win. Head-drift gate after the import phase re-checks `git rev-parse HEAD`; if HEAD moved (someone ran `git checkout` / `git pull` mid-sync), the bookmark refuses to advance. Vanished files now record a failedFiles entry instead of silent-skip — the silent-skip-then-advance pathology that survived prior hardening passes is dead. Worker engines wrap in try/finally so disconnect always fires (panic-path leak fix). Both PGLite-detection sites use `engine.kind === 'pglite'`. CLI accepts `--workers N` (alias `--concurrency N`), validated via `parseWorkers`. Explicit `--workers` bypasses the auto-path file-count floor; auto path defers to `autoConcurrency()`. Banner moved to stderr. **v0.34.2.0:** the inline `.sort()` over add/mod paths is replaced with `sortNewestFirst(addsAndMods)` from `src/core/sort-newest-first.ts`, so the newest-first descending-lex policy lives in one helper shared with `gbrain import` instead of drifting across two files. **v0.40.3.0 (productionized from PR #1314 by @garrytan-agents):** the load-bearing change is the lock-identity invariant — `performSync` now defaults to a per-source lock id (`gbrain-sync:<sourceId>`) whenever `opts.sourceId` is set, AND wraps the writer window in `withRefreshingLock` from `src/core/db-lock.ts` so long-running sources (the PR motivator: media-corpus / 250K+ chunks) don't lose their lock at the 30-min TTL mid-run. Legacy single-default-source brains keep the bare `tryAcquireDbLock(SYNC_LOCK_ID)` path for back-compat. `SyncOpts.lockId?: string` is the explicit override (escape hatch; production callers don't set it). The fix closes the bug class Codex's outside-voice review caught in the original PR — a `sync --all` worker on per-source lock racing against `sync --source foo` on global lock would have silently corrupted the same source. `gbrain sync --all` got a continuous worker pool: `parseWorkers`-validated `--parallel N` (default `min(sourceCount, --workers, DEFAULT_PARALLEL_SOURCES=4)` from `src/core/sync-concurrency.ts`), `parallel` long-lived async workers pulling from a shared FIFO queue (no head-of-line blocking from wave dispatch), per-source `withSourcePrefix(src.id, ...)` wrap from `src/core/console-prefix.ts` so every `slog`/`serr` line carries `[<source-id>] ` for kubectl-style greppable parallel output. `--skip-failed` and `--retry-failed` reject with a paste-ready error when combined with `--parallel > 1` (the brain-global `sync-failures.jsonl` has no per-source scope; v0.41+ TODO source-scopes it). Connection-budget stderr warning fires when `parallel × workers × 2 > 16` (the `× 2 per-file pool` factor: each per-file worker opens its own `PostgresEngine` with `poolSize=2`, so `--parallel 4 --workers 4` is actually 32 connections, not 16 — Codex caught the 2× understatement in the original plan). New exports `resolveParallelism`, `syncOneSource`, `buildSyncStatusReport`, `printSyncStatusReport`, `SyncStatusReport` back the `gbrain sources status` dashboard surface (D3 → `sources` subcommand, not a `sync` flag, so reads and writes don't share a verb). Stable `--json` envelope `{schema_version: 1, sources, parallel, ok_count, error_count}` on stdout under `--json`; human banners route to stderr via the `humanSink` helper so `jq` parses cleanly. Exit matrix: 0 = all ok, 1 = any error, 2 = cost-prompt-not-confirmed (unchanged). The dashboard SQL is the canonical `content_chunks ch JOIN pages pg ON pg.id = ch.page_id WHERE pg.deleted_at IS NULL` shape with `archived = false` filter at the caller — the original PR shipped `chunks ch JOIN ON page_slug`, which would have crashed on PGLite parse and silently zeroed on Postgres via the swallow-catch. Embedding column resolved via `resolveEmbeddingColumn(undefined, cfg)` from `src/core/search/embedding-column.ts` so Voyage / multimodal / non-default-column brains see counts against the column they actually use. Errors propagate from the dashboard SQL — no swallow-catch (Q2 sub-fix). The IRON RULE regression lives in `test/e2e/sync-status-pglite.test.ts`: real PGLite seeds 2 sources × pages × chunks, soft-deletes 1 page, archives 1 source, validates the SQL excludes both AND the active embedding column is the one used. 38 in-file `console.log`/`console.error` call sites inside `performSync*` migrated to `slog`/`serr` (top-level `runSync` orchestrator console calls intentionally stay outside the prefix scope). **v0.41.25.0 (supersedes PR #1538):** sync delete loop rewritten as interleaved per-batch resolve+delete using `engine.resolveSlugsByPaths` + `engine.deletePages` from `src/core/engine.ts`. 73K-delete commit drops from ~146K SQL round-trips (~5h on Postgres+pgbouncer) to ~292 round-trips (~2min) — closes the cascade-staleness bug class where one big-delete commit jammed every other source's sync for hours. Per-batch try-catch decomposes batch DELETE failures to per-slug `deletePage` fallback so a transient blip on batch 73 doesn't lose 500 deletes; unrecoverable per-slug failures land in `failedFiles` (matches the existing import-loop pattern). `pagesAffected` filters to D6 confirmed-deleted slugs (downstream extract/embed stop wasting work on phantoms). Same batched slug-resolve treatment for the rename loop (`gbrain sync` renames — Phase 3 in the plan; the per-file `importFile` stays per-file, only the slug-resolve N+1 gets batched). The `resolveSlugByPathOrSourcePath` single-call helper at `sync.ts:267` now delegates to `engine.resolveSlugsByPaths` when `sourceId` is set (D8 DRY — one owner of the SQL + fallback semantics), keeping legacy `executeRaw` fallback for the no-sourceId path which is functionally dead post-v0.34.1's source-resolution wiring. Legacy no-sourceId branch in the delete loop preserves correctness on a slow path; the engine batch methods require sourceId so the new fast path only fires when production threading is intact. `failedFiles` declaration hoisted to top of `performSyncInner` so both delete decompose and import loops feed the same sync-bookmark gate. **v0.41.31.0 (mode-aware cost gate):** the v0.20.0 unconditional `sync --all` ConfirmationRequired gate is superseded by a mode-aware gate driven by `willEmbedSynchronously` + `shouldBlockSync` from `src/core/embedding.ts`. federated_v2 is resolved ONCE up front (shared with the fan-out below). On the DEFERRED path (v2 on, parallel) embedding goes to per-source `embed-backfill` jobs that carry their own `$X/source/24h` cap (default $25, `SPEND_CAP_CONFIG_KEY` from `embed-backfill-submit.ts`), so the gate is INFORMATIONAL — it prints an FYI naming the backfill cap + current backlog (`sumStaleChunkChars({signature})` priced via `estimateCostFromChars` at `currentEmbeddingPricePerMTok()`) + the count of already-queued `embed-backfill` jobs (TODO-2 visibility), and NEVER exits 2. On the INLINE path (v2 off, or `--serial` without `--no-embed`) the BLOCKING gate fires only when the NEW-CONTENT estimate exceeds the `sync.cost_gate_min_usd` floor (default $0.50). The new-content estimate (`estimateInlineNewTokens`) contributes ZERO for sources provably unchanged since last sync (`isSourceUnchangedSinceSync` HEAD==last_commit + clean tree, AND `chunker_version === CURRENT`) — mirrors doctor's `sync_freshness` + sync's own do-work gate — and the full-tree ceiling for changed sources. F2 fix: the pre-existing stale backlog (NULL embeddings + signature drift) is shown informationally but NEVER gated on, because sync doesn't sweep it inline (`gbrain embed --stale` does); gating on it would block every inline cron after a model swap. New helpers `resolveCostGateFloorUsd(engine)` (reads `sync.cost_gate_min_usd`, fail-open $0.50, accepts 0) + `resolveBackfillCapUsd(engine)`. The `sync --all` SELECT widens to carry `last_commit + chunker_version` (both columns predate v0.41; no migration). JSON envelopes gain `mode` + `gate` discriminators (`dry_run | deferred_notice | below_floor | confirmation_required`). `SyncStatusReportSource` gains `backfill_queued` / `backfill_active` / `backfill_last_completed_at` (best-effort; 0/null on pre-minions brains). All embedding-cost preview math reads the configured model name via `getEmbeddingModelName()` (no hardcoded OpenAI). Pinned by `test/sync-cost-gate.serial.test.ts` + `test/sync-cost-preview.test.ts`. **v0.41.37.0 (#1569):** new `SyncOpts.noSchemaPack` (CLI `--no-schema-pack`, threaded through `performSync` AND `syncOneSource` so `sync --all` honors it) — skips `loadActivePack` entirely so no user-supplied pack page-type regex runs during import; pages fall back to legacy prefix typing. Escape hatch for completing a sync when a suspect pack regex is the suspected cause of a wedge (re-run extraction later). Same release adds a per-file BEGIN heartbeat: `if (process.env.GBRAIN_SYNC_TRACE) serr('[sync] begin import: <path>')` emitted BEFORE `importFile` (the existing `progress.tick` fires only AFTER `importFile` returns — useless when one file wedges). Off by default to avoid a line per file on huge brains; `serr` is source-prefix-aware so under `--workers >1` / `--all` the stuck file is the begin-line with no matching completion. Triage doc: `docs/architecture/serve-sync-concurrency.md` (PGLite single-writer serve↔sync contention — stop `gbrain serve` before a large PGLite sync — plus the `GBRAIN_SYNC_TRACE` + `--no-schema-pack` hang-triage recipes). **v0.42.11.0 (#1633) — orphan-pileup / unkillable sync:** exports `resolveSyncHardDeadline(args, {isTty, env, defaultNonTtySec?})` (pure; precedence `--no-hard-deadline` > `--hard-deadline <s>` > `--timeout <s>`(non-`--all`, auto-arms the hard backstop) > `GBRAIN_SYNC_MAX_RUNTIME_SECONDS` env > non-TTY default 3600s > none; `HARD_DEADLINE_GRACE_SEC=30`) + `composeAbortSignals(...signals)` (CQ2 — `AbortSignal.any` over the defined signals; single→passthrough, none→undefined). `src/cli.ts` installs `installProcessWatchdog` (see `src/core/process-watchdog.ts`) for `command === 'sync'` BEFORE `connectEngine` using that resolution, disposing it in the dispatch `finally` — so a runaway sync (incl. a connect-phase hang) is SIGKILLed by the deadline even when its event loop is starved, making the cron orphan-pileup structurally impossible. Part B graceful-cancel: `runSync` registers a SIGINT handler on both the single-source and `--all` paths that aborts an interrupt `AbortController` composed (via `composeAbortSignals`) with the per-source `--timeout` signal — Ctrl-C returns a clean `partial` + releases the lock via the normal finally instead of a hard cut that leaks the lock (process-cleanup.ts already owns SIGTERM lock-release + exit; the watchdog owns the hard kill). `src/core/db-lock.ts:withRefreshingLock` now `unref()`s its refresh `setInterval`. NOT root-causing the spin (no repro; #1633 stays open for it — leading lead is ReDoS in pack link-inference, partially mitigated by v0.41.37.0's redos-guard); the watchdog heartbeat + existing `[gbrain phase]` breadcrumbs are the diagnosis surface. Per-file cooperative yielding was evaluated and skipped (the import loop already `await`s per file → already a yield point; only true mid-file synchronous starvation matters, which the watchdog covers). Pinned by `test/sync-hard-deadline.test.ts` (resolution precedence + `composeAbortSignals`).
 - `src/commands/import.ts` — `gbrain import` CLI + `runImport` library entrypoint. v0.34.2.0 replaces the prior positional-index checkpoint (`processedIndex: N` into a sorted file list) with a path-set checkpoint via `src/core/import-checkpoint.ts`. The walk still applies `sortNewestFirst()` for embed-cost ordering, but checkpoint correctness no longer depends on sort order. A file enters `completed: Set<relativePath>` only when its `processFile` returns success (including content-hash short-circuit no-ops); failed files never enter the set, so the next run retries them automatically with no manual `~/.gbrain/import-checkpoint.json` delete. Three bug classes died: parallel-import-with-slow-worker drops the slow file on crash-resume (closed — the slow file isn't in `completed` until its own `processFile` resolves), failed-file-bumps-counter-past-itself (closed — failures don't add to `completed`), and v0.33.x sort-flip-drops-newest-N-on-cross-version-resume (closed — order is no longer part of the checkpoint). Old positional checkpoints are detected and discarded with a stderr line on first resume; re-walking is cheap because `content_hash` short-circuits unchanged files. Checkpoint persists every 100 successful adds, not every 100 processed files, so a long failure tail doesn't churn the JSON. Pinned by `test/import-checkpoint.test.ts` (18 unit cases over the helpers) + `test/import-resume.test.ts` (5 integration cases under PGLite, including the SLUG_MISMATCH retry regression codex caught during plan-eng-review).
 - `src/core/import-checkpoint.ts` (v0.34.2.0) — `loadCheckpoint(brainDir)`, `saveCheckpoint(brainDir, completed)`, `resumeFilter(files, completed, brainDir)`, `clearCheckpoint()`, plus the `ImportCheckpoint` type. Path-set checkpoint format (`{schema_version, brainDir, completed: string[]}`) replaces the v0.33.x positional `{processedIndex: N}` format. Atomic write via `.tmp` + `rename()` so a mid-write crash never leaves a partial JSON. `loadCheckpoint` returns `null` on: missing file, malformed JSON, brainDir mismatch (you ran import against a different brain), and the old positional format (logged to stderr before being discarded). `resumeFilter` returns `{toProcess, skippedCount}` — pure, no I/O, deterministic. `clearCheckpoint` is no-op-on-missing for clean-exit cleanup. Honors `GBRAIN_HOME` via `gbrainPath()` so test isolation via `withEnv({GBRAIN_HOME: tmpdir})` works without monkey-patching the fs layer. Best-effort persistence — `saveCheckpoint` logs warnings on write errors but never throws, so import keeps making progress even if disk is full.
 - `src/core/sort-newest-first.ts` (v0.34.2.0) — single source of truth for the descending-lex sort that `gbrain import` and `gbrain sync` both apply. Mutates in place (Array.prototype.sort semantics), returns the same array reference for fluent chaining. Empty/single-element inputs short-circuit. Future ordering changes flip one line in this helper instead of touching two CLI commands. Pinned by `test/sort-newest-first.test.ts` (5 hermetic cases: descending order, mixed prefixes, empty input, single-element input, in-place-mutation contract).