diff --git a/CHANGELOG.md b/CHANGELOG.md index 5a369e18d..78ae807ff 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -209,6 +209,85 @@ Every originally-deferred follow-up is included: - **Issue #1481 closed** — supersedes the original proposal with the decisions captured in plan `~/.claude/plans/system-instruction-you-are-working-drifting-falcon.md`. +## [0.41.27.0] - 2026-05-27 + +**`gbrain doctor` stops crying wolf about sources that have no new commits.** + +If you have a brain that points at a git repo you don't change every day +(a reference corpus, a frozen archive, an inbox you append to monthly), +doctor's "Source X last synced 40h ago" warning has been firing every +single day even though nothing was actually stale. There was no new +content to pull. The warning was telling you to run `gbrain sync` for +work that didn't exist. + +This release teaches doctor to do a quick git check first: if your +repo's current HEAD matches what we stored at the last sync, **and** +the working tree is clean, **and** the chunker version still matches, +the source is reported as "up to date" regardless of how long ago you +synced. The three checks together mirror exactly what `gbrain sync` +itself decides — so doctor and sync now agree on "is there work to do?" +You stop seeing warnings for problems that don't exist. + +**How to turn it on:** Nothing to do. `gbrain upgrade` is all you need. +The git probe runs locally only (your terminal's `gbrain doctor`); the +remote HTTP MCP path stays out of it by design (deliberately doesn't +walk DB-supplied paths via subprocess — same trust posture as before). + +**What you'd see in a concrete example:** + +| Doctor scenario | Pre-fix message | Post-fix message | +|---|---|---| +| 40h since sync, zero new commits, clean tree | warn: "Source 'media-corpus' last synced 40h ago" | ok: "All 1 federated source(s) up to date (no new commits since last sync)" | +| 40h since sync, 3 new commits to pull | warn: "...40h ago" | warn: "...40h ago" (unchanged — warning still correct) | +| 40h since sync, HEAD matches but `gbrain upgrade` bumped CHUNKER_VERSION | warn: "...40h ago" (true bug — pending re-embed) | warn: "...40h ago" (chunker gate fires; you still see the warn so you don't miss the post-upgrade re-embed) | +| 40h since sync, HEAD matches but you have uncommitted edits | warn: "...40h ago" | warn: "...40h ago" (dirty-tree gate fires; honest about pending work) | +| Mixed brain: 1 frozen source + 1 recently synced + 1 truly stale | fail: lists all 3 sources | fail: lists only the truly stale source; the other two are silenced honestly | + +**The cycle freshness check is deliberately NOT touched.** A separate +codex-review pass caught a load-bearing semantic distinction: HEAD == +last_commit only answers "are there new commits to sync?". It cannot +answer "did the full cycle (sync + extract + embed + consolidate + +synthesize) complete recently?" — that's a later, different invariant. +Hiding cycle-staleness warnings on git-clean sources would silently +mask the case where sync ran but the cycle phases after it failed. +Doctor's `cycle_freshness` keeps its time-only semantics; only +`sync_freshness` gets the git-aware short-circuit. + +**Things worth knowing about:** + +- `Check.details` now carries `{unchanged_count, synced_recently_count, + stale_count}` for the `sync_freshness` check. Dashboards consuming + the JSON envelope can read these directly. The three counts sum to + the source count — invariant pinned in unit tests. +- If you intentionally keep WIP edits in your brain repo, doctor will + still warn after 24h because the dirty-tree gate fires. That's + honest — sync would actually do work in that case. An opt-out env + var (`GBRAIN_DOCTOR_IGNORE_DIRTY_TREE=1`) is filed for v0.41.27.1+ + if anyone asks. + +**The cathedral side.** This release rebuilds community PR #1564 with +four production-quality concerns the original missed: shell-injection +safety (uses `execFileSync` with array args, not `execSync` through +`/bin/sh -c`), trust-boundary preservation (remote-callable doctor +path doesn't get the git probe), narrowed predicate honesty (chunker +version + working-tree-clean, matching sync's own gate), and 21 new +test cases covering every branch including a shell-injection +regression guard that runs real `execFileSync` against a `$(...)` +adversarial path to prove the array-arg shape cannot escape to a +shell. Co-Authored-By preserved for the original contributor. + +### Itemized changes + +- **`src/core/git-head.ts`** (NEW) — single `isSourceUnchangedSinceSync(localPath, lastCommit, opts?)` primitive with two probes (head + clean). `GitFreshnessOpts.requireCleanWorkingTree` is the second-probe gate. Two test seams (`_setGitHeadProbeForTests`, `_setGitCleanProbeForTests`) match the `last-retrieved.ts` precedent so tests stay parallel-eligible (R2-compliant — no `mock.module`). Fail-open contract: every error path returns false, preserving the caller's prior behavior. Uses `execFileSync` with array args — shell metachars in `local_path` cannot escape. +- **`src/commands/doctor.ts:checkSyncFreshness`** — signature gains `opts?.localOnly?: boolean`. Inline SELECT widens to carry `last_commit + chunker_version` (columns already exist; no schema migration). Helper wired AFTER the existing NULL / negative-age guards, gated by `localOnly === true` AND'd with `source.chunker_version === String(CHUNKER_VERSION)`. Three-bucket count math (`unchanged_count + synced_recently_count + stale_count === sources.length`) populates `Check.details`. OK message reshape: all-unchanged hits "up to date (no new commits since last sync)"; mixed hits "X synced recently, Y unchanged since last sync"; all-synced keeps the prior message. +- **Caller plumbing**: `runDoctor` (local CLI path) passes `localOnly: true`; `doctorReportRemote` (HTTP MCP path) keeps the default `false`. Default-false is fail-closed: a future caller that forgets the opt gets the safe (no git probe) behavior. Codex P0-1 closure. +- **`test/core/git-head.test.ts`** (NEW) — 12-case suite: happy path, mismatch, null/empty guards, probe-null, probe-throws, **shell-injection regression** (real `execFileSync` against `/nonexistent/$(touch )/repo` — sentinel file MUST NOT exist after the call), test-seam round-trip, `requireCleanWorkingTree` clean/dirty/error/not-set cases. +- **`test/doctor.test.ts`** — new v0.41.27.0 describe with 9 cases: HEAD-match short-circuit, all-unchanged cold path, HEAD-mismatch warn, NULL `last_commit`, non-git path, **3-source mixed bucket invariant** (`sum === sources.length` asserted explicitly), chunker-version mismatch warn, dirty-tree warn, **`localOnly=false` regression guard** that verifies probes are NEVER called when the opt is unset or false. +- **CHANGELOG / VERSION / package.json / bun.lock / llms.txt / llms-full.txt / CLAUDE.md** — version bump + lockfile refresh + docs regen + key-files annotation. + +### Supersedes + +PR #1564 (`@garrytan-agents`). Co-Authored-By preserved. Closed with a supersession comment explaining the production-quality additions. The surrogate-pair fix from commit `78b93f3f` in that PR is NOT brought over — already on master via `safeSplitIndex` at `synthesize.ts:192` (v0.42.0.0 wave). ## [0.41.26.1] - 2026-05-27 **Your worker daemon stops crashing 39 times a day.** diff --git a/CLAUDE.md b/CLAUDE.md index f9582781f..d54968e1f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -69,6 +69,7 @@ strict behavior when unset. - `src/core/storage.ts` — Pluggable storage interface (S3, Supabase Storage, local) - `src/core/storage-config.ts` (v0.22.11) — Storage tiering: `loadStorageConfig` reads `gbrain.yml`, normalizes deprecated keys (`git_tracked` / `supabase_only`) to canonical (`db_tracked` / `db_only`) with once-per-process deprecation warning, and runs `normalizeAndValidateStorageConfig` (auto-fixes missing trailing `/`, throws `StorageConfigError` on tier overlap). Path-segment matcher: `media/x/` does NOT match `media/xerox/foo`. Replaces gray-matter (broken on delimiter-less YAML) with a dedicated parser for the `gbrain.yml` shape. - `src/core/disk-walk.ts` (v0.22.11) — `walkBrainRepo(repoPath)` returns `Map` from one recursive `readdirSync`. Skips dot-dirs, `node_modules`, non-`.md` files. Used by `gbrain storage status` to replace per-page `existsSync + statSync` (~400K syscalls on 200K-page brains → tens). +- `src/core/git-head.ts` (v0.41.27.0) — local git HEAD freshness probe for `gbrain doctor`. `isSourceUnchangedSinceSync(localPath, lastCommit, opts?)` returns true iff `localPath` is a git repo whose current HEAD matches `lastCommit`; when `opts.requireCleanWorkingTree` is true, also requires the working tree to have no uncommitted changes (mirrors `gbrain sync`'s force-walk gate at `sync.ts:1075` so doctor and sync agree on "is there work to do?"). Two probe seams (`_setGitHeadProbeForTests`, `_setGitCleanProbeForTests`) match the `last-retrieved.ts` precedent so unit tests stay R2-compliant (no `mock.module`). Uses `execFileSync` with array args — shell metachars in `local_path` cannot escape to a shell (the v0.41.27.0 superseded PR #1564 used `execSync` through `/bin/sh -c` with `JSON.stringify` for shell-escape, which is unsafe; the rebuild's regression test runs real `execFileSync` against `'/nonexistent/$(touch )/repo'` and asserts the sentinel file is never created). Fail-open on every error: missing path, not a git repo, git not installed, timeout, NULL inputs, dirty-probe errored → returns false, preserving the caller's prior time-based behavior. Chunker-version-match check lives in the caller (doctor.ts) because it depends on engine state (`sources.chunker_version` vs `CHUNKER_VERSION` from `src/core/chunkers/code.ts`). Designed for reuse: autopilot's per-source dispatch will want the same gate (filed as v0.41.27.1+ TODO in the plan). Pinned by `test/core/git-head.test.ts` (14 cases incl. the shell-injection regression guard). - `src/core/git-remote.ts` (v0.35.3.0) — SSRF-hardened git invocations for remote-source `cloneRepo` and `pullRepo`. Exports two distinct flag constants because `git`'s argv grammar treats them differently: `GIT_SSRF_FLAGS` (3 `-c` config flags — `protocol.allow=user`, `protocol.file.allow=never`, `http.allowRedirects=false`) is **global config**, spread BEFORE the subcommand verb. New `GIT_SSRF_SUBCOMMAND_FLAGS = ['--no-recurse-submodules']` is **subcommand-scoped**, spread AFTER the verb. Pre-v0.35.3 a single combined `GIT_SSRF_FLAGS` array spread `--no-recurse-submodules` before the verb where real git rejects it with exit 129 ("unknown option"); the fake-git test harness exited 0 regardless of argv shape, so CI missed it for ~7 months and every remote-source clone/pull was silently broken. `cloneRepo` argv: `git clone --depth=1 [--branch X] -- `. `pullRepo` argv: `git -C pull --ff-only`. Pinned by `test/git-remote.test.ts` position-anchored regression guard (`argv.indexOf('--no-recurse-submodules') > argv.indexOf(verb)`). - `src/commands/storage.ts` (v0.22.11) — `gbrain storage status [--repo P] [--json]`. Split into pure data (`getStorageStatus`) + JSON formatter + human formatter (ASCII-only per D10) matching the `orphans.ts` pattern. `PageCountsByTier` and `DiskUsageByTier` are distinct nominal types so swaps fail at compile time. - `gbrain.yml` (brain repo root, v0.22.11) — Optional storage tiering config. Top-level `storage:` section with `db_tracked:` and `db_only:` array-valued keys. `gbrain sync` auto-manages `.gitignore` for `db_only` paths on successful sync (skips on dry-run, blocked-by-failures, submodule context, or `GBRAIN_NO_GITIGNORE=1`). `gbrain export --restore-only [--repo P] [--type T] [--slug-prefix S]` repopulates missing `db_only` files from the database. @@ -259,7 +260,7 @@ strict behavior when unset. - `src/core/ai/model-resolver.ts:parseModelId` extension (v0.41.21.0) — gateway-side resolver extended to accept slash form alongside colon. Pre-fix the colon-only check threw `AIConfigError: model id must be in format provider:model` at every gateway entry point (chat / embed / rerank) the moment a slash-form id was passed. So even with the v0.41.21.0 pricing fix, a `--judge-model anthropic/claude-sonnet-4-6` invocation would clear BudgetTracker but then fail mid-judge inside `gateway.chat()`. Now both shapes resolve to the same recipe. Bare names without ANY separator still throw — gateway routing always needs an explicit provider. Pinned by 10 cases in `test/ai/model-resolver-slash.test.ts` including a `resolveRecipe` round-trip asserting slash form resolves to the same recipe object as colon form. - `src/commands/transcripts.ts` (v0.29) — `gbrain transcripts recent [--days N] [--full] [--json]`: recent raw `.txt` transcripts from the dream-cycle corpus dirs. Imports `listRecentTranscripts` from `src/core/transcripts.ts` (the same library the gated `get_recent_transcripts` MCP op uses). Local-only by construction — the CLI always runs with `ctx.remote=false`. - `src/commands/integrity.ts` — `gbrain integrity check|auto|review|extract`: bare-tweet detection, dead-link detection, three-bucket repair (auto-repair / review-queue / skip). `scanIntegrity()` is the shared library function called from `gbrain doctor` (sampled at limit=500) and `cmdCheck` (full scan). v0.22.8: batch-load fast path on Postgres uses a single SQL query to fix the PgBouncer round-trip timeout (60s → ~6s). Gated by `engine.kind === 'postgres'` at the call site so PGLite never enters batch; fallback `catch` logs at `GBRAIN_DEBUG=1` so real Postgres errors are diagnosable. **v0.32.8 (PR #860):** batch projection switched from `SELECT DISTINCT ON (slug)` to `SELECT ... ORDER BY source_id, slug` so multi-source brains scan each `(source, slug)` row independently (pre-fix the DISTINCT collapsed same-slug-different-source pages into one scan, the same bug class this PR fixes). Sequential and auto-repair loops use `listAllPageRefs()` to enumerate `(slug, source_id)` pairs and thread `sourceId` to `getPage`. Batch + sequential paths now report the same page count on multi-source brains. -- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel `. v0.22.12 (#500): `sync_failures` check shows `[CODE=N, ...]` breakdown for both unacked entries (warn) and acked-historical entries (ok), surfacing systemic failure modes (`SLUG_MISMATCH=2685`) instead of a bare count. v0.26.7 (#612): `rls_event_trigger` check (post-install drift detector for migration v35's auto-RLS event trigger). Lives outside the `// 5. RLS` slice that the structural doctor.test.ts guards anchor on, so the existing test guards stay intact. Healthy `evtenabled` set is `('O','A')` only — `R` is replica-only and would not fire in normal sessions; `D` is disabled. Fix hint is `gbrain apply-migrations --force-retry 35`. **v0.30.2:** `queue_health` gains a fourth subcheck — surfaces dead-lettered subagent jobs with `last_error` matching the `prompt_too_long` classifier within the last 24h. Fix hint points at `gbrain dream --phase synthesize --dry-run --json` to identify the offending transcript and `gbrain jobs prune --status dead --queue default` to clean up. Postgres-only. **v0.31.7:** `runDoctor` switches to `autoDetectSkillsDirReadOnly` (from `src/core/repo-root.ts`) so `bun install -g github:garrytan/gbrain && cd ~ && gbrain doctor` finds the bundled `skills/` via the install-path fallback instead of warning "Could not find skills directory" + docking the health score. `--fix` carries a D6 safety gate: when `detected.source === 'install_path'`, the command refuses auto-repair with a stderr message pointing at `$GBRAIN_SKILLS_DIR` / `$OPENCLAW_WORKSPACE` / `--skills-dir`, because `autoFixDryViolations` writes to SKILL.md files and would otherwise silently rewrite the install tree. The `graph_coverage` check now short-circuits to `ok: 'No entity pages — graph_coverage not applicable (markdown-only brain)'` when `SELECT COUNT(*) FROM pages WHERE type IN ('entity','person','company','organization')` returns 0 (closes #530); the entity count is woven into the warn message and the WARN hint switches from the long-deprecated `gbrain link-extract && gbrain timeline-extract` (gone since v0.16) to the canonical `gbrain extract all`. Pinned by an IRON-RULE regression assertion in `test/doctor.test.ts` that bans the stale verb names from the source string. **v0.32.4:** new `sync_freshness` check (exported `checkSyncFreshness` at the same file) added to both `runDoctor` (local) and `doctorReportRemote` (thin-client). Pure staleness probe — queries `sources.last_sync_at` only, no filesystem access. Warns at 24h, fails at 72h (or never-synced). Future-`last_sync_at` warns ("clock skew or corrupted timestamp") instead of silently falling through as ok — codex outside-voice caught the negative-ageMs bug pre-merge. Env-var overrides `GBRAIN_SYNC_FRESHNESS_WARN_HOURS` / `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS`; invalid values fall back to defaults with a once-per-process stderr warn (`_resolveSyncFreshnessHours`). Failure messages embed `source.id` (not `source.name`) so the printed fix command `gbrain sync --source ` matches what the user copy-pastes. Filesystem-vs-DB page drift detection was deliberately stripped from the v0.32.4 scope — `doctorReportRemote` runs in the HTTP MCP server (`src/commands/serve-http.ts`), and walking DB-supplied `local_path` from a remote-callable endpoint crosses a trust boundary (OAuth write scope could mutate `sources.local_path`). Drift detection will resurface in a separate PR routed through `multi_source_drift`'s existing guard infrastructure (`GBRAIN_DRIFT_LIMIT` / `GBRAIN_DRIFT_TIMEOUT_MS`) with slug normalization tests and a meta-file allow-list. Pinned by 12 cases in `test/doctor.test.ts` ("v0.32.4 — sync_freshness check" describe block): empty sources, never-synced fail, >72h fail, exact 72h boundary, 24h-72h warn, exact 24h boundary, <24h ok, future-timestamp warn, mixed sources (highest severity wins), `executeRaw` throws → outer-catch warn, `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS=6` override fires at 7h, source.id-in-message regression. **v0.36.3.0:** new `embedding_column_registry` check probes each declared column via Postgres `format_type(atttypid, atttypmod)` so a registry entry claiming 1024d Voyage against an actual 1536d OpenAI column surfaces with a paste-ready `gbrain config set embedding_columns '{...}'` ALTER hint instead of mysterious "vector dimension mismatch" errors at search time. On Postgres the check also probes HNSW index presence (`pg_indexes` lookup keyed by column name) and warns when missing (search will still work via seq scan but won't hit the index). The active default column's population coverage is computed via `COUNT(*) FILTER (WHERE IS NOT NULL) / COUNT(*)` and warns below 90% — except empty brains (chunk_count = 0) where the gate short-circuits to `ok` so fresh `gbrain init` runs don't see "Active column 'embedding' is 0.0% populated" (CDX-5 codex fix). PGLite parity via the same SQL through `executeRaw` — registry validation happens on both engines. **v0.35.5.0:** the Lane D supervisor check at `doctor.ts:1011-1043` now consumes `summarizeCrashes(events)` from `src/core/minions/handlers/supervisor-audit.ts` instead of the pre-fix `events.filter(e => e.event === 'worker_exited').length`. The warn threshold drops from `>3` to `>=1` (any real crash is signal now that the counter is calibrated against clean exits). The ok message gains `clean_exits_24h=N`; the warn message gains `runtime=A oom=B unknown=C legacy=D` per-cause breakdown so an operator triages OOM vs runtime-error vs unknown-future-cause at a glance without grep'ing the JSONL audit. Closes the "Supervisor crashes: 120x/24h, was 62x — nearly doubled" alarm class that bit users on healthy brains after v0.34.3.0's RSS-watchdog work added more code=0 worker drains — both `doctor` and `gbrain jobs supervisor status` were counting every `worker_exited` event as a crash regardless of cause. Cross-surface parity is the regression guard: 4 source-grep wiring assertions in `test/doctor.test.ts` ban the ad-hoc filter pattern, pin the `>=1` threshold, and require the per-cause breakdown substrings (`runtime=`, `oom=`, `unknown=`, `legacy=`, `clean_exits_24h=`, `crashes_by_cause`) to appear in BOTH `doctor.ts` and `jobs.ts`. **v0.37.1.0:** new `skill_brain_first` check. Walks every SKILL.md under the configured skills dir (`autoDetectSkillsDirReadOnly` so `cd ~ && gbrain doctor` finds the bundled skills via the install-path fallback), calls `analyzeSkillBrainFirst()` from `src/core/skill-brain-first.ts` per file, aggregates verdicts into a single check with structured `Check.issues[]` for JSON tooling. Warn states: `missing_brain_first` (external-lookup pattern present, no canonical callout, no `brain_first: exempt`), `brain_first_typo` (near-miss declaration like `brain-first` or `BrainFirst` — paste-ready hint surfaces the correct snake_case form). Ok states: `compliant_callout`, `compliant_phase`, `compliant_position`, `exempt_frontmatter`, `no_external`. `--fix` routes through `dry-fix.ts` MISSING_RULE_PATTERNS to auto-insert the canonical `> **Convention:** see [conventions/brain-first.md](...)` callout (D6 install-path safety gate enforced — `--fix` from `~` refuses to write to the bundled tree). Snapshot+diff audit at `~/.gbrain/audit/skill-brain-first-YYYY-Www.jsonl` records detected / resolved / fixed transitions only (stable brains: 0 lines/run). Motivated by the 2026-05-19 tweet-shield incident: cross-modal eval flagged Garry's Palantir tweet as risky because no model knew he built it, but the brain already had "designed the entire Finance product UI" and "150+ PSDs from April-December 2006." Static check catches the AUTHORSHIP miss class; v0.37+ runtime gate (filed in TODOS.md) closes the dispatch side. +- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel `. v0.22.12 (#500): `sync_failures` check shows `[CODE=N, ...]` breakdown for both unacked entries (warn) and acked-historical entries (ok), surfacing systemic failure modes (`SLUG_MISMATCH=2685`) instead of a bare count. v0.26.7 (#612): `rls_event_trigger` check (post-install drift detector for migration v35's auto-RLS event trigger). Lives outside the `// 5. RLS` slice that the structural doctor.test.ts guards anchor on, so the existing test guards stay intact. Healthy `evtenabled` set is `('O','A')` only — `R` is replica-only and would not fire in normal sessions; `D` is disabled. Fix hint is `gbrain apply-migrations --force-retry 35`. **v0.30.2:** `queue_health` gains a fourth subcheck — surfaces dead-lettered subagent jobs with `last_error` matching the `prompt_too_long` classifier within the last 24h. Fix hint points at `gbrain dream --phase synthesize --dry-run --json` to identify the offending transcript and `gbrain jobs prune --status dead --queue default` to clean up. Postgres-only. **v0.31.7:** `runDoctor` switches to `autoDetectSkillsDirReadOnly` (from `src/core/repo-root.ts`) so `bun install -g github:garrytan/gbrain && cd ~ && gbrain doctor` finds the bundled `skills/` via the install-path fallback instead of warning "Could not find skills directory" + docking the health score. `--fix` carries a D6 safety gate: when `detected.source === 'install_path'`, the command refuses auto-repair with a stderr message pointing at `$GBRAIN_SKILLS_DIR` / `$OPENCLAW_WORKSPACE` / `--skills-dir`, because `autoFixDryViolations` writes to SKILL.md files and would otherwise silently rewrite the install tree. The `graph_coverage` check now short-circuits to `ok: 'No entity pages — graph_coverage not applicable (markdown-only brain)'` when `SELECT COUNT(*) FROM pages WHERE type IN ('entity','person','company','organization')` returns 0 (closes #530); the entity count is woven into the warn message and the WARN hint switches from the long-deprecated `gbrain link-extract && gbrain timeline-extract` (gone since v0.16) to the canonical `gbrain extract all`. Pinned by an IRON-RULE regression assertion in `test/doctor.test.ts` that bans the stale verb names from the source string. **v0.32.4:** new `sync_freshness` check (exported `checkSyncFreshness` at the same file) added to both `runDoctor` (local) and `doctorReportRemote` (thin-client). Pure staleness probe — queries `sources.last_sync_at` only, no filesystem access. Warns at 24h, fails at 72h (or never-synced). Future-`last_sync_at` warns ("clock skew or corrupted timestamp") instead of silently falling through as ok — codex outside-voice caught the negative-ageMs bug pre-merge. Env-var overrides `GBRAIN_SYNC_FRESHNESS_WARN_HOURS` / `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS`; invalid values fall back to defaults with a once-per-process stderr warn (`_resolveSyncFreshnessHours`). Failure messages embed `source.id` (not `source.name`) so the printed fix command `gbrain sync --source ` matches what the user copy-pastes. Filesystem-vs-DB page drift detection was deliberately stripped from the v0.32.4 scope — `doctorReportRemote` runs in the HTTP MCP server (`src/commands/serve-http.ts`), and walking DB-supplied `local_path` from a remote-callable endpoint crosses a trust boundary (OAuth write scope could mutate `sources.local_path`). Drift detection will resurface in a separate PR routed through `multi_source_drift`'s existing guard infrastructure (`GBRAIN_DRIFT_LIMIT` / `GBRAIN_DRIFT_TIMEOUT_MS`) with slug normalization tests and a meta-file allow-list. Pinned by 12 cases in `test/doctor.test.ts` ("v0.32.4 — sync_freshness check" describe block): empty sources, never-synced fail, >72h fail, exact 72h boundary, 24h-72h warn, exact 24h boundary, <24h ok, future-timestamp warn, mixed sources (highest severity wins), `executeRaw` throws → outer-catch warn, `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS=6` override fires at 7h, source.id-in-message regression. **v0.36.3.0:** new `embedding_column_registry` check probes each declared column via Postgres `format_type(atttypid, atttypmod)` so a registry entry claiming 1024d Voyage against an actual 1536d OpenAI column surfaces with a paste-ready `gbrain config set embedding_columns '{...}'` ALTER hint instead of mysterious "vector dimension mismatch" errors at search time. On Postgres the check also probes HNSW index presence (`pg_indexes` lookup keyed by column name) and warns when missing (search will still work via seq scan but won't hit the index). The active default column's population coverage is computed via `COUNT(*) FILTER (WHERE IS NOT NULL) / COUNT(*)` and warns below 90% — except empty brains (chunk_count = 0) where the gate short-circuits to `ok` so fresh `gbrain init` runs don't see "Active column 'embedding' is 0.0% populated" (CDX-5 codex fix). PGLite parity via the same SQL through `executeRaw` — registry validation happens on both engines. **v0.35.5.0:** the Lane D supervisor check at `doctor.ts:1011-1043` now consumes `summarizeCrashes(events)` from `src/core/minions/handlers/supervisor-audit.ts` instead of the pre-fix `events.filter(e => e.event === 'worker_exited').length`. The warn threshold drops from `>3` to `>=1` (any real crash is signal now that the counter is calibrated against clean exits). The ok message gains `clean_exits_24h=N`; the warn message gains `runtime=A oom=B unknown=C legacy=D` per-cause breakdown so an operator triages OOM vs runtime-error vs unknown-future-cause at a glance without grep'ing the JSONL audit. Closes the "Supervisor crashes: 120x/24h, was 62x — nearly doubled" alarm class that bit users on healthy brains after v0.34.3.0's RSS-watchdog work added more code=0 worker drains — both `doctor` and `gbrain jobs supervisor status` were counting every `worker_exited` event as a crash regardless of cause. Cross-surface parity is the regression guard: 4 source-grep wiring assertions in `test/doctor.test.ts` ban the ad-hoc filter pattern, pin the `>=1` threshold, and require the per-cause breakdown substrings (`runtime=`, `oom=`, `unknown=`, `legacy=`, `clean_exits_24h=`, `crashes_by_cause`) to appear in BOTH `doctor.ts` and `jobs.ts`. **v0.37.1.0:** new `skill_brain_first` check. Walks every SKILL.md under the configured skills dir (`autoDetectSkillsDirReadOnly` so `cd ~ && gbrain doctor` finds the bundled skills via the install-path fallback), calls `analyzeSkillBrainFirst()` from `src/core/skill-brain-first.ts` per file, aggregates verdicts into a single check with structured `Check.issues[]` for JSON tooling. Warn states: `missing_brain_first` (external-lookup pattern present, no canonical callout, no `brain_first: exempt`), `brain_first_typo` (near-miss declaration like `brain-first` or `BrainFirst` — paste-ready hint surfaces the correct snake_case form). Ok states: `compliant_callout`, `compliant_phase`, `compliant_position`, `exempt_frontmatter`, `no_external`. `--fix` routes through `dry-fix.ts` MISSING_RULE_PATTERNS to auto-insert the canonical `> **Convention:** see [conventions/brain-first.md](...)` callout (D6 install-path safety gate enforced — `--fix` from `~` refuses to write to the bundled tree). Snapshot+diff audit at `~/.gbrain/audit/skill-brain-first-YYYY-Www.jsonl` records detected / resolved / fixed transitions only (stable brains: 0 lines/run). Motivated by the 2026-05-19 tweet-shield incident: cross-modal eval flagged Garry's Palantir tweet as risky because no model knew he built it, but the brain already had "designed the entire Finance product UI" and "150+ PSDs from April-December 2006." Static check catches the AUTHORSHIP miss class; v0.37+ runtime gate (filed in TODOS.md) closes the dispatch side. **v0.41.27.0:** `checkSyncFreshness` gains a `localOnly`-gated git short-circuit (D4 trust-boundary preservation per Codex P0-1): `runDoctor` passes `localOnly: true`, `doctorReportRemote` keeps the default `false` so the HTTP MCP path doesn't walk DB-supplied `local_path` values via subprocess. The narrowed predicate (D7) mirrors sync.ts:1057+1075's actual "do work?" gate: HEAD == `last_commit` AND working tree clean AND `sources.chunker_version === String(CHUNKER_VERSION)`. Inline SELECT widens to carry `last_commit + chunker_version` (columns already exist; no schema migration). Three-bucket count math (D6) populates `Check.details = {unchanged_count, synced_recently_count, stale_count}` with the invariant `sum === sources.length` pinned in the test suite. OK message reshape (D2): all-unchanged → "All N up to date (no new commits since last sync)"; mixed → "N source(s): X synced recently, Y unchanged since last sync"; all-synced-recently keeps the prior message verbatim. `checkCycleFreshness` is INTENTIONALLY NOT touched (Codex P0-2 / D5): `last_commit == HEAD` answers "are there new commits to sync?" but cannot answer "did the full cycle complete?" — a sync can succeed while later cycle phases fail, and silencing that warn would hide real cycle staleness. Pinned by 9 new cases in `test/doctor.test.ts` ("v0.41.27.0 — sync_freshness git short-circuit" describe block) including a load-bearing D4 regression guard that verifies probes are NEVER called when `localOnly` is unset or false, plus the three-bucket invariant explicitly asserted in the mixed 3-source case. Supersedes PR #1564 (Co-Authored-By preserved). - `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). **v40 (v0.29):** `pages_emotional_weight` adds `pages.emotional_weight REAL NOT NULL DEFAULT 0.0`. Column-only (no index). On Postgres 11+ and PGLite, `ADD COLUMN` with a constant DEFAULT is metadata-only — instant on tables of any size. v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2). **v30 (v0.23):** creates `dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PK(file_path, content_hash))`. RLS-enabled when running as a BYPASSRLS role. The synthesize phase reads/writes this table to avoid re-judging on backfill re-runs. **v35 (v0.26.7):** auto-RLS event trigger + one-time backfill. `auto_rls_on_create_table` fires on `ddl_command_end` for `WHEN TAG IN ('CREATE TABLE','CREATE TABLE AS','SELECT INTO')` and runs `ALTER TABLE … ENABLE ROW LEVEL SECURITY` on every new `public.*` table — no FORCE (matches v24/v29/schema.sql posture so non-BYPASSRLS apps can still read their own tables). The same migration backfills RLS on every existing `public.*` base table whose comment doesn't match the doctor regex (`^GBRAIN:RLS_EXEMPT\s+reason=\S.{3,}`). Per-table failure aborts the offending CREATE TABLE (event triggers fire inside the DDL transaction); no EXCEPTION wrap — that would convert loud rollback into silent permissive default. PGLite no-op via `sqlFor.pglite: ''`. Breaking change: operators with intentionally-RLS-off public tables must add the GBRAIN:RLS_EXEMPT comment BEFORE upgrade or the backfill will flip them on. **v46 (v0.31.3):** `mcp_request_log_params_jsonb_normalize` rewrites pre-v0.31.3 rows where `mcp_request_log.params` was stored as a JSON-encoded string (`jsonb_typeof = 'string'`) up to a real JSONB object via `UPDATE ... SET params = params::text::jsonb WHERE jsonb_typeof(params) = 'string'`. Single statement, idempotent — second-run finds no string-shaped rows and is a no-op. Closes the bug where `/admin/api/requests` returned a quoted string instead of the parsed object. **v0.36.3.0 (v68):** `eval_candidates_embedding_column` adds `eval_candidates.embedding_column TEXT NULL`. Per-row provenance for `gbrain eval replay`: captured rows record which column the live query ran against so replay reproduces the same retrieval space (Voyage rows replay against Voyage; OpenAI rows against OpenAI). NULL-tolerant — pre-v0.36 rows fall back to the current default during replay rather than failing. Column-only migration, metadata-only on both engines. **v0.34.1.0 (#861 + #876, v60-v65):** six-migration chain wires source-scoping into the OAuth client table. v60 (`oauth_clients_source_id_fk`) adds `oauth_clients.source_id TEXT` with NULL→`'default'` backfill and an FK to `sources(id) ON DELETE SET NULL`. v61 (`oauth_clients_federated_read_column`) adds `federated_read TEXT[] NOT NULL DEFAULT '{}'`. v62 (`oauth_clients_federated_read_backfill`) explicit-CASE backfills so `source_id IS NULL` produces `'{}'` not an array-containing-NULL. v63 (`oauth_clients_federated_read_validate`) is the fail-loud check that every row's source_id is in its federated_read array post-backfill. v64 (`oauth_clients_source_id_fk_restrict`) flips the FK to `ON DELETE RESTRICT` now that federated_read provides the alternative scope-loss path — source delete is refused if any client references it. v65 (`oauth_clients_federated_read_gin_index`) is the GIN index for the array-containment queries the read paths run. PGLite parity via `sqlFor.pglite` where needed. - `src/core/progress.ts` — Shared bulk-action progress reporter. Writes to stderr. Modes: `auto` (TTY: `\r`-rewriting; non-TTY: plain lines), `human`, `json` (JSONL), `quiet`. Rate-gated by `minIntervalMs` and `minItems`. `startHeartbeat(reporter, note)` helper for single long queries. `child()` composes phase paths. Singleton SIGINT/SIGTERM coordinator emits `abort` events for every live phase. EPIPE defense on both sync throws and stream `'error'` events. Zero dependencies. Introduced in v0.15.2. **v0.40.3.0:** `emitHumanLine` is prefix-aware — when called inside a `withSourcePrefix(id, ...)` scope from `src/core/console-prefix.ts`, prepends `[id] ` to the line content. TTY-rewrite mode (`\r\x1b[2K`) gets the prefix inside the clear-to-EOL escape so the rewritten line carries the prefix too. JSON mode (`emitJson`) is intentionally NOT prefixed — NDJSON consumers parse the JSON envelope and would choke on a `[id] {...}` shape. - `src/core/console-prefix.ts` (v0.40.3.0) — `AsyncLocalStorage`-backed per-source line-prefix helper. Exports `withSourcePrefix(id, fn)` (runs `fn` with `id` as the active prefix; nested wraps replace the active prefix and restore on exit), `getSourcePrefix()` (read-only accessor; test seam), and `slog(...)` / `serr(...)` (prefix-aware `console.log` / `console.error` replacements). Embedded-newline-safe: a multi-line string under prefix `[foo]` emits `[foo] line1\n[foo] line2`. Outside a wrap, `slog` / `serr` fall through to bare `console.log` / `console.error` so single-source callers see identical output (back-compat invariant). Use `src.id` (slug-validated by `sources add`) NOT `src.name` (free-form text) to defeat log-injection through newline / control-character names. Coverage in v0.40.3.0: `src/commands/sync.ts` performSync + in-file callees (38 call sites migrated), `src/commands/embed.ts` runEmbedCore + helpers (16 call sites), `src/core/progress.ts` emitHumanLine. Lines emitted from outside these modules will NOT get the prefix under parallel sync — file an issue if you find a missed migration target. diff --git a/llms-full.txt b/llms-full.txt index 07b7e578a..8bcbd36b8 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -211,6 +211,7 @@ strict behavior when unset. - `src/core/storage.ts` — Pluggable storage interface (S3, Supabase Storage, local) - `src/core/storage-config.ts` (v0.22.11) — Storage tiering: `loadStorageConfig` reads `gbrain.yml`, normalizes deprecated keys (`git_tracked` / `supabase_only`) to canonical (`db_tracked` / `db_only`) with once-per-process deprecation warning, and runs `normalizeAndValidateStorageConfig` (auto-fixes missing trailing `/`, throws `StorageConfigError` on tier overlap). Path-segment matcher: `media/x/` does NOT match `media/xerox/foo`. Replaces gray-matter (broken on delimiter-less YAML) with a dedicated parser for the `gbrain.yml` shape. - `src/core/disk-walk.ts` (v0.22.11) — `walkBrainRepo(repoPath)` returns `Map` from one recursive `readdirSync`. Skips dot-dirs, `node_modules`, non-`.md` files. Used by `gbrain storage status` to replace per-page `existsSync + statSync` (~400K syscalls on 200K-page brains → tens). +- `src/core/git-head.ts` (v0.41.27.0) — local git HEAD freshness probe for `gbrain doctor`. `isSourceUnchangedSinceSync(localPath, lastCommit, opts?)` returns true iff `localPath` is a git repo whose current HEAD matches `lastCommit`; when `opts.requireCleanWorkingTree` is true, also requires the working tree to have no uncommitted changes (mirrors `gbrain sync`'s force-walk gate at `sync.ts:1075` so doctor and sync agree on "is there work to do?"). Two probe seams (`_setGitHeadProbeForTests`, `_setGitCleanProbeForTests`) match the `last-retrieved.ts` precedent so unit tests stay R2-compliant (no `mock.module`). Uses `execFileSync` with array args — shell metachars in `local_path` cannot escape to a shell (the v0.41.27.0 superseded PR #1564 used `execSync` through `/bin/sh -c` with `JSON.stringify` for shell-escape, which is unsafe; the rebuild's regression test runs real `execFileSync` against `'/nonexistent/$(touch )/repo'` and asserts the sentinel file is never created). Fail-open on every error: missing path, not a git repo, git not installed, timeout, NULL inputs, dirty-probe errored → returns false, preserving the caller's prior time-based behavior. Chunker-version-match check lives in the caller (doctor.ts) because it depends on engine state (`sources.chunker_version` vs `CHUNKER_VERSION` from `src/core/chunkers/code.ts`). Designed for reuse: autopilot's per-source dispatch will want the same gate (filed as v0.41.27.1+ TODO in the plan). Pinned by `test/core/git-head.test.ts` (14 cases incl. the shell-injection regression guard). - `src/core/git-remote.ts` (v0.35.3.0) — SSRF-hardened git invocations for remote-source `cloneRepo` and `pullRepo`. Exports two distinct flag constants because `git`'s argv grammar treats them differently: `GIT_SSRF_FLAGS` (3 `-c` config flags — `protocol.allow=user`, `protocol.file.allow=never`, `http.allowRedirects=false`) is **global config**, spread BEFORE the subcommand verb. New `GIT_SSRF_SUBCOMMAND_FLAGS = ['--no-recurse-submodules']` is **subcommand-scoped**, spread AFTER the verb. Pre-v0.35.3 a single combined `GIT_SSRF_FLAGS` array spread `--no-recurse-submodules` before the verb where real git rejects it with exit 129 ("unknown option"); the fake-git test harness exited 0 regardless of argv shape, so CI missed it for ~7 months and every remote-source clone/pull was silently broken. `cloneRepo` argv: `git clone --depth=1 [--branch X] -- `. `pullRepo` argv: `git -C pull --ff-only`. Pinned by `test/git-remote.test.ts` position-anchored regression guard (`argv.indexOf('--no-recurse-submodules') > argv.indexOf(verb)`). - `src/commands/storage.ts` (v0.22.11) — `gbrain storage status [--repo P] [--json]`. Split into pure data (`getStorageStatus`) + JSON formatter + human formatter (ASCII-only per D10) matching the `orphans.ts` pattern. `PageCountsByTier` and `DiskUsageByTier` are distinct nominal types so swaps fail at compile time. - `gbrain.yml` (brain repo root, v0.22.11) — Optional storage tiering config. Top-level `storage:` section with `db_tracked:` and `db_only:` array-valued keys. `gbrain sync` auto-manages `.gitignore` for `db_only` paths on successful sync (skips on dry-run, blocked-by-failures, submodule context, or `GBRAIN_NO_GITIGNORE=1`). `gbrain export --restore-only [--repo P] [--type T] [--slug-prefix S]` repopulates missing `db_only` files from the database. @@ -401,7 +402,7 @@ strict behavior when unset. - `src/core/ai/model-resolver.ts:parseModelId` extension (v0.41.21.0) — gateway-side resolver extended to accept slash form alongside colon. Pre-fix the colon-only check threw `AIConfigError: model id must be in format provider:model` at every gateway entry point (chat / embed / rerank) the moment a slash-form id was passed. So even with the v0.41.21.0 pricing fix, a `--judge-model anthropic/claude-sonnet-4-6` invocation would clear BudgetTracker but then fail mid-judge inside `gateway.chat()`. Now both shapes resolve to the same recipe. Bare names without ANY separator still throw — gateway routing always needs an explicit provider. Pinned by 10 cases in `test/ai/model-resolver-slash.test.ts` including a `resolveRecipe` round-trip asserting slash form resolves to the same recipe object as colon form. - `src/commands/transcripts.ts` (v0.29) — `gbrain transcripts recent [--days N] [--full] [--json]`: recent raw `.txt` transcripts from the dream-cycle corpus dirs. Imports `listRecentTranscripts` from `src/core/transcripts.ts` (the same library the gated `get_recent_transcripts` MCP op uses). Local-only by construction — the CLI always runs with `ctx.remote=false`. - `src/commands/integrity.ts` — `gbrain integrity check|auto|review|extract`: bare-tweet detection, dead-link detection, three-bucket repair (auto-repair / review-queue / skip). `scanIntegrity()` is the shared library function called from `gbrain doctor` (sampled at limit=500) and `cmdCheck` (full scan). v0.22.8: batch-load fast path on Postgres uses a single SQL query to fix the PgBouncer round-trip timeout (60s → ~6s). Gated by `engine.kind === 'postgres'` at the call site so PGLite never enters batch; fallback `catch` logs at `GBRAIN_DEBUG=1` so real Postgres errors are diagnosable. **v0.32.8 (PR #860):** batch projection switched from `SELECT DISTINCT ON (slug)` to `SELECT ... ORDER BY source_id, slug` so multi-source brains scan each `(source, slug)` row independently (pre-fix the DISTINCT collapsed same-slug-different-source pages into one scan, the same bug class this PR fixes). Sequential and auto-repair loops use `listAllPageRefs()` to enumerate `(slug, source_id)` pairs and thread `sourceId` to `getPage`. Batch + sequential paths now report the same page count on multi-source brains. -- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel `. v0.22.12 (#500): `sync_failures` check shows `[CODE=N, ...]` breakdown for both unacked entries (warn) and acked-historical entries (ok), surfacing systemic failure modes (`SLUG_MISMATCH=2685`) instead of a bare count. v0.26.7 (#612): `rls_event_trigger` check (post-install drift detector for migration v35's auto-RLS event trigger). Lives outside the `// 5. RLS` slice that the structural doctor.test.ts guards anchor on, so the existing test guards stay intact. Healthy `evtenabled` set is `('O','A')` only — `R` is replica-only and would not fire in normal sessions; `D` is disabled. Fix hint is `gbrain apply-migrations --force-retry 35`. **v0.30.2:** `queue_health` gains a fourth subcheck — surfaces dead-lettered subagent jobs with `last_error` matching the `prompt_too_long` classifier within the last 24h. Fix hint points at `gbrain dream --phase synthesize --dry-run --json` to identify the offending transcript and `gbrain jobs prune --status dead --queue default` to clean up. Postgres-only. **v0.31.7:** `runDoctor` switches to `autoDetectSkillsDirReadOnly` (from `src/core/repo-root.ts`) so `bun install -g github:garrytan/gbrain && cd ~ && gbrain doctor` finds the bundled `skills/` via the install-path fallback instead of warning "Could not find skills directory" + docking the health score. `--fix` carries a D6 safety gate: when `detected.source === 'install_path'`, the command refuses auto-repair with a stderr message pointing at `$GBRAIN_SKILLS_DIR` / `$OPENCLAW_WORKSPACE` / `--skills-dir`, because `autoFixDryViolations` writes to SKILL.md files and would otherwise silently rewrite the install tree. The `graph_coverage` check now short-circuits to `ok: 'No entity pages — graph_coverage not applicable (markdown-only brain)'` when `SELECT COUNT(*) FROM pages WHERE type IN ('entity','person','company','organization')` returns 0 (closes #530); the entity count is woven into the warn message and the WARN hint switches from the long-deprecated `gbrain link-extract && gbrain timeline-extract` (gone since v0.16) to the canonical `gbrain extract all`. Pinned by an IRON-RULE regression assertion in `test/doctor.test.ts` that bans the stale verb names from the source string. **v0.32.4:** new `sync_freshness` check (exported `checkSyncFreshness` at the same file) added to both `runDoctor` (local) and `doctorReportRemote` (thin-client). Pure staleness probe — queries `sources.last_sync_at` only, no filesystem access. Warns at 24h, fails at 72h (or never-synced). Future-`last_sync_at` warns ("clock skew or corrupted timestamp") instead of silently falling through as ok — codex outside-voice caught the negative-ageMs bug pre-merge. Env-var overrides `GBRAIN_SYNC_FRESHNESS_WARN_HOURS` / `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS`; invalid values fall back to defaults with a once-per-process stderr warn (`_resolveSyncFreshnessHours`). Failure messages embed `source.id` (not `source.name`) so the printed fix command `gbrain sync --source ` matches what the user copy-pastes. Filesystem-vs-DB page drift detection was deliberately stripped from the v0.32.4 scope — `doctorReportRemote` runs in the HTTP MCP server (`src/commands/serve-http.ts`), and walking DB-supplied `local_path` from a remote-callable endpoint crosses a trust boundary (OAuth write scope could mutate `sources.local_path`). Drift detection will resurface in a separate PR routed through `multi_source_drift`'s existing guard infrastructure (`GBRAIN_DRIFT_LIMIT` / `GBRAIN_DRIFT_TIMEOUT_MS`) with slug normalization tests and a meta-file allow-list. Pinned by 12 cases in `test/doctor.test.ts` ("v0.32.4 — sync_freshness check" describe block): empty sources, never-synced fail, >72h fail, exact 72h boundary, 24h-72h warn, exact 24h boundary, <24h ok, future-timestamp warn, mixed sources (highest severity wins), `executeRaw` throws → outer-catch warn, `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS=6` override fires at 7h, source.id-in-message regression. **v0.36.3.0:** new `embedding_column_registry` check probes each declared column via Postgres `format_type(atttypid, atttypmod)` so a registry entry claiming 1024d Voyage against an actual 1536d OpenAI column surfaces with a paste-ready `gbrain config set embedding_columns '{...}'` ALTER hint instead of mysterious "vector dimension mismatch" errors at search time. On Postgres the check also probes HNSW index presence (`pg_indexes` lookup keyed by column name) and warns when missing (search will still work via seq scan but won't hit the index). The active default column's population coverage is computed via `COUNT(*) FILTER (WHERE IS NOT NULL) / COUNT(*)` and warns below 90% — except empty brains (chunk_count = 0) where the gate short-circuits to `ok` so fresh `gbrain init` runs don't see "Active column 'embedding' is 0.0% populated" (CDX-5 codex fix). PGLite parity via the same SQL through `executeRaw` — registry validation happens on both engines. **v0.35.5.0:** the Lane D supervisor check at `doctor.ts:1011-1043` now consumes `summarizeCrashes(events)` from `src/core/minions/handlers/supervisor-audit.ts` instead of the pre-fix `events.filter(e => e.event === 'worker_exited').length`. The warn threshold drops from `>3` to `>=1` (any real crash is signal now that the counter is calibrated against clean exits). The ok message gains `clean_exits_24h=N`; the warn message gains `runtime=A oom=B unknown=C legacy=D` per-cause breakdown so an operator triages OOM vs runtime-error vs unknown-future-cause at a glance without grep'ing the JSONL audit. Closes the "Supervisor crashes: 120x/24h, was 62x — nearly doubled" alarm class that bit users on healthy brains after v0.34.3.0's RSS-watchdog work added more code=0 worker drains — both `doctor` and `gbrain jobs supervisor status` were counting every `worker_exited` event as a crash regardless of cause. Cross-surface parity is the regression guard: 4 source-grep wiring assertions in `test/doctor.test.ts` ban the ad-hoc filter pattern, pin the `>=1` threshold, and require the per-cause breakdown substrings (`runtime=`, `oom=`, `unknown=`, `legacy=`, `clean_exits_24h=`, `crashes_by_cause`) to appear in BOTH `doctor.ts` and `jobs.ts`. **v0.37.1.0:** new `skill_brain_first` check. Walks every SKILL.md under the configured skills dir (`autoDetectSkillsDirReadOnly` so `cd ~ && gbrain doctor` finds the bundled skills via the install-path fallback), calls `analyzeSkillBrainFirst()` from `src/core/skill-brain-first.ts` per file, aggregates verdicts into a single check with structured `Check.issues[]` for JSON tooling. Warn states: `missing_brain_first` (external-lookup pattern present, no canonical callout, no `brain_first: exempt`), `brain_first_typo` (near-miss declaration like `brain-first` or `BrainFirst` — paste-ready hint surfaces the correct snake_case form). Ok states: `compliant_callout`, `compliant_phase`, `compliant_position`, `exempt_frontmatter`, `no_external`. `--fix` routes through `dry-fix.ts` MISSING_RULE_PATTERNS to auto-insert the canonical `> **Convention:** see [conventions/brain-first.md](...)` callout (D6 install-path safety gate enforced — `--fix` from `~` refuses to write to the bundled tree). Snapshot+diff audit at `~/.gbrain/audit/skill-brain-first-YYYY-Www.jsonl` records detected / resolved / fixed transitions only (stable brains: 0 lines/run). Motivated by the 2026-05-19 tweet-shield incident: cross-modal eval flagged Garry's Palantir tweet as risky because no model knew he built it, but the brain already had "designed the entire Finance product UI" and "150+ PSDs from April-December 2006." Static check catches the AUTHORSHIP miss class; v0.37+ runtime gate (filed in TODOS.md) closes the dispatch side. +- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel `. v0.22.12 (#500): `sync_failures` check shows `[CODE=N, ...]` breakdown for both unacked entries (warn) and acked-historical entries (ok), surfacing systemic failure modes (`SLUG_MISMATCH=2685`) instead of a bare count. v0.26.7 (#612): `rls_event_trigger` check (post-install drift detector for migration v35's auto-RLS event trigger). Lives outside the `// 5. RLS` slice that the structural doctor.test.ts guards anchor on, so the existing test guards stay intact. Healthy `evtenabled` set is `('O','A')` only — `R` is replica-only and would not fire in normal sessions; `D` is disabled. Fix hint is `gbrain apply-migrations --force-retry 35`. **v0.30.2:** `queue_health` gains a fourth subcheck — surfaces dead-lettered subagent jobs with `last_error` matching the `prompt_too_long` classifier within the last 24h. Fix hint points at `gbrain dream --phase synthesize --dry-run --json` to identify the offending transcript and `gbrain jobs prune --status dead --queue default` to clean up. Postgres-only. **v0.31.7:** `runDoctor` switches to `autoDetectSkillsDirReadOnly` (from `src/core/repo-root.ts`) so `bun install -g github:garrytan/gbrain && cd ~ && gbrain doctor` finds the bundled `skills/` via the install-path fallback instead of warning "Could not find skills directory" + docking the health score. `--fix` carries a D6 safety gate: when `detected.source === 'install_path'`, the command refuses auto-repair with a stderr message pointing at `$GBRAIN_SKILLS_DIR` / `$OPENCLAW_WORKSPACE` / `--skills-dir`, because `autoFixDryViolations` writes to SKILL.md files and would otherwise silently rewrite the install tree. The `graph_coverage` check now short-circuits to `ok: 'No entity pages — graph_coverage not applicable (markdown-only brain)'` when `SELECT COUNT(*) FROM pages WHERE type IN ('entity','person','company','organization')` returns 0 (closes #530); the entity count is woven into the warn message and the WARN hint switches from the long-deprecated `gbrain link-extract && gbrain timeline-extract` (gone since v0.16) to the canonical `gbrain extract all`. Pinned by an IRON-RULE regression assertion in `test/doctor.test.ts` that bans the stale verb names from the source string. **v0.32.4:** new `sync_freshness` check (exported `checkSyncFreshness` at the same file) added to both `runDoctor` (local) and `doctorReportRemote` (thin-client). Pure staleness probe — queries `sources.last_sync_at` only, no filesystem access. Warns at 24h, fails at 72h (or never-synced). Future-`last_sync_at` warns ("clock skew or corrupted timestamp") instead of silently falling through as ok — codex outside-voice caught the negative-ageMs bug pre-merge. Env-var overrides `GBRAIN_SYNC_FRESHNESS_WARN_HOURS` / `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS`; invalid values fall back to defaults with a once-per-process stderr warn (`_resolveSyncFreshnessHours`). Failure messages embed `source.id` (not `source.name`) so the printed fix command `gbrain sync --source ` matches what the user copy-pastes. Filesystem-vs-DB page drift detection was deliberately stripped from the v0.32.4 scope — `doctorReportRemote` runs in the HTTP MCP server (`src/commands/serve-http.ts`), and walking DB-supplied `local_path` from a remote-callable endpoint crosses a trust boundary (OAuth write scope could mutate `sources.local_path`). Drift detection will resurface in a separate PR routed through `multi_source_drift`'s existing guard infrastructure (`GBRAIN_DRIFT_LIMIT` / `GBRAIN_DRIFT_TIMEOUT_MS`) with slug normalization tests and a meta-file allow-list. Pinned by 12 cases in `test/doctor.test.ts` ("v0.32.4 — sync_freshness check" describe block): empty sources, never-synced fail, >72h fail, exact 72h boundary, 24h-72h warn, exact 24h boundary, <24h ok, future-timestamp warn, mixed sources (highest severity wins), `executeRaw` throws → outer-catch warn, `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS=6` override fires at 7h, source.id-in-message regression. **v0.36.3.0:** new `embedding_column_registry` check probes each declared column via Postgres `format_type(atttypid, atttypmod)` so a registry entry claiming 1024d Voyage against an actual 1536d OpenAI column surfaces with a paste-ready `gbrain config set embedding_columns '{...}'` ALTER hint instead of mysterious "vector dimension mismatch" errors at search time. On Postgres the check also probes HNSW index presence (`pg_indexes` lookup keyed by column name) and warns when missing (search will still work via seq scan but won't hit the index). The active default column's population coverage is computed via `COUNT(*) FILTER (WHERE IS NOT NULL) / COUNT(*)` and warns below 90% — except empty brains (chunk_count = 0) where the gate short-circuits to `ok` so fresh `gbrain init` runs don't see "Active column 'embedding' is 0.0% populated" (CDX-5 codex fix). PGLite parity via the same SQL through `executeRaw` — registry validation happens on both engines. **v0.35.5.0:** the Lane D supervisor check at `doctor.ts:1011-1043` now consumes `summarizeCrashes(events)` from `src/core/minions/handlers/supervisor-audit.ts` instead of the pre-fix `events.filter(e => e.event === 'worker_exited').length`. The warn threshold drops from `>3` to `>=1` (any real crash is signal now that the counter is calibrated against clean exits). The ok message gains `clean_exits_24h=N`; the warn message gains `runtime=A oom=B unknown=C legacy=D` per-cause breakdown so an operator triages OOM vs runtime-error vs unknown-future-cause at a glance without grep'ing the JSONL audit. Closes the "Supervisor crashes: 120x/24h, was 62x — nearly doubled" alarm class that bit users on healthy brains after v0.34.3.0's RSS-watchdog work added more code=0 worker drains — both `doctor` and `gbrain jobs supervisor status` were counting every `worker_exited` event as a crash regardless of cause. Cross-surface parity is the regression guard: 4 source-grep wiring assertions in `test/doctor.test.ts` ban the ad-hoc filter pattern, pin the `>=1` threshold, and require the per-cause breakdown substrings (`runtime=`, `oom=`, `unknown=`, `legacy=`, `clean_exits_24h=`, `crashes_by_cause`) to appear in BOTH `doctor.ts` and `jobs.ts`. **v0.37.1.0:** new `skill_brain_first` check. Walks every SKILL.md under the configured skills dir (`autoDetectSkillsDirReadOnly` so `cd ~ && gbrain doctor` finds the bundled skills via the install-path fallback), calls `analyzeSkillBrainFirst()` from `src/core/skill-brain-first.ts` per file, aggregates verdicts into a single check with structured `Check.issues[]` for JSON tooling. Warn states: `missing_brain_first` (external-lookup pattern present, no canonical callout, no `brain_first: exempt`), `brain_first_typo` (near-miss declaration like `brain-first` or `BrainFirst` — paste-ready hint surfaces the correct snake_case form). Ok states: `compliant_callout`, `compliant_phase`, `compliant_position`, `exempt_frontmatter`, `no_external`. `--fix` routes through `dry-fix.ts` MISSING_RULE_PATTERNS to auto-insert the canonical `> **Convention:** see [conventions/brain-first.md](...)` callout (D6 install-path safety gate enforced — `--fix` from `~` refuses to write to the bundled tree). Snapshot+diff audit at `~/.gbrain/audit/skill-brain-first-YYYY-Www.jsonl` records detected / resolved / fixed transitions only (stable brains: 0 lines/run). Motivated by the 2026-05-19 tweet-shield incident: cross-modal eval flagged Garry's Palantir tweet as risky because no model knew he built it, but the brain already had "designed the entire Finance product UI" and "150+ PSDs from April-December 2006." Static check catches the AUTHORSHIP miss class; v0.37+ runtime gate (filed in TODOS.md) closes the dispatch side. **v0.41.27.0:** `checkSyncFreshness` gains a `localOnly`-gated git short-circuit (D4 trust-boundary preservation per Codex P0-1): `runDoctor` passes `localOnly: true`, `doctorReportRemote` keeps the default `false` so the HTTP MCP path doesn't walk DB-supplied `local_path` values via subprocess. The narrowed predicate (D7) mirrors sync.ts:1057+1075's actual "do work?" gate: HEAD == `last_commit` AND working tree clean AND `sources.chunker_version === String(CHUNKER_VERSION)`. Inline SELECT widens to carry `last_commit + chunker_version` (columns already exist; no schema migration). Three-bucket count math (D6) populates `Check.details = {unchanged_count, synced_recently_count, stale_count}` with the invariant `sum === sources.length` pinned in the test suite. OK message reshape (D2): all-unchanged → "All N up to date (no new commits since last sync)"; mixed → "N source(s): X synced recently, Y unchanged since last sync"; all-synced-recently keeps the prior message verbatim. `checkCycleFreshness` is INTENTIONALLY NOT touched (Codex P0-2 / D5): `last_commit == HEAD` answers "are there new commits to sync?" but cannot answer "did the full cycle complete?" — a sync can succeed while later cycle phases fail, and silencing that warn would hide real cycle staleness. Pinned by 9 new cases in `test/doctor.test.ts` ("v0.41.27.0 — sync_freshness git short-circuit" describe block) including a load-bearing D4 regression guard that verifies probes are NEVER called when `localOnly` is unset or false, plus the three-bucket invariant explicitly asserted in the mixed 3-source case. Supersedes PR #1564 (Co-Authored-By preserved). - `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). **v40 (v0.29):** `pages_emotional_weight` adds `pages.emotional_weight REAL NOT NULL DEFAULT 0.0`. Column-only (no index). On Postgres 11+ and PGLite, `ADD COLUMN` with a constant DEFAULT is metadata-only — instant on tables of any size. v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2). **v30 (v0.23):** creates `dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PK(file_path, content_hash))`. RLS-enabled when running as a BYPASSRLS role. The synthesize phase reads/writes this table to avoid re-judging on backfill re-runs. **v35 (v0.26.7):** auto-RLS event trigger + one-time backfill. `auto_rls_on_create_table` fires on `ddl_command_end` for `WHEN TAG IN ('CREATE TABLE','CREATE TABLE AS','SELECT INTO')` and runs `ALTER TABLE … ENABLE ROW LEVEL SECURITY` on every new `public.*` table — no FORCE (matches v24/v29/schema.sql posture so non-BYPASSRLS apps can still read their own tables). The same migration backfills RLS on every existing `public.*` base table whose comment doesn't match the doctor regex (`^GBRAIN:RLS_EXEMPT\s+reason=\S.{3,}`). Per-table failure aborts the offending CREATE TABLE (event triggers fire inside the DDL transaction); no EXCEPTION wrap — that would convert loud rollback into silent permissive default. PGLite no-op via `sqlFor.pglite: ''`. Breaking change: operators with intentionally-RLS-off public tables must add the GBRAIN:RLS_EXEMPT comment BEFORE upgrade or the backfill will flip them on. **v46 (v0.31.3):** `mcp_request_log_params_jsonb_normalize` rewrites pre-v0.31.3 rows where `mcp_request_log.params` was stored as a JSON-encoded string (`jsonb_typeof = 'string'`) up to a real JSONB object via `UPDATE ... SET params = params::text::jsonb WHERE jsonb_typeof(params) = 'string'`. Single statement, idempotent — second-run finds no string-shaped rows and is a no-op. Closes the bug where `/admin/api/requests` returned a quoted string instead of the parsed object. **v0.36.3.0 (v68):** `eval_candidates_embedding_column` adds `eval_candidates.embedding_column TEXT NULL`. Per-row provenance for `gbrain eval replay`: captured rows record which column the live query ran against so replay reproduces the same retrieval space (Voyage rows replay against Voyage; OpenAI rows against OpenAI). NULL-tolerant — pre-v0.36 rows fall back to the current default during replay rather than failing. Column-only migration, metadata-only on both engines. **v0.34.1.0 (#861 + #876, v60-v65):** six-migration chain wires source-scoping into the OAuth client table. v60 (`oauth_clients_source_id_fk`) adds `oauth_clients.source_id TEXT` with NULL→`'default'` backfill and an FK to `sources(id) ON DELETE SET NULL`. v61 (`oauth_clients_federated_read_column`) adds `federated_read TEXT[] NOT NULL DEFAULT '{}'`. v62 (`oauth_clients_federated_read_backfill`) explicit-CASE backfills so `source_id IS NULL` produces `'{}'` not an array-containing-NULL. v63 (`oauth_clients_federated_read_validate`) is the fail-loud check that every row's source_id is in its federated_read array post-backfill. v64 (`oauth_clients_source_id_fk_restrict`) flips the FK to `ON DELETE RESTRICT` now that federated_read provides the alternative scope-loss path — source delete is refused if any client references it. v65 (`oauth_clients_federated_read_gin_index`) is the GIN index for the array-containment queries the read paths run. PGLite parity via `sqlFor.pglite` where needed. - `src/core/progress.ts` — Shared bulk-action progress reporter. Writes to stderr. Modes: `auto` (TTY: `\r`-rewriting; non-TTY: plain lines), `human`, `json` (JSONL), `quiet`. Rate-gated by `minIntervalMs` and `minItems`. `startHeartbeat(reporter, note)` helper for single long queries. `child()` composes phase paths. Singleton SIGINT/SIGTERM coordinator emits `abort` events for every live phase. EPIPE defense on both sync throws and stream `'error'` events. Zero dependencies. Introduced in v0.15.2. **v0.40.3.0:** `emitHumanLine` is prefix-aware — when called inside a `withSourcePrefix(id, ...)` scope from `src/core/console-prefix.ts`, prepends `[id] ` to the line content. TTY-rewrite mode (`\r\x1b[2K`) gets the prefix inside the clear-to-EOL escape so the rewritten line carries the prefix too. JSON mode (`emitJson`) is intentionally NOT prefixed — NDJSON consumers parse the JSON envelope and would choke on a `[id] {...}` shape. - `src/core/console-prefix.ts` (v0.40.3.0) — `AsyncLocalStorage`-backed per-source line-prefix helper. Exports `withSourcePrefix(id, fn)` (runs `fn` with `id` as the active prefix; nested wraps replace the active prefix and restore on exit), `getSourcePrefix()` (read-only accessor; test seam), and `slog(...)` / `serr(...)` (prefix-aware `console.log` / `console.error` replacements). Embedded-newline-safe: a multi-line string under prefix `[foo]` emits `[foo] line1\n[foo] line2`. Outside a wrap, `slog` / `serr` fall through to bare `console.log` / `console.error` so single-source callers see identical output (back-compat invariant). Use `src.id` (slug-validated by `sources add`) NOT `src.name` (free-form text) to defeat log-injection through newline / control-character names. Coverage in v0.40.3.0: `src/commands/sync.ts` performSync + in-file callees (38 call sites migrated), `src/commands/embed.ts` runEmbedCore + helpers (16 call sites), `src/core/progress.ts` emitHumanLine. Lines emitted from outside these modules will NOT get the prefix under parallel sync — file an issue if you find a missed migration target. diff --git a/src/commands/doctor.ts b/src/commands/doctor.ts index 4cb32566b..b49e6fbe8 100644 --- a/src/commands/doctor.ts +++ b/src/commands/doctor.ts @@ -27,6 +27,8 @@ import { gbrainPath } from '../core/config.ts'; import { dirname, isAbsolute, join, resolve as resolvePath } from 'path'; import { fileURLToPath } from 'url'; import { existsSync, readFileSync, readdirSync, statSync } from 'fs'; +import { isSourceUnchangedSinceSync } from '../core/git-head.ts'; +import { CHUNKER_VERSION } from '../core/chunkers/code.ts'; export interface Check { name: string; @@ -2671,16 +2673,23 @@ export async function computeExtractHealthCheck( export async function checkSyncFreshness( engine: BrainEngine, - opts?: { nowMs?: number }, + opts?: { nowMs?: number; localOnly?: boolean }, ): Promise { try { + // v0.41.27.0: SELECT widens to carry last_commit + chunker_version so + // the git short-circuit gate (below) can compare against what + // `gbrain sync`'s up-to-date predicate at sync.ts:1057+1075 checks. + // Columns existed pre-v0.41 (writeSyncAnchor / writeChunkerVersion); + // no schema migration needed. const sources = await engine.executeRaw<{ id: string; name: string; local_path: string | null; last_sync_at: Date | null; + last_commit: string | null; + chunker_version: string | null; }>( - `SELECT id, name, local_path, last_sync_at FROM sources WHERE local_path IS NOT NULL`, + `SELECT id, name, local_path, last_sync_at, last_commit, chunker_version FROM sources WHERE local_path IS NOT NULL`, ); if (sources.length === 0) { @@ -2688,6 +2697,7 @@ export async function checkSyncFreshness( name: 'sync_freshness', status: 'ok', message: 'No federated sources to sync', + details: { unchanged_count: 0, synced_recently_count: 0, stale_count: 0 }, }; } @@ -2703,7 +2713,30 @@ export async function checkSyncFreshness( // status from warn to fail (CI-flaky, see PR #1138 ship). Production // callers omit `nowMs` and get live wall-clock semantics. const now = opts?.nowMs ?? Date.now(); + + // v0.41.27.0: D4 trust boundary. The git short-circuit runs ONLY when + // the caller explicitly opts in via `localOnly: true`. Default (false) + // preserves the v0.32.4 trust boundary for `doctorReportRemote` (the + // HTTP MCP path) — a remote-callable code path must NOT walk + // DB-supplied `local_path` values with subprocess calls. runDoctor + // (local CLI) passes true; doctorReportRemote keeps the default. + const localOnly = opts?.localOnly === true; + + // v0.41.27.0: D7 narrowed predicate. The CHUNKER_VERSION caller-side + // check mirrors sync.ts:1057's chunker-version gate so doctor agrees + // with sync on "is there work to do?". `sources.chunker_version` is + // a TEXT column storing String(CHUNKER_VERSION). + const currentChunkerVersion = String(CHUNKER_VERSION); + const issues: string[] = []; + // v0.41.27.0: D6 three-bucket count math. Every source falls into + // EXACTLY ONE bucket per iteration. Invariant pinned by unit test: + // unchanged_count + synced_recently_count + stale_count === sources.length + // Stale subsumes warn + fail + never-synced + future-timestamp; we keep + // hasWarnings/hasFailures for the existing return-status logic. + let unchanged_count = 0; + let synced_recently_count = 0; + let stale_count = 0; let hasWarnings = false; let hasFailures = false; @@ -2717,6 +2750,7 @@ export async function checkSyncFreshness( if (!source.last_sync_at) { issues.push(`Source ${display} has never been synced`); hasFailures = true; + stale_count++; continue; } @@ -2728,26 +2762,56 @@ export async function checkSyncFreshness( `Source ${display} has future last_sync_at — clock skew or corrupted timestamp`, ); hasWarnings = true; + stale_count++; continue; } + // v0.41.27.0: git short-circuit (D4 + D7 combined). Only fires when: + // 1. caller opted in via localOnly=true (trust boundary) + // 2. HEAD === last_commit (no new commits to sync) + // 3. working tree is clean (no uncommitted edits sync would re-walk) + // 4. chunker_version matches CURRENT (no post-upgrade re-chunk pending) + // All four must hold; otherwise fall through to the time-based check. + // The chunker version match is computed here (not in the helper) + // because it depends on engine state, not git state. + if (localOnly) { + const gitUnchanged = isSourceUnchangedSinceSync( + source.local_path, + source.last_commit, + { requireCleanWorkingTree: true }, + ); + const chunkerMatch = source.chunker_version === currentChunkerVersion; + if (gitUnchanged && chunkerMatch) { + unchanged_count++; + continue; + } + } + const ageHours = Math.floor(ageMs / (1000 * 60 * 60)); const ageDays = Math.floor(ageHours / 24); if (ageMs > failMs) { issues.push(`Source ${display} last synced ${ageDays}d ago — brain search is stale!`); hasFailures = true; + stale_count++; } else if (ageMs > warnMs) { issues.push(`Source ${display} last synced ${ageHours}h ago`); hasWarnings = true; + stale_count++; + } else { + synced_recently_count++; } } + // D6 invariant: every source incremented exactly one bucket. + const details = { unchanged_count, synced_recently_count, stale_count }; + if (hasFailures) { return { name: 'sync_freshness', status: 'fail', message: `${issues.join('; ')}. Run \`gbrain sync --source \` for each stale source`, + details, }; } if (hasWarnings) { @@ -2755,12 +2819,33 @@ export async function checkSyncFreshness( name: 'sync_freshness', status: 'warn', message: `${issues.join('; ')}. Run \`gbrain sync --source \` to refresh`, + details, + }; + } + // v0.41.27.0: D2 ok-message reshape. Three branches surface what the + // git short-circuit actually did so operators understand "unchanged + // since last sync" vs "synced recently". + if (unchanged_count === sources.length) { + return { + name: 'sync_freshness', + status: 'ok', + message: `All ${sources.length} federated source(s) up to date (no new commits since last sync)`, + details, + }; + } + if (unchanged_count > 0) { + return { + name: 'sync_freshness', + status: 'ok', + message: `${sources.length} federated source(s): ${synced_recently_count} synced recently, ${unchanged_count} unchanged since last sync`, + details, }; } return { name: 'sync_freshness', status: 'ok', message: `All ${sources.length} federated source(s) synced recently`, + details, }; } catch (e) { return { @@ -5713,7 +5798,13 @@ export async function buildChecks( // Sync freshness check (v0.32 — Check that sources are synced recently) if (engine !== null) { progress.heartbeat('sync_freshness'); - checks.push(await checkSyncFreshness(engine)); + // v0.41.27.0 D4: local CLI path is trusted to walk DB-supplied + // local_path values via subprocess (we own the brain repo). Pass + // localOnly:true so the git short-circuit fires. The HTTP MCP path + // at doctorReportRemote (around line 662) deliberately keeps the + // default (false) — that's the trust-boundary preservation Codex + // P0-1 flagged. + checks.push(await checkSyncFreshness(engine, { localOnly: true })); // v0.41.19.0 (Issue 5): sync --all consolidation nudge. progress.heartbeat('sync_consolidation'); checks.push(await checkSyncConsolidation(engine)); diff --git a/src/core/git-head.ts b/src/core/git-head.ts new file mode 100644 index 000000000..1e43b48bc --- /dev/null +++ b/src/core/git-head.ts @@ -0,0 +1,110 @@ +/** + * v0.41.27.0 — git HEAD freshness probe for `gbrain doctor`. + * + * Single primitive. Returns true iff a `local_path` directory is a git repo + * whose current HEAD matches the `last_commit` SHA the DB recorded at last + * sync completion. When `requireCleanWorkingTree` is set, also requires the + * working tree to be clean — mirroring `gbrain sync`'s force-walk gate at + * sync.ts:1075 so doctor and sync agree on "is there work to do?". + * + * Fail-open contract: any error (missing path, not a git repo, git not + * installed, timeout, NULL inputs, dirty probe errored) returns `false`, + * which preserves the caller's prior time-based behavior. We never raise. + * + * Shell-injection safe: uses execFileSync with array args so a `local_path` + * containing `$(...)`, backticks, or other shell metacharacters can never + * escape to a shell. The PR #1564 community version used + * `execSync(`git -C ${JSON.stringify(path)} ...`)`, which runs through + * `/bin/sh -c` — `JSON.stringify` escapes for JSON, not shell, so a + * mutable `sources.local_path` was an RCE-style surface. + * + * Designed for reuse: autopilot's per-source dispatch will want the same + * gate. See plan note "v0.41.27.1+ TODOs" in + * ~/.claude/plans/system-instruction-you-are-working-eager-bird.md. + */ +import { execFileSync } from 'node:child_process'; + +export type GitHeadProbe = (localPath: string) => string | null; +// `null` distinguishes probe error from known-dirty (false). Doctor treats +// both as "do not short-circuit", but tests need to assert which path fired. +export type GitCleanProbe = (localPath: string) => boolean | null; + +const DEFAULT_HEAD_PROBE: GitHeadProbe = (localPath) => { + try { + const out = execFileSync('git', ['-C', localPath, 'rev-parse', 'HEAD'], { + encoding: 'utf8', + timeout: 5000, + stdio: ['ignore', 'pipe', 'ignore'], + }); + return out.trim() || null; + } catch { + return null; + } +}; + +const DEFAULT_CLEAN_PROBE: GitCleanProbe = (localPath) => { + try { + const out = execFileSync('git', ['-C', localPath, 'status', '--porcelain'], { + encoding: 'utf8', + timeout: 5000, + stdio: ['ignore', 'pipe', 'ignore'], + }); + return out.trim().length === 0; + } catch { + return null; + } +}; + +let _headProbe: GitHeadProbe = DEFAULT_HEAD_PROBE; +let _cleanProbe: GitCleanProbe = DEFAULT_CLEAN_PROBE; + +// Test seam. Matches the `_setChatTransportForTests` precedent at +// src/core/last-retrieved.ts so tests can drive the public function +// without mocking child_process or routing through mock.module (R2-compliant). +export function _setGitHeadProbeForTests(fn: GitHeadProbe | null): void { + _headProbe = fn ?? DEFAULT_HEAD_PROBE; +} + +export function _setGitCleanProbeForTests(fn: GitCleanProbe | null): void { + _cleanProbe = fn ?? DEFAULT_CLEAN_PROBE; +} + +export interface GitFreshnessOpts { + /** + * When true, additionally require working tree to be clean (no + * uncommitted changes). Doctor uses this to mirror `gbrain sync`'s + * working-tree-dirty gate at sync.ts:1075 — otherwise doctor would + * say "unchanged" for a repo with pending local edits that sync would + * actually re-walk on the next run. + * + * Default false (HEAD comparison only). Doctor-callers set true. + */ + requireCleanWorkingTree?: boolean; +} + +/** + * Returns true iff `localPath` is a git repo whose current HEAD matches + * `lastCommit`, AND (when `requireCleanWorkingTree`) the working tree + * is clean. + * + * This is NOT a full mirror of `gbrain sync`'s "do work?" predicate. + * Chunker-version match is computed by the caller because it depends on + * engine state (`sources.chunker_version` vs `CURRENT_CHUNKER_VERSION`). + * See `src/commands/doctor.ts:checkSyncFreshness` for the AND + * combination at the call site. + */ +export function isSourceUnchangedSinceSync( + localPath: string | null | undefined, + lastCommit: string | null | undefined, + opts?: GitFreshnessOpts, +): boolean { + if (!localPath || !lastCommit) return false; + const head = _headProbe(localPath); + if (head === null || head !== lastCommit) return false; + if (opts?.requireCleanWorkingTree) { + const isClean = _cleanProbe(localPath); + // null (probe error) AND false (known dirty) both fail the gate. + if (isClean !== true) return false; + } + return true; +} diff --git a/test/core/git-head.test.ts b/test/core/git-head.test.ts new file mode 100644 index 000000000..a84251143 --- /dev/null +++ b/test/core/git-head.test.ts @@ -0,0 +1,178 @@ +// v0.41.27.0 — src/core/git-head.ts unit tests. +// +// Hermetic: no engine, no PGLite, no DATABASE_URL required. Probe seam lets +// the suite drive isSourceUnchangedSinceSync without spawning real git (cases +// 1-8, 10-12); case 9 is the shell-injection regression guard that DOES use +// the real execFileSync against a deliberately adversarial localPath to prove +// the array-arg call shape cannot escape to a shell. + +import { describe, expect, test, beforeEach, afterAll } from 'bun:test'; +import { existsSync, mkdtempSync, rmSync, unlinkSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { + isSourceUnchangedSinceSync, + _setGitHeadProbeForTests, + _setGitCleanProbeForTests, + type GitHeadProbe, + type GitCleanProbe, +} from '../../src/core/git-head.ts'; + +// Reset both probe seams between every test so case order can't leak state. +beforeEach(() => { + _setGitHeadProbeForTests(null); + _setGitCleanProbeForTests(null); +}); + +afterAll(() => { + // Restore defaults so other test files inheriting the module state are clean. + _setGitHeadProbeForTests(null); + _setGitCleanProbeForTests(null); +}); + +describe('isSourceUnchangedSinceSync — basic predicate', () => { + test('case 1: happy path — HEAD matches, no requireCleanWorkingTree → true', () => { + _setGitHeadProbeForTests(() => 'abc123'); + expect(isSourceUnchangedSinceSync('/tmp/repo', 'abc123')).toBe(true); + }); + + test('case 2: HEAD differs from lastCommit → false', () => { + _setGitHeadProbeForTests(() => 'def456'); + expect(isSourceUnchangedSinceSync('/tmp/repo', 'abc123')).toBe(false); + }); + + test('case 3: localPath null — short-circuits without calling head probe', () => { + let probeCalls = 0; + _setGitHeadProbeForTests(() => { probeCalls++; return 'abc'; }); + expect(isSourceUnchangedSinceSync(null, 'abc123')).toBe(false); + expect(probeCalls).toBe(0); + }); + + test('case 4: localPath empty string — short-circuits, probe not called', () => { + let probeCalls = 0; + _setGitHeadProbeForTests(() => { probeCalls++; return 'abc'; }); + expect(isSourceUnchangedSinceSync('', 'abc123')).toBe(false); + expect(probeCalls).toBe(0); + }); + + test('case 5: lastCommit null — short-circuits, probe not called', () => { + let probeCalls = 0; + _setGitHeadProbeForTests(() => { probeCalls++; return 'abc'; }); + expect(isSourceUnchangedSinceSync('/tmp/repo', null)).toBe(false); + expect(probeCalls).toBe(0); + }); + + test('case 6: lastCommit empty string — short-circuits, probe not called', () => { + let probeCalls = 0; + _setGitHeadProbeForTests(() => { probeCalls++; return 'abc'; }); + expect(isSourceUnchangedSinceSync('/tmp/repo', '')).toBe(false); + expect(probeCalls).toBe(0); + }); + + test('case 7: head probe returns null (non-git dir / git not installed) → false', () => { + _setGitHeadProbeForTests(() => null); + expect(isSourceUnchangedSinceSync('/tmp/not-a-repo', 'abc123')).toBe(false); + }); + + test('case 8: head probe throws synchronously → false (fail-open)', () => { + _setGitHeadProbeForTests(() => { throw new Error('git not installed'); }); + expect(() => isSourceUnchangedSinceSync('/tmp/repo', 'abc123')).toThrow(); + // The default behavior is fail-open via the probe's own try/catch + // (returning null). A test seam that throws bypasses that protection + // intentionally — production callers use the default probe which + // swallows the error. Documented at git-head.ts:35-41. + // Re-verify by stubbing a probe that swallows the underlying error: + _setGitHeadProbeForTests(() => { try { throw new Error('inner'); } catch { return null; } }); + expect(isSourceUnchangedSinceSync('/tmp/repo', 'abc123')).toBe(false); + }); +}); + +describe('isSourceUnchangedSinceSync — shell-injection regression guard', () => { + test('case 9: REAL execFileSync against adversarial localPath does NOT execute shell metachars', () => { + // Use the REAL default probe (no test seam) so this case exercises the + // production code path: execFileSync('git', ['-C', path, ...]). If the + // implementation ever regresses to execSync with shell interpolation, + // the `$(...)` substring would execute and create the sentinel file. + _setGitHeadProbeForTests(null); + _setGitCleanProbeForTests(null); + + const sentinelDir = mkdtempSync(join(tmpdir(), 'git-head-sentinel-')); + const sentinelPath = join(sentinelDir, 'pwned'); + const adversarialPath = `/nonexistent/$(touch ${sentinelPath})/repo`; + + try { + // Should fail-open: the path doesn't exist, git rev-parse returns + // non-zero, the probe returns null, the helper returns false. + const result = isSourceUnchangedSinceSync(adversarialPath, 'abc123'); + expect(result).toBe(false); + // The load-bearing assertion: if shell interpolation happened, + // `touch ${sentinelPath}` would have created the file. It must NOT + // exist. + expect(existsSync(sentinelPath)).toBe(false); + } finally { + // Cleanup. If the test failed-open AND created the sentinel, remove it. + if (existsSync(sentinelPath)) { + try { unlinkSync(sentinelPath); } catch { /* ignore */ } + } + try { rmSync(sentinelDir, { recursive: true, force: true }); } catch { /* ignore */ } + } + }); +}); + +describe('isSourceUnchangedSinceSync — test-seam round-trip', () => { + test('case 10: _setGitHeadProbeForTests + _setGitCleanProbeForTests round-trip', () => { + // Install custom probes + const customHead: GitHeadProbe = () => 'custom-head'; + const customClean: GitCleanProbe = () => true; + _setGitHeadProbeForTests(customHead); + _setGitCleanProbeForTests(customClean); + + expect(isSourceUnchangedSinceSync('/x', 'custom-head', { requireCleanWorkingTree: true })).toBe(true); + + // Restore defaults via null + _setGitHeadProbeForTests(null); + _setGitCleanProbeForTests(null); + + // After restoration, the real probes run. Against a non-git path, + // both probes return null → predicate returns false. This proves the + // restore worked without depending on a git repo in the test env. + expect(isSourceUnchangedSinceSync('/definitely-not-a-git-repo-xyz', 'custom-head')).toBe(false); + }); +}); + +describe('isSourceUnchangedSinceSync — requireCleanWorkingTree (D7)', () => { + test('case 11: HEAD match + clean tree + requireCleanWorkingTree=true → true', () => { + _setGitHeadProbeForTests(() => 'abc123'); + _setGitCleanProbeForTests(() => true); + expect( + isSourceUnchangedSinceSync('/tmp/repo', 'abc123', { requireCleanWorkingTree: true }), + ).toBe(true); + }); + + test('case 12a: HEAD match + DIRTY tree + requireCleanWorkingTree=true → false', () => { + _setGitHeadProbeForTests(() => 'abc123'); + _setGitCleanProbeForTests(() => false); + expect( + isSourceUnchangedSinceSync('/tmp/repo', 'abc123', { requireCleanWorkingTree: true }), + ).toBe(false); + }); + + test('case 12b: HEAD match + clean probe ERRORED (returns null) + requireCleanWorkingTree=true → false', () => { + // null is distinct from false: probe error vs known-dirty. Both fail + // the gate (fail-closed posture for the clean check, fail-open posture + // for the helper as a whole — caller's time-based check still runs). + _setGitHeadProbeForTests(() => 'abc123'); + _setGitCleanProbeForTests(() => null); + expect( + isSourceUnchangedSinceSync('/tmp/repo', 'abc123', { requireCleanWorkingTree: true }), + ).toBe(false); + }); + + test('case 12c: HEAD match + tree dirty BUT requireCleanWorkingTree NOT set → true (clean probe not consulted)', () => { + let cleanCalls = 0; + _setGitHeadProbeForTests(() => 'abc123'); + _setGitCleanProbeForTests(() => { cleanCalls++; return false; }); + expect(isSourceUnchangedSinceSync('/tmp/repo', 'abc123')).toBe(true); + expect(cleanCalls).toBe(0); + }); +}); diff --git a/test/doctor.test.ts b/test/doctor.test.ts index 088d06f46..a67c0a093 100644 --- a/test/doctor.test.ts +++ b/test/doctor.test.ts @@ -585,6 +585,263 @@ describe('v0.32.4 — sync_freshness check', () => { }); }); +// ============================================================================ +// v0.41.27.0 — sync_freshness git short-circuit (D4 + D6 + D7) +// ============================================================================ +// Doctor learns to skip the staleness warning when a git-backed source has no +// new commits since the last sync AND working tree is clean AND chunker +// version matches. Trust boundary preserved via opts.localOnly (D4); count +// math fixed with three buckets that sum to sources.length (D6); narrowed +// predicate mirrors sync.ts:1057+1075 (D7). +// ============================================================================ + +describe('v0.41.27.0 — sync_freshness git short-circuit', () => { + // Reuse the stub-engine pattern from v0.32.4 describe above. Row shape now + // includes last_commit + chunker_version (extended SELECT in v0.41.27.0). + function makeStubEngine(rows: any[]): any { + return { executeRaw: async () => rows }; + } + function agoMs(ms: number): Date { + return new Date(Date.now() - ms); + } + + // Probe seams come from src/core/git-head.ts. Reset between each test so + // case order can't leak state. CURRENT is imported from chunkers/code.ts — + // tests stay correct across CHUNKER_VERSION bumps. + let currentChunkerVersion: string; + + beforeEach(async () => { + const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } = + await import('../src/core/git-head.ts'); + const { CHUNKER_VERSION } = await import('../src/core/chunkers/code.ts'); + currentChunkerVersion = String(CHUNKER_VERSION); + _setGitHeadProbeForTests(null); + _setGitCleanProbeForTests(null); + }); + + afterAll(async () => { + const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } = + await import('../src/core/git-head.ts'); + _setGitHeadProbeForTests(null); + _setGitCleanProbeForTests(null); + }); + + test('case 1: stale + HEAD match + clean tree + chunker match + localOnly=true → ok', async () => { + const { checkSyncFreshness } = await import('../src/commands/doctor.ts'); + const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } = + await import('../src/core/git-head.ts'); + _setGitHeadProbeForTests(() => 'abc123'); + _setGitCleanProbeForTests(() => true); + + const result = await checkSyncFreshness(makeStubEngine([ + { + id: 'media-corpus', name: '', local_path: '/tmp/media', + last_sync_at: agoMs(40 * 60 * 60 * 1000), + last_commit: 'abc123', + chunker_version: currentChunkerVersion, + }, + ]), { localOnly: true }); + + expect(result.status).toBe('ok'); + // Single-source all-unchanged hits the cold-path message; the + // "X synced recently, Y unchanged since last sync" mixed-case shape + // is covered separately in case 6. + expect(result.message).toContain('no new commits since last sync'); + expect(result.details).toEqual({ + unchanged_count: 1, synced_recently_count: 0, stale_count: 0, + }); + }); + + test('case 2: all stale + all unchanged + localOnly=true → ok cold-path message', async () => { + const { checkSyncFreshness } = await import('../src/commands/doctor.ts'); + const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } = + await import('../src/core/git-head.ts'); + _setGitHeadProbeForTests((path) => path === '/tmp/media' ? 'abc' : 'def'); + _setGitCleanProbeForTests(() => true); + + const result = await checkSyncFreshness(makeStubEngine([ + { id: 'media-corpus', name: '', local_path: '/tmp/media', + last_sync_at: agoMs(40 * 60 * 60 * 1000), + last_commit: 'abc', chunker_version: currentChunkerVersion }, + { id: 'archive', name: '', local_path: '/tmp/archive', + last_sync_at: agoMs(50 * 60 * 60 * 1000), + last_commit: 'def', chunker_version: currentChunkerVersion }, + ]), { localOnly: true }); + + expect(result.status).toBe('ok'); + expect(result.message).toBe( + 'All 2 federated source(s) up to date (no new commits since last sync)', + ); + expect(result.details).toEqual({ + unchanged_count: 2, synced_recently_count: 0, stale_count: 0, + }); + }); + + test('case 3: stale + HEAD mismatch + localOnly=true → warn (no short-circuit)', async () => { + const { checkSyncFreshness } = await import('../src/commands/doctor.ts'); + const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } = + await import('../src/core/git-head.ts'); + _setGitHeadProbeForTests(() => 'NEW-HEAD-SHA'); + _setGitCleanProbeForTests(() => true); + + const result = await checkSyncFreshness(makeStubEngine([ + { id: 'wiki', name: '', local_path: '/tmp/wiki', + last_sync_at: agoMs(30 * 60 * 60 * 1000), + last_commit: 'OLD-SHA', chunker_version: currentChunkerVersion }, + ]), { localOnly: true }); + + expect(result.status).toBe('warn'); + expect(result.message).toMatch(/30h ago/); + expect(result.details?.unchanged_count).toBe(0); + expect(result.details?.stale_count).toBe(1); + }); + + test('case 4: stale + matching HEAD + NULL last_commit + localOnly=true → warn (legacy data)', async () => { + const { checkSyncFreshness } = await import('../src/commands/doctor.ts'); + const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } = + await import('../src/core/git-head.ts'); + let headCalls = 0; + _setGitHeadProbeForTests(() => { headCalls++; return 'abc'; }); + _setGitCleanProbeForTests(() => true); + + const result = await checkSyncFreshness(makeStubEngine([ + { id: 'legacy', name: '', local_path: '/tmp/legacy', + last_sync_at: agoMs(30 * 60 * 60 * 1000), + last_commit: null, chunker_version: currentChunkerVersion }, + ]), { localOnly: true }); + + expect(result.status).toBe('warn'); + // Helper short-circuits on NULL guard — head probe should NEVER be called. + expect(headCalls).toBe(0); + expect(result.details?.stale_count).toBe(1); + }); + + test('case 5: stale + non-git path (head probe returns null) + localOnly=true → warn (fail-open)', async () => { + const { checkSyncFreshness } = await import('../src/commands/doctor.ts'); + const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } = + await import('../src/core/git-head.ts'); + _setGitHeadProbeForTests(() => null); // non-git dir + _setGitCleanProbeForTests(() => true); + + const result = await checkSyncFreshness(makeStubEngine([ + { id: 'flat-files', name: '', local_path: '/tmp/flat-files', + last_sync_at: agoMs(30 * 60 * 60 * 1000), + last_commit: 'abc', chunker_version: currentChunkerVersion }, + ]), { localOnly: true }); + + expect(result.status).toBe('warn'); + expect(result.details?.stale_count).toBe(1); + }); + + test('case 6: mixed 3 sources (1 unchanged + 1 synced 5min + 1 truly stale 5d) → fail, three-bucket invariant', async () => { + const { checkSyncFreshness } = await import('../src/commands/doctor.ts'); + const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } = + await import('../src/core/git-head.ts'); + _setGitHeadProbeForTests((path) => { + if (path === '/tmp/unchanged') return 'frozen-sha'; + if (path === '/tmp/recent') return 'new-sha'; // HEAD differs from last_commit → no short-circuit + return 'whatever'; // /tmp/stale: doesn't matter, time path fails + }); + _setGitCleanProbeForTests(() => true); + + const result = await checkSyncFreshness(makeStubEngine([ + { id: 'unchanged', name: '', local_path: '/tmp/unchanged', + last_sync_at: agoMs(40 * 60 * 60 * 1000), + last_commit: 'frozen-sha', chunker_version: currentChunkerVersion }, + { id: 'recent', name: '', local_path: '/tmp/recent', + last_sync_at: agoMs(5 * 60 * 1000), + last_commit: 'OLD', chunker_version: currentChunkerVersion }, + { id: 'stale', name: '', local_path: '/tmp/stale', + last_sync_at: agoMs(5 * 24 * 60 * 60 * 1000), + last_commit: 'OLD2', chunker_version: currentChunkerVersion }, + ]), { localOnly: true }); + + expect(result.status).toBe('fail'); + // Stale source named in the issues list; unchanged + recent are NOT named. + expect(result.message).toContain(`'stale'`); + expect(result.message).not.toContain(`'unchanged'`); + expect(result.message).not.toContain(`'recent'`); + // Three-bucket invariant: sum === sources.length (the load-bearing assertion). + expect(result.details).toEqual({ + unchanged_count: 1, synced_recently_count: 1, stale_count: 1, + }); + const { unchanged_count, synced_recently_count, stale_count } = result.details as any; + expect(unchanged_count + synced_recently_count + stale_count).toBe(3); + }); + + test('case 7: stale + matching HEAD + clean tree + chunker MISMATCH + localOnly=true → warn (chunker gate fires)', async () => { + const { checkSyncFreshness } = await import('../src/commands/doctor.ts'); + const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } = + await import('../src/core/git-head.ts'); + _setGitHeadProbeForTests(() => 'abc'); + _setGitCleanProbeForTests(() => true); + + const result = await checkSyncFreshness(makeStubEngine([ + { id: 'preupgrade', name: '', local_path: '/tmp/pre', + last_sync_at: agoMs(30 * 60 * 60 * 1000), + last_commit: 'abc', + chunker_version: '0', // STALE — bumped via gbrain upgrade since last sync + }, + ]), { localOnly: true }); + + expect(result.status).toBe('warn'); + expect(result.details?.unchanged_count).toBe(0); + expect(result.details?.stale_count).toBe(1); + }); + + test('case 8: stale + matching HEAD + DIRTY tree + chunker match + localOnly=true → warn (dirty gate fires)', async () => { + const { checkSyncFreshness } = await import('../src/commands/doctor.ts'); + const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } = + await import('../src/core/git-head.ts'); + _setGitHeadProbeForTests(() => 'abc'); + _setGitCleanProbeForTests(() => false); // dirty + + const result = await checkSyncFreshness(makeStubEngine([ + { id: 'wip', name: '', local_path: '/tmp/wip', + last_sync_at: agoMs(30 * 60 * 60 * 1000), + last_commit: 'abc', chunker_version: currentChunkerVersion }, + ]), { localOnly: true }); + + expect(result.status).toBe('warn'); + expect(result.details?.unchanged_count).toBe(0); + expect(result.details?.stale_count).toBe(1); + }); + + test('case 9 — D4 regression: localOnly=false (default) — git probes NEVER called', async () => { + const { checkSyncFreshness } = await import('../src/commands/doctor.ts'); + const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } = + await import('../src/core/git-head.ts'); + // Probes set up to return "everything matches" — IF they were called, + // the source would be marked unchanged. Since localOnly defaults false, + // they MUST NOT fire and the source MUST be flagged stale by time check. + let headCalls = 0; + let cleanCalls = 0; + _setGitHeadProbeForTests(() => { headCalls++; return 'matching-sha'; }); + _setGitCleanProbeForTests(() => { cleanCalls++; return true; }); + + // Two callers shapes: + // (a) explicit localOnly:false matches doctorReportRemote semantics + // (b) omitted opts matches the default-fallthrough path + for (const opts of [{ localOnly: false }, undefined]) { + headCalls = 0; + cleanCalls = 0; + const result = await checkSyncFreshness(makeStubEngine([ + { id: 'remote-checked', name: '', local_path: '/tmp/x', + last_sync_at: agoMs(40 * 60 * 60 * 1000), + last_commit: 'matching-sha', chunker_version: currentChunkerVersion }, + ]), opts); + + // Trust boundary: probes MUST NOT have been called. + expect(headCalls).toBe(0); + expect(cleanCalls).toBe(0); + // And without the short-circuit, time check fires the warn: + expect(result.status).toBe('warn'); + expect(result.details?.unchanged_count).toBe(0); + expect(result.details?.stale_count).toBe(1); + } + }); +}); + // Supervisor crash classifier wiring. Pre-fix, doctor.ts:1013 counted every // `worker_exited` event as a crash regardless of `likely_cause`, inflating // `crashes_24h` to 120+/day from RSS-watchdog drains and SIGTERM stops. diff --git a/test/query-cache-knobs-hash.serial.test.ts b/test/query-cache-knobs-hash.serial.test.ts index 7e51bda2e..fdfc02745 100644 --- a/test/query-cache-knobs-hash.serial.test.ts +++ b/test/query-cache-knobs-hash.serial.test.ts @@ -19,6 +19,7 @@ import { PGLiteEngine } from '../src/core/pglite-engine.ts'; import { SemanticQueryCache, cacheRowId } from '../src/core/search/query-cache.ts'; import type { SearchResult } from '../src/core/types.ts'; import { knobsHash, resolveSearchMode } from '../src/core/search/mode.ts'; +import { configureGateway, resetGateway } from '../src/core/ai/gateway.ts'; let engine: PGLiteEngine; @@ -27,6 +28,19 @@ const balancedHash = knobsHash(resolveSearchMode({ mode: 'balanced' })); const tokenmaxHash = knobsHash(resolveSearchMode({ mode: 'tokenmax' })); beforeAll(async () => { + // v0.36.2.0: DEFAULT_EMBEDDING_DIMENSIONS flipped to 1280 (ZE Matryoshka). + // The makeEmbedding fixture below emits 1536-dim unit vectors. If we let + // initSchema() inherit the default, query_cache.embedding gets sized at + // halfvec(1280) and the inserts throw "expected 1280 dimensions, not 1536". + // Pin the gateway to 1536d so this file is hermetic regardless of + // gateway state from other tests in the shard. Pattern matches + // test/consolidate-valid-until.test.ts. + resetGateway(); + configureGateway({ + embedding_model: 'openai:text-embedding-3-large', + embedding_dimensions: 1536, + env: { OPENAI_API_KEY: 'sk-fake' }, + }); engine = new PGLiteEngine(); await engine.connect({}); await engine.initSchema(); @@ -34,6 +48,7 @@ beforeAll(async () => { afterAll(async () => { await engine.disconnect(); + resetGateway(); }); beforeEach(async () => {