v0.41.27.0 fix(doctor): git-aware sync_freshness (supersedes #1564) (#1573)

* feat(doctor): add isSourceUnchangedSinceSync git-head primitive

src/core/git-head.ts is a single-responsibility module: probe git HEAD +
working-tree clean state for a local_path, compare against the
last_commit SHA the DB stored at last sync completion. Designed for
reuse (autopilot's per-source dispatch will want the same gate, filed
as v0.41.27.1+ TODO).

Shell-injection safe: uses execFileSync with array args. The
superseded community PR #1564 used execSync through /bin/sh -c with
JSON.stringify for shell-escape, which is unsafe — JSON.stringify
escapes for JSON, not shell.

Fail-open contract: every error path returns false, preserving the
caller's prior time-based behavior. Two test seams
(_setGitHeadProbeForTests, _setGitCleanProbeForTests) match the
last-retrieved.ts precedent so unit tests stay parallel-eligible (no
mock.module per CLAUDE.md R2).

Companion test suite: 14 cases including a load-bearing
shell-injection regression guard that runs real execFileSync against
'/nonexistent/$(touch <sentinel>)/repo' and asserts the sentinel
file is never created.

* feat(doctor): git-aware sync_freshness check + 9 new test cases

checkSyncFreshness gains opts.localOnly: boolean. When local CLI
caller passes true, doctor short-circuits the staleness warning iff
HEAD == sources.last_commit AND working tree is clean AND
sources.chunker_version matches CURRENT (mirrors sync.ts:1057+1075's
own "do work?" predicate, so doctor and sync agree).

Inline SELECT widens to carry last_commit + chunker_version (columns
already exist; no schema migration). Three-bucket count math
(unchanged_count + synced_recently_count + stale_count ===
sources.length) populates Check.details for dashboards / JSON
consumers. OK-message reshape:
 - all-unchanged → "All N up to date (no new commits since last sync)"
 - mixed        → "N source(s): X synced recently, Y unchanged since last sync"
 - all-recent   → "All N synced recently" (back-compat).

Trust boundary preserved (Codex P0-1): runDoctor (local CLI, trusted)
passes localOnly: true; doctorReportRemote (HTTP MCP, untrusted)
keeps the default false. Default-false is fail-closed — a future
caller that forgets the opt gets the safe no-probe behavior.

checkCycleFreshness is INTENTIONALLY NOT touched (Codex P0-2):
last_commit == HEAD answers "new commits to sync?" but cannot answer
"did the full cycle complete?" — silencing cycle warns on git-clean
sources would mask the case where sync ran but extract/embed/
consolidate/synthesize failed.

Test coverage: 9 new cases in test/doctor.test.ts including
 - HEAD-match short-circuit + cold-path message
 - HEAD-mismatch warn
 - NULL last_commit (legacy data) → warn
 - non-git local_path (probe returns null) → warn (fail-open)
 - 3-source mixed bucket invariant (sum === length)
 - chunker-version mismatch warn (dirty-tree-clean still gates)
 - dirty-tree warn (HEAD-match still gates)
 - D4 regression guard: localOnly=false (default) NEVER calls probes

All 87 tests across git-head + doctor + cycle-freshness suites pass.
bun run verify clean (28/28 checks).

Co-Authored-By: garrytan-agents <me@garrytan.com>

* chore: bump version and changelog (v0.41.27.0)

Supersedes community PR #1564. Co-Authored-By preserved.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(test): pin gateway to 1536d for query-cache-knobs-hash suite

CI failed on the v0.41.27.0 ship after merging origin/master because
test/query-cache-knobs-hash.test.ts predates the v0.36.2.0 flip of
DEFAULT_EMBEDDING_DIMENSIONS from 1536 → 1280 (ZE Matryoshka).

Without an explicit gateway pin, initSchema() sizes query_cache.embedding
at halfvec(1280), but the test fixture (makeEmbedding at line 44) emits
1536-dim unit vectors. Result: 5 cases in the
"SemanticQueryCache cross-mode isolation (CDX-4 hotfix)" describe
crashed with "expected 1280 dimensions, not 1536".

Fix mirrors test/consolidate-valid-until.test.ts (the canonical
gateway-pin pattern that landed when the default flipped). resetGateway()
+ configureGateway({embedding_dimensions: 1536}) in beforeAll forces the
schema to size at halfvec(1536) regardless of cross-file gateway state.
resetGateway() in afterAll restores defaults so the next file in the
shard isn't poisoned.

Verified: 9/9 cases pass; bun run verify 29/29 clean.

---------

Co-authored-by: garrytan-agents <me@garrytan.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-05-28 17:58:21 -07:00
committed by GitHub
co-authored by garrytan-agents Claude Opus 4.7
parent 6ae94301a6
commit cb1b5f91f7
10 changed files with 739 additions and 7 deletions
+79
View File
@@ -2,6 +2,85 @@
All notable changes to GBrain will be documented in this file.
## [0.41.27.0] - 2026-05-27
**`gbrain doctor` stops crying wolf about sources that have no new commits.**
If you have a brain that points at a git repo you don't change every day
(a reference corpus, a frozen archive, an inbox you append to monthly),
doctor's "Source X last synced 40h ago" warning has been firing every
single day even though nothing was actually stale. There was no new
content to pull. The warning was telling you to run `gbrain sync` for
work that didn't exist.
This release teaches doctor to do a quick git check first: if your
repo's current HEAD matches what we stored at the last sync, **and**
the working tree is clean, **and** the chunker version still matches,
the source is reported as "up to date" regardless of how long ago you
synced. The three checks together mirror exactly what `gbrain sync`
itself decides — so doctor and sync now agree on "is there work to do?"
You stop seeing warnings for problems that don't exist.
**How to turn it on:** Nothing to do. `gbrain upgrade` is all you need.
The git probe runs locally only (your terminal's `gbrain doctor`); the
remote HTTP MCP path stays out of it by design (deliberately doesn't
walk DB-supplied paths via subprocess — same trust posture as before).
**What you'd see in a concrete example:**
| Doctor scenario | Pre-fix message | Post-fix message |
|---|---|---|
| 40h since sync, zero new commits, clean tree | warn: "Source 'media-corpus' last synced 40h ago" | ok: "All 1 federated source(s) up to date (no new commits since last sync)" |
| 40h since sync, 3 new commits to pull | warn: "...40h ago" | warn: "...40h ago" (unchanged — warning still correct) |
| 40h since sync, HEAD matches but `gbrain upgrade` bumped CHUNKER_VERSION | warn: "...40h ago" (true bug — pending re-embed) | warn: "...40h ago" (chunker gate fires; you still see the warn so you don't miss the post-upgrade re-embed) |
| 40h since sync, HEAD matches but you have uncommitted edits | warn: "...40h ago" | warn: "...40h ago" (dirty-tree gate fires; honest about pending work) |
| Mixed brain: 1 frozen source + 1 recently synced + 1 truly stale | fail: lists all 3 sources | fail: lists only the truly stale source; the other two are silenced honestly |
**The cycle freshness check is deliberately NOT touched.** A separate
codex-review pass caught a load-bearing semantic distinction: HEAD ==
last_commit only answers "are there new commits to sync?". It cannot
answer "did the full cycle (sync + extract + embed + consolidate +
synthesize) complete recently?" — that's a later, different invariant.
Hiding cycle-staleness warnings on git-clean sources would silently
mask the case where sync ran but the cycle phases after it failed.
Doctor's `cycle_freshness` keeps its time-only semantics; only
`sync_freshness` gets the git-aware short-circuit.
**Things worth knowing about:**
- `Check.details` now carries `{unchanged_count, synced_recently_count,
stale_count}` for the `sync_freshness` check. Dashboards consuming
the JSON envelope can read these directly. The three counts sum to
the source count — invariant pinned in unit tests.
- If you intentionally keep WIP edits in your brain repo, doctor will
still warn after 24h because the dirty-tree gate fires. That's
honest — sync would actually do work in that case. An opt-out env
var (`GBRAIN_DOCTOR_IGNORE_DIRTY_TREE=1`) is filed for v0.41.27.1+
if anyone asks.
**The cathedral side.** This release rebuilds community PR #1564 with
four production-quality concerns the original missed: shell-injection
safety (uses `execFileSync` with array args, not `execSync` through
`/bin/sh -c`), trust-boundary preservation (remote-callable doctor
path doesn't get the git probe), narrowed predicate honesty (chunker
version + working-tree-clean, matching sync's own gate), and 21 new
test cases covering every branch including a shell-injection
regression guard that runs real `execFileSync` against a `$(...)`
adversarial path to prove the array-arg shape cannot escape to a
shell. Co-Authored-By preserved for the original contributor.
### Itemized changes
- **`src/core/git-head.ts`** (NEW) — single `isSourceUnchangedSinceSync(localPath, lastCommit, opts?)` primitive with two probes (head + clean). `GitFreshnessOpts.requireCleanWorkingTree` is the second-probe gate. Two test seams (`_setGitHeadProbeForTests`, `_setGitCleanProbeForTests`) match the `last-retrieved.ts` precedent so tests stay parallel-eligible (R2-compliant — no `mock.module`). Fail-open contract: every error path returns false, preserving the caller's prior behavior. Uses `execFileSync` with array args — shell metachars in `local_path` cannot escape.
- **`src/commands/doctor.ts:checkSyncFreshness`** — signature gains `opts?.localOnly?: boolean`. Inline SELECT widens to carry `last_commit + chunker_version` (columns already exist; no schema migration). Helper wired AFTER the existing NULL / negative-age guards, gated by `localOnly === true` AND'd with `source.chunker_version === String(CHUNKER_VERSION)`. Three-bucket count math (`unchanged_count + synced_recently_count + stale_count === sources.length`) populates `Check.details`. OK message reshape: all-unchanged hits "up to date (no new commits since last sync)"; mixed hits "X synced recently, Y unchanged since last sync"; all-synced keeps the prior message.
- **Caller plumbing**: `runDoctor` (local CLI path) passes `localOnly: true`; `doctorReportRemote` (HTTP MCP path) keeps the default `false`. Default-false is fail-closed: a future caller that forgets the opt gets the safe (no git probe) behavior. Codex P0-1 closure.
- **`test/core/git-head.test.ts`** (NEW) — 12-case suite: happy path, mismatch, null/empty guards, probe-null, probe-throws, **shell-injection regression** (real `execFileSync` against `/nonexistent/$(touch <sentinel>)/repo` — sentinel file MUST NOT exist after the call), test-seam round-trip, `requireCleanWorkingTree` clean/dirty/error/not-set cases.
- **`test/doctor.test.ts`** — new v0.41.27.0 describe with 9 cases: HEAD-match short-circuit, all-unchanged cold path, HEAD-mismatch warn, NULL `last_commit`, non-git path, **3-source mixed bucket invariant** (`sum === sources.length` asserted explicitly), chunker-version mismatch warn, dirty-tree warn, **`localOnly=false` regression guard** that verifies probes are NEVER called when the opt is unset or false.
- **CHANGELOG / VERSION / package.json / bun.lock / llms.txt / llms-full.txt / CLAUDE.md** — version bump + lockfile refresh + docs regen + key-files annotation.
### Supersedes
PR #1564 (`@garrytan-agents`). Co-Authored-By preserved. Closed with a supersession comment explaining the production-quality additions. The surrogate-pair fix from commit `78b93f3f` in that PR is NOT brought over — already on master via `safeSplitIndex` at `synthesize.ts:192` (v0.42.0.0 wave).
## [0.41.26.1] - 2026-05-27
**Your worker daemon stops crashing 39 times a day.**
+2 -1
View File
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -1 +1 @@
0.41.26.1
0.41.27.0
+2 -1
View File
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -141,5 +141,5 @@
"bun": ">=1.3.10"
},
"license": "MIT",
"version": "0.41.26.1"
"version": "0.41.27.0"
}
+94 -3
View File
@@ -27,6 +27,8 @@ import { gbrainPath } from '../core/config.ts';
import { dirname, isAbsolute, join, resolve as resolvePath } from 'path';
import { fileURLToPath } from 'url';
import { existsSync, readFileSync, readdirSync, statSync } from 'fs';
import { isSourceUnchangedSinceSync } from '../core/git-head.ts';
import { CHUNKER_VERSION } from '../core/chunkers/code.ts';
export interface Check {
name: string;
@@ -2671,16 +2673,23 @@ export async function computeExtractHealthCheck(
export async function checkSyncFreshness(
engine: BrainEngine,
opts?: { nowMs?: number },
opts?: { nowMs?: number; localOnly?: boolean },
): Promise<Check> {
try {
// v0.41.27.0: SELECT widens to carry last_commit + chunker_version so
// the git short-circuit gate (below) can compare against what
// `gbrain sync`'s up-to-date predicate at sync.ts:1057+1075 checks.
// Columns existed pre-v0.41 (writeSyncAnchor / writeChunkerVersion);
// no schema migration needed.
const sources = await engine.executeRaw<{
id: string;
name: string;
local_path: string | null;
last_sync_at: Date | null;
last_commit: string | null;
chunker_version: string | null;
}>(
`SELECT id, name, local_path, last_sync_at FROM sources WHERE local_path IS NOT NULL`,
`SELECT id, name, local_path, last_sync_at, last_commit, chunker_version FROM sources WHERE local_path IS NOT NULL`,
);
if (sources.length === 0) {
@@ -2688,6 +2697,7 @@ export async function checkSyncFreshness(
name: 'sync_freshness',
status: 'ok',
message: 'No federated sources to sync',
details: { unchanged_count: 0, synced_recently_count: 0, stale_count: 0 },
};
}
@@ -2703,7 +2713,30 @@ export async function checkSyncFreshness(
// status from warn to fail (CI-flaky, see PR #1138 ship). Production
// callers omit `nowMs` and get live wall-clock semantics.
const now = opts?.nowMs ?? Date.now();
// v0.41.27.0: D4 trust boundary. The git short-circuit runs ONLY when
// the caller explicitly opts in via `localOnly: true`. Default (false)
// preserves the v0.32.4 trust boundary for `doctorReportRemote` (the
// HTTP MCP path) — a remote-callable code path must NOT walk
// DB-supplied `local_path` values with subprocess calls. runDoctor
// (local CLI) passes true; doctorReportRemote keeps the default.
const localOnly = opts?.localOnly === true;
// v0.41.27.0: D7 narrowed predicate. The CHUNKER_VERSION caller-side
// check mirrors sync.ts:1057's chunker-version gate so doctor agrees
// with sync on "is there work to do?". `sources.chunker_version` is
// a TEXT column storing String(CHUNKER_VERSION).
const currentChunkerVersion = String(CHUNKER_VERSION);
const issues: string[] = [];
// v0.41.27.0: D6 three-bucket count math. Every source falls into
// EXACTLY ONE bucket per iteration. Invariant pinned by unit test:
// unchanged_count + synced_recently_count + stale_count === sources.length
// Stale subsumes warn + fail + never-synced + future-timestamp; we keep
// hasWarnings/hasFailures for the existing return-status logic.
let unchanged_count = 0;
let synced_recently_count = 0;
let stale_count = 0;
let hasWarnings = false;
let hasFailures = false;
@@ -2717,6 +2750,7 @@ export async function checkSyncFreshness(
if (!source.last_sync_at) {
issues.push(`Source ${display} has never been synced`);
hasFailures = true;
stale_count++;
continue;
}
@@ -2728,26 +2762,56 @@ export async function checkSyncFreshness(
`Source ${display} has future last_sync_at — clock skew or corrupted timestamp`,
);
hasWarnings = true;
stale_count++;
continue;
}
// v0.41.27.0: git short-circuit (D4 + D7 combined). Only fires when:
// 1. caller opted in via localOnly=true (trust boundary)
// 2. HEAD === last_commit (no new commits to sync)
// 3. working tree is clean (no uncommitted edits sync would re-walk)
// 4. chunker_version matches CURRENT (no post-upgrade re-chunk pending)
// All four must hold; otherwise fall through to the time-based check.
// The chunker version match is computed here (not in the helper)
// because it depends on engine state, not git state.
if (localOnly) {
const gitUnchanged = isSourceUnchangedSinceSync(
source.local_path,
source.last_commit,
{ requireCleanWorkingTree: true },
);
const chunkerMatch = source.chunker_version === currentChunkerVersion;
if (gitUnchanged && chunkerMatch) {
unchanged_count++;
continue;
}
}
const ageHours = Math.floor(ageMs / (1000 * 60 * 60));
const ageDays = Math.floor(ageHours / 24);
if (ageMs > failMs) {
issues.push(`Source ${display} last synced ${ageDays}d ago — brain search is stale!`);
hasFailures = true;
stale_count++;
} else if (ageMs > warnMs) {
issues.push(`Source ${display} last synced ${ageHours}h ago`);
hasWarnings = true;
stale_count++;
} else {
synced_recently_count++;
}
}
// D6 invariant: every source incremented exactly one bucket.
const details = { unchanged_count, synced_recently_count, stale_count };
if (hasFailures) {
return {
name: 'sync_freshness',
status: 'fail',
message: `${issues.join('; ')}. Run \`gbrain sync --source <id>\` for each stale source`,
details,
};
}
if (hasWarnings) {
@@ -2755,12 +2819,33 @@ export async function checkSyncFreshness(
name: 'sync_freshness',
status: 'warn',
message: `${issues.join('; ')}. Run \`gbrain sync --source <id>\` to refresh`,
details,
};
}
// v0.41.27.0: D2 ok-message reshape. Three branches surface what the
// git short-circuit actually did so operators understand "unchanged
// since last sync" vs "synced recently".
if (unchanged_count === sources.length) {
return {
name: 'sync_freshness',
status: 'ok',
message: `All ${sources.length} federated source(s) up to date (no new commits since last sync)`,
details,
};
}
if (unchanged_count > 0) {
return {
name: 'sync_freshness',
status: 'ok',
message: `${sources.length} federated source(s): ${synced_recently_count} synced recently, ${unchanged_count} unchanged since last sync`,
details,
};
}
return {
name: 'sync_freshness',
status: 'ok',
message: `All ${sources.length} federated source(s) synced recently`,
details,
};
} catch (e) {
return {
@@ -5713,7 +5798,13 @@ export async function buildChecks(
// Sync freshness check (v0.32 — Check that sources are synced recently)
if (engine !== null) {
progress.heartbeat('sync_freshness');
checks.push(await checkSyncFreshness(engine));
// v0.41.27.0 D4: local CLI path is trusted to walk DB-supplied
// local_path values via subprocess (we own the brain repo). Pass
// localOnly:true so the git short-circuit fires. The HTTP MCP path
// at doctorReportRemote (around line 662) deliberately keeps the
// default (false) — that's the trust-boundary preservation Codex
// P0-1 flagged.
checks.push(await checkSyncFreshness(engine, { localOnly: true }));
// v0.41.19.0 (Issue 5): sync --all consolidation nudge.
progress.heartbeat('sync_consolidation');
checks.push(await checkSyncConsolidation(engine));
+110
View File
@@ -0,0 +1,110 @@
/**
* v0.41.27.0 — git HEAD freshness probe for `gbrain doctor`.
*
* Single primitive. Returns true iff a `local_path` directory is a git repo
* whose current HEAD matches the `last_commit` SHA the DB recorded at last
* sync completion. When `requireCleanWorkingTree` is set, also requires the
* working tree to be clean — mirroring `gbrain sync`'s force-walk gate at
* sync.ts:1075 so doctor and sync agree on "is there work to do?".
*
* Fail-open contract: any error (missing path, not a git repo, git not
* installed, timeout, NULL inputs, dirty probe errored) returns `false`,
* which preserves the caller's prior time-based behavior. We never raise.
*
* Shell-injection safe: uses execFileSync with array args so a `local_path`
* containing `$(...)`, backticks, or other shell metacharacters can never
* escape to a shell. The PR #1564 community version used
* `execSync(`git -C ${JSON.stringify(path)} ...`)`, which runs through
* `/bin/sh -c` — `JSON.stringify` escapes for JSON, not shell, so a
* mutable `sources.local_path` was an RCE-style surface.
*
* Designed for reuse: autopilot's per-source dispatch will want the same
* gate. See plan note "v0.41.27.1+ TODOs" in
* ~/.claude/plans/system-instruction-you-are-working-eager-bird.md.
*/
import { execFileSync } from 'node:child_process';
export type GitHeadProbe = (localPath: string) => string | null;
// `null` distinguishes probe error from known-dirty (false). Doctor treats
// both as "do not short-circuit", but tests need to assert which path fired.
export type GitCleanProbe = (localPath: string) => boolean | null;
const DEFAULT_HEAD_PROBE: GitHeadProbe = (localPath) => {
try {
const out = execFileSync('git', ['-C', localPath, 'rev-parse', 'HEAD'], {
encoding: 'utf8',
timeout: 5000,
stdio: ['ignore', 'pipe', 'ignore'],
});
return out.trim() || null;
} catch {
return null;
}
};
const DEFAULT_CLEAN_PROBE: GitCleanProbe = (localPath) => {
try {
const out = execFileSync('git', ['-C', localPath, 'status', '--porcelain'], {
encoding: 'utf8',
timeout: 5000,
stdio: ['ignore', 'pipe', 'ignore'],
});
return out.trim().length === 0;
} catch {
return null;
}
};
let _headProbe: GitHeadProbe = DEFAULT_HEAD_PROBE;
let _cleanProbe: GitCleanProbe = DEFAULT_CLEAN_PROBE;
// Test seam. Matches the `_setChatTransportForTests` precedent at
// src/core/last-retrieved.ts so tests can drive the public function
// without mocking child_process or routing through mock.module (R2-compliant).
export function _setGitHeadProbeForTests(fn: GitHeadProbe | null): void {
_headProbe = fn ?? DEFAULT_HEAD_PROBE;
}
export function _setGitCleanProbeForTests(fn: GitCleanProbe | null): void {
_cleanProbe = fn ?? DEFAULT_CLEAN_PROBE;
}
export interface GitFreshnessOpts {
/**
* When true, additionally require working tree to be clean (no
* uncommitted changes). Doctor uses this to mirror `gbrain sync`'s
* working-tree-dirty gate at sync.ts:1075 — otherwise doctor would
* say "unchanged" for a repo with pending local edits that sync would
* actually re-walk on the next run.
*
* Default false (HEAD comparison only). Doctor-callers set true.
*/
requireCleanWorkingTree?: boolean;
}
/**
* Returns true iff `localPath` is a git repo whose current HEAD matches
* `lastCommit`, AND (when `requireCleanWorkingTree`) the working tree
* is clean.
*
* This is NOT a full mirror of `gbrain sync`'s "do work?" predicate.
* Chunker-version match is computed by the caller because it depends on
* engine state (`sources.chunker_version` vs `CURRENT_CHUNKER_VERSION`).
* See `src/commands/doctor.ts:checkSyncFreshness` for the AND
* combination at the call site.
*/
export function isSourceUnchangedSinceSync(
localPath: string | null | undefined,
lastCommit: string | null | undefined,
opts?: GitFreshnessOpts,
): boolean {
if (!localPath || !lastCommit) return false;
const head = _headProbe(localPath);
if (head === null || head !== lastCommit) return false;
if (opts?.requireCleanWorkingTree) {
const isClean = _cleanProbe(localPath);
// null (probe error) AND false (known dirty) both fail the gate.
if (isClean !== true) return false;
}
return true;
}
+178
View File
@@ -0,0 +1,178 @@
// v0.41.27.0 — src/core/git-head.ts unit tests.
//
// Hermetic: no engine, no PGLite, no DATABASE_URL required. Probe seam lets
// the suite drive isSourceUnchangedSinceSync without spawning real git (cases
// 1-8, 10-12); case 9 is the shell-injection regression guard that DOES use
// the real execFileSync against a deliberately adversarial localPath to prove
// the array-arg call shape cannot escape to a shell.
import { describe, expect, test, beforeEach, afterAll } from 'bun:test';
import { existsSync, mkdtempSync, rmSync, unlinkSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import {
isSourceUnchangedSinceSync,
_setGitHeadProbeForTests,
_setGitCleanProbeForTests,
type GitHeadProbe,
type GitCleanProbe,
} from '../../src/core/git-head.ts';
// Reset both probe seams between every test so case order can't leak state.
beforeEach(() => {
_setGitHeadProbeForTests(null);
_setGitCleanProbeForTests(null);
});
afterAll(() => {
// Restore defaults so other test files inheriting the module state are clean.
_setGitHeadProbeForTests(null);
_setGitCleanProbeForTests(null);
});
describe('isSourceUnchangedSinceSync — basic predicate', () => {
test('case 1: happy path — HEAD matches, no requireCleanWorkingTree → true', () => {
_setGitHeadProbeForTests(() => 'abc123');
expect(isSourceUnchangedSinceSync('/tmp/repo', 'abc123')).toBe(true);
});
test('case 2: HEAD differs from lastCommit → false', () => {
_setGitHeadProbeForTests(() => 'def456');
expect(isSourceUnchangedSinceSync('/tmp/repo', 'abc123')).toBe(false);
});
test('case 3: localPath null — short-circuits without calling head probe', () => {
let probeCalls = 0;
_setGitHeadProbeForTests(() => { probeCalls++; return 'abc'; });
expect(isSourceUnchangedSinceSync(null, 'abc123')).toBe(false);
expect(probeCalls).toBe(0);
});
test('case 4: localPath empty string — short-circuits, probe not called', () => {
let probeCalls = 0;
_setGitHeadProbeForTests(() => { probeCalls++; return 'abc'; });
expect(isSourceUnchangedSinceSync('', 'abc123')).toBe(false);
expect(probeCalls).toBe(0);
});
test('case 5: lastCommit null — short-circuits, probe not called', () => {
let probeCalls = 0;
_setGitHeadProbeForTests(() => { probeCalls++; return 'abc'; });
expect(isSourceUnchangedSinceSync('/tmp/repo', null)).toBe(false);
expect(probeCalls).toBe(0);
});
test('case 6: lastCommit empty string — short-circuits, probe not called', () => {
let probeCalls = 0;
_setGitHeadProbeForTests(() => { probeCalls++; return 'abc'; });
expect(isSourceUnchangedSinceSync('/tmp/repo', '')).toBe(false);
expect(probeCalls).toBe(0);
});
test('case 7: head probe returns null (non-git dir / git not installed) → false', () => {
_setGitHeadProbeForTests(() => null);
expect(isSourceUnchangedSinceSync('/tmp/not-a-repo', 'abc123')).toBe(false);
});
test('case 8: head probe throws synchronously → false (fail-open)', () => {
_setGitHeadProbeForTests(() => { throw new Error('git not installed'); });
expect(() => isSourceUnchangedSinceSync('/tmp/repo', 'abc123')).toThrow();
// The default behavior is fail-open via the probe's own try/catch
// (returning null). A test seam that throws bypasses that protection
// intentionally — production callers use the default probe which
// swallows the error. Documented at git-head.ts:35-41.
// Re-verify by stubbing a probe that swallows the underlying error:
_setGitHeadProbeForTests(() => { try { throw new Error('inner'); } catch { return null; } });
expect(isSourceUnchangedSinceSync('/tmp/repo', 'abc123')).toBe(false);
});
});
describe('isSourceUnchangedSinceSync — shell-injection regression guard', () => {
test('case 9: REAL execFileSync against adversarial localPath does NOT execute shell metachars', () => {
// Use the REAL default probe (no test seam) so this case exercises the
// production code path: execFileSync('git', ['-C', path, ...]). If the
// implementation ever regresses to execSync with shell interpolation,
// the `$(...)` substring would execute and create the sentinel file.
_setGitHeadProbeForTests(null);
_setGitCleanProbeForTests(null);
const sentinelDir = mkdtempSync(join(tmpdir(), 'git-head-sentinel-'));
const sentinelPath = join(sentinelDir, 'pwned');
const adversarialPath = `/nonexistent/$(touch ${sentinelPath})/repo`;
try {
// Should fail-open: the path doesn't exist, git rev-parse returns
// non-zero, the probe returns null, the helper returns false.
const result = isSourceUnchangedSinceSync(adversarialPath, 'abc123');
expect(result).toBe(false);
// The load-bearing assertion: if shell interpolation happened,
// `touch ${sentinelPath}` would have created the file. It must NOT
// exist.
expect(existsSync(sentinelPath)).toBe(false);
} finally {
// Cleanup. If the test failed-open AND created the sentinel, remove it.
if (existsSync(sentinelPath)) {
try { unlinkSync(sentinelPath); } catch { /* ignore */ }
}
try { rmSync(sentinelDir, { recursive: true, force: true }); } catch { /* ignore */ }
}
});
});
describe('isSourceUnchangedSinceSync — test-seam round-trip', () => {
test('case 10: _setGitHeadProbeForTests + _setGitCleanProbeForTests round-trip', () => {
// Install custom probes
const customHead: GitHeadProbe = () => 'custom-head';
const customClean: GitCleanProbe = () => true;
_setGitHeadProbeForTests(customHead);
_setGitCleanProbeForTests(customClean);
expect(isSourceUnchangedSinceSync('/x', 'custom-head', { requireCleanWorkingTree: true })).toBe(true);
// Restore defaults via null
_setGitHeadProbeForTests(null);
_setGitCleanProbeForTests(null);
// After restoration, the real probes run. Against a non-git path,
// both probes return null → predicate returns false. This proves the
// restore worked without depending on a git repo in the test env.
expect(isSourceUnchangedSinceSync('/definitely-not-a-git-repo-xyz', 'custom-head')).toBe(false);
});
});
describe('isSourceUnchangedSinceSync — requireCleanWorkingTree (D7)', () => {
test('case 11: HEAD match + clean tree + requireCleanWorkingTree=true → true', () => {
_setGitHeadProbeForTests(() => 'abc123');
_setGitCleanProbeForTests(() => true);
expect(
isSourceUnchangedSinceSync('/tmp/repo', 'abc123', { requireCleanWorkingTree: true }),
).toBe(true);
});
test('case 12a: HEAD match + DIRTY tree + requireCleanWorkingTree=true → false', () => {
_setGitHeadProbeForTests(() => 'abc123');
_setGitCleanProbeForTests(() => false);
expect(
isSourceUnchangedSinceSync('/tmp/repo', 'abc123', { requireCleanWorkingTree: true }),
).toBe(false);
});
test('case 12b: HEAD match + clean probe ERRORED (returns null) + requireCleanWorkingTree=true → false', () => {
// null is distinct from false: probe error vs known-dirty. Both fail
// the gate (fail-closed posture for the clean check, fail-open posture
// for the helper as a whole — caller's time-based check still runs).
_setGitHeadProbeForTests(() => 'abc123');
_setGitCleanProbeForTests(() => null);
expect(
isSourceUnchangedSinceSync('/tmp/repo', 'abc123', { requireCleanWorkingTree: true }),
).toBe(false);
});
test('case 12c: HEAD match + tree dirty BUT requireCleanWorkingTree NOT set → true (clean probe not consulted)', () => {
let cleanCalls = 0;
_setGitHeadProbeForTests(() => 'abc123');
_setGitCleanProbeForTests(() => { cleanCalls++; return false; });
expect(isSourceUnchangedSinceSync('/tmp/repo', 'abc123')).toBe(true);
expect(cleanCalls).toBe(0);
});
});
+257
View File
@@ -585,6 +585,263 @@ describe('v0.32.4 — sync_freshness check', () => {
});
});
// ============================================================================
// v0.41.27.0 — sync_freshness git short-circuit (D4 + D6 + D7)
// ============================================================================
// Doctor learns to skip the staleness warning when a git-backed source has no
// new commits since the last sync AND working tree is clean AND chunker
// version matches. Trust boundary preserved via opts.localOnly (D4); count
// math fixed with three buckets that sum to sources.length (D6); narrowed
// predicate mirrors sync.ts:1057+1075 (D7).
// ============================================================================
describe('v0.41.27.0 — sync_freshness git short-circuit', () => {
// Reuse the stub-engine pattern from v0.32.4 describe above. Row shape now
// includes last_commit + chunker_version (extended SELECT in v0.41.27.0).
function makeStubEngine(rows: any[]): any {
return { executeRaw: async () => rows };
}
function agoMs(ms: number): Date {
return new Date(Date.now() - ms);
}
// Probe seams come from src/core/git-head.ts. Reset between each test so
// case order can't leak state. CURRENT is imported from chunkers/code.ts —
// tests stay correct across CHUNKER_VERSION bumps.
let currentChunkerVersion: string;
beforeEach(async () => {
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
const { CHUNKER_VERSION } = await import('../src/core/chunkers/code.ts');
currentChunkerVersion = String(CHUNKER_VERSION);
_setGitHeadProbeForTests(null);
_setGitCleanProbeForTests(null);
});
afterAll(async () => {
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
_setGitHeadProbeForTests(null);
_setGitCleanProbeForTests(null);
});
test('case 1: stale + HEAD match + clean tree + chunker match + localOnly=true → ok', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
_setGitHeadProbeForTests(() => 'abc123');
_setGitCleanProbeForTests(() => true);
const result = await checkSyncFreshness(makeStubEngine([
{
id: 'media-corpus', name: '', local_path: '/tmp/media',
last_sync_at: agoMs(40 * 60 * 60 * 1000),
last_commit: 'abc123',
chunker_version: currentChunkerVersion,
},
]), { localOnly: true });
expect(result.status).toBe('ok');
// Single-source all-unchanged hits the cold-path message; the
// "X synced recently, Y unchanged since last sync" mixed-case shape
// is covered separately in case 6.
expect(result.message).toContain('no new commits since last sync');
expect(result.details).toEqual({
unchanged_count: 1, synced_recently_count: 0, stale_count: 0,
});
});
test('case 2: all stale + all unchanged + localOnly=true → ok cold-path message', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
_setGitHeadProbeForTests((path) => path === '/tmp/media' ? 'abc' : 'def');
_setGitCleanProbeForTests(() => true);
const result = await checkSyncFreshness(makeStubEngine([
{ id: 'media-corpus', name: '', local_path: '/tmp/media',
last_sync_at: agoMs(40 * 60 * 60 * 1000),
last_commit: 'abc', chunker_version: currentChunkerVersion },
{ id: 'archive', name: '', local_path: '/tmp/archive',
last_sync_at: agoMs(50 * 60 * 60 * 1000),
last_commit: 'def', chunker_version: currentChunkerVersion },
]), { localOnly: true });
expect(result.status).toBe('ok');
expect(result.message).toBe(
'All 2 federated source(s) up to date (no new commits since last sync)',
);
expect(result.details).toEqual({
unchanged_count: 2, synced_recently_count: 0, stale_count: 0,
});
});
test('case 3: stale + HEAD mismatch + localOnly=true → warn (no short-circuit)', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
_setGitHeadProbeForTests(() => 'NEW-HEAD-SHA');
_setGitCleanProbeForTests(() => true);
const result = await checkSyncFreshness(makeStubEngine([
{ id: 'wiki', name: '', local_path: '/tmp/wiki',
last_sync_at: agoMs(30 * 60 * 60 * 1000),
last_commit: 'OLD-SHA', chunker_version: currentChunkerVersion },
]), { localOnly: true });
expect(result.status).toBe('warn');
expect(result.message).toMatch(/30h ago/);
expect(result.details?.unchanged_count).toBe(0);
expect(result.details?.stale_count).toBe(1);
});
test('case 4: stale + matching HEAD + NULL last_commit + localOnly=true → warn (legacy data)', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
let headCalls = 0;
_setGitHeadProbeForTests(() => { headCalls++; return 'abc'; });
_setGitCleanProbeForTests(() => true);
const result = await checkSyncFreshness(makeStubEngine([
{ id: 'legacy', name: '', local_path: '/tmp/legacy',
last_sync_at: agoMs(30 * 60 * 60 * 1000),
last_commit: null, chunker_version: currentChunkerVersion },
]), { localOnly: true });
expect(result.status).toBe('warn');
// Helper short-circuits on NULL guard — head probe should NEVER be called.
expect(headCalls).toBe(0);
expect(result.details?.stale_count).toBe(1);
});
test('case 5: stale + non-git path (head probe returns null) + localOnly=true → warn (fail-open)', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
_setGitHeadProbeForTests(() => null); // non-git dir
_setGitCleanProbeForTests(() => true);
const result = await checkSyncFreshness(makeStubEngine([
{ id: 'flat-files', name: '', local_path: '/tmp/flat-files',
last_sync_at: agoMs(30 * 60 * 60 * 1000),
last_commit: 'abc', chunker_version: currentChunkerVersion },
]), { localOnly: true });
expect(result.status).toBe('warn');
expect(result.details?.stale_count).toBe(1);
});
test('case 6: mixed 3 sources (1 unchanged + 1 synced 5min + 1 truly stale 5d) → fail, three-bucket invariant', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
_setGitHeadProbeForTests((path) => {
if (path === '/tmp/unchanged') return 'frozen-sha';
if (path === '/tmp/recent') return 'new-sha'; // HEAD differs from last_commit → no short-circuit
return 'whatever'; // /tmp/stale: doesn't matter, time path fails
});
_setGitCleanProbeForTests(() => true);
const result = await checkSyncFreshness(makeStubEngine([
{ id: 'unchanged', name: '', local_path: '/tmp/unchanged',
last_sync_at: agoMs(40 * 60 * 60 * 1000),
last_commit: 'frozen-sha', chunker_version: currentChunkerVersion },
{ id: 'recent', name: '', local_path: '/tmp/recent',
last_sync_at: agoMs(5 * 60 * 1000),
last_commit: 'OLD', chunker_version: currentChunkerVersion },
{ id: 'stale', name: '', local_path: '/tmp/stale',
last_sync_at: agoMs(5 * 24 * 60 * 60 * 1000),
last_commit: 'OLD2', chunker_version: currentChunkerVersion },
]), { localOnly: true });
expect(result.status).toBe('fail');
// Stale source named in the issues list; unchanged + recent are NOT named.
expect(result.message).toContain(`'stale'`);
expect(result.message).not.toContain(`'unchanged'`);
expect(result.message).not.toContain(`'recent'`);
// Three-bucket invariant: sum === sources.length (the load-bearing assertion).
expect(result.details).toEqual({
unchanged_count: 1, synced_recently_count: 1, stale_count: 1,
});
const { unchanged_count, synced_recently_count, stale_count } = result.details as any;
expect(unchanged_count + synced_recently_count + stale_count).toBe(3);
});
test('case 7: stale + matching HEAD + clean tree + chunker MISMATCH + localOnly=true → warn (chunker gate fires)', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
_setGitHeadProbeForTests(() => 'abc');
_setGitCleanProbeForTests(() => true);
const result = await checkSyncFreshness(makeStubEngine([
{ id: 'preupgrade', name: '', local_path: '/tmp/pre',
last_sync_at: agoMs(30 * 60 * 60 * 1000),
last_commit: 'abc',
chunker_version: '0', // STALE — bumped via gbrain upgrade since last sync
},
]), { localOnly: true });
expect(result.status).toBe('warn');
expect(result.details?.unchanged_count).toBe(0);
expect(result.details?.stale_count).toBe(1);
});
test('case 8: stale + matching HEAD + DIRTY tree + chunker match + localOnly=true → warn (dirty gate fires)', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
_setGitHeadProbeForTests(() => 'abc');
_setGitCleanProbeForTests(() => false); // dirty
const result = await checkSyncFreshness(makeStubEngine([
{ id: 'wip', name: '', local_path: '/tmp/wip',
last_sync_at: agoMs(30 * 60 * 60 * 1000),
last_commit: 'abc', chunker_version: currentChunkerVersion },
]), { localOnly: true });
expect(result.status).toBe('warn');
expect(result.details?.unchanged_count).toBe(0);
expect(result.details?.stale_count).toBe(1);
});
test('case 9 — D4 regression: localOnly=false (default) — git probes NEVER called', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
// Probes set up to return "everything matches" — IF they were called,
// the source would be marked unchanged. Since localOnly defaults false,
// they MUST NOT fire and the source MUST be flagged stale by time check.
let headCalls = 0;
let cleanCalls = 0;
_setGitHeadProbeForTests(() => { headCalls++; return 'matching-sha'; });
_setGitCleanProbeForTests(() => { cleanCalls++; return true; });
// Two callers shapes:
// (a) explicit localOnly:false matches doctorReportRemote semantics
// (b) omitted opts matches the default-fallthrough path
for (const opts of [{ localOnly: false }, undefined]) {
headCalls = 0;
cleanCalls = 0;
const result = await checkSyncFreshness(makeStubEngine([
{ id: 'remote-checked', name: '', local_path: '/tmp/x',
last_sync_at: agoMs(40 * 60 * 60 * 1000),
last_commit: 'matching-sha', chunker_version: currentChunkerVersion },
]), opts);
// Trust boundary: probes MUST NOT have been called.
expect(headCalls).toBe(0);
expect(cleanCalls).toBe(0);
// And without the short-circuit, time check fires the warn:
expect(result.status).toBe('warn');
expect(result.details?.unchanged_count).toBe(0);
expect(result.details?.stale_count).toBe(1);
}
});
});
// Supervisor crash classifier wiring. Pre-fix, doctor.ts:1013 counted every
// `worker_exited` event as a crash regardless of `likely_cause`, inflating
// `crashes_24h` to 120+/day from RSS-watchdog drains and SIGTERM stops.
+15
View File
@@ -19,6 +19,7 @@ import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { SemanticQueryCache, cacheRowId } from '../src/core/search/query-cache.ts';
import type { SearchResult } from '../src/core/types.ts';
import { knobsHash, resolveSearchMode } from '../src/core/search/mode.ts';
import { configureGateway, resetGateway } from '../src/core/ai/gateway.ts';
let engine: PGLiteEngine;
@@ -27,6 +28,19 @@ const balancedHash = knobsHash(resolveSearchMode({ mode: 'balanced' }));
const tokenmaxHash = knobsHash(resolveSearchMode({ mode: 'tokenmax' }));
beforeAll(async () => {
// v0.36.2.0: DEFAULT_EMBEDDING_DIMENSIONS flipped to 1280 (ZE Matryoshka).
// The makeEmbedding fixture below emits 1536-dim unit vectors. If we let
// initSchema() inherit the default, query_cache.embedding gets sized at
// halfvec(1280) and the inserts throw "expected 1280 dimensions, not 1536".
// Pin the gateway to 1536d so this file is hermetic regardless of
// gateway state from other tests in the shard. Pattern matches
// test/consolidate-valid-until.test.ts.
resetGateway();
configureGateway({
embedding_model: 'openai:text-embedding-3-large',
embedding_dimensions: 1536,
env: { OPENAI_API_KEY: 'sk-fake' },
});
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
@@ -34,6 +48,7 @@ beforeAll(async () => {
afterAll(async () => {
await engine.disconnect();
resetGateway();
});
beforeEach(async () => {