mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
* feat(core): shared computeSyncDelta + spend-posture module (#2139) sync-delta.ts: ONE implementation of "what changed since last_commit", consumed by both the sync executor and the inline cost estimator so the gate's dollar figure can't drift from what the sync imports. spend-posture.ts: spend.posture config + parseUsdLimit/formatUsdLimit off-switch parsing (off/unlimited/none → Infinity; undefined at the budget boundary so ledger rows never serialize null). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(sync): delta-aware cost estimator + non-TTY auto-defer + per-source failure acks (#2139) The inline-embed cost gate was a ~400x phantom: it priced the entire tree whenever the working tree was dirty (always, on an active brain), then blocked the daily cron with exit 2. Now: - performSyncInner + the estimator both route through computeSyncDelta, so the estimate mirrors execution (fetch-first delta; dirty-but-caught-up tree → $0). - shouldBlockSync is posture-aware; non-TTY above floor AUTO-DEFERS embeds to capped backfill jobs (exit 0) instead of wedging — single shared runInlineCostGate on both --all and single-source paths. - --full prices delta + stale backlog (full sync sweeps it inline). - off/unlimited on the cost knobs; tokenmax bypasses the backfill cap (still ledgered) but never the cooldown. - --skip-failed/--retry-failed scoped per source; the D15 parallel refusal is lifted (the #1939 ledger is per-source + lock-serialized). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(config): register spend-control keys + validate spend.posture (#2139) Adds spend.posture + the five previously --force-only spend knobs to KNOWN_CONFIG_KEYS so `config set` accepts them directly (removes the archaeology the issue complained about), and rejects invalid spend.posture values at set time with a paste-ready hint. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reindex,enrich,onboard): spend.posture across the remaining cost gates (#2139) reindex-code: tokenmax makes the cost gate informational; --max-cost accepts off/unlimited. enrich + onboard --auto: tokenmax lifts the refuse-without-cap guardrail and runs UNCAPPED (spend still ledgered by BudgetTracker). Explicit --max-usd always wins over posture. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: cost-gate, delta estimator, spend-posture, off-switch coverage (#2139) New sync-delta + sync-cost-estimate unit suites; rewritten cost-gate serial tests (auto-defer instead of exit 2, posture, off-switch, format split, single-source); parseUsdLimit/posture-aware shouldBlockSync; backfill cap-off + tokenmax-bypass + cooldown-still-refuses; config known-key acceptance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(spend-controls): single spend-control surface + ref-map + follow-up TODOs (#2139) New docs/operations/spend-controls.md (every gate, key, default, off switch, posture interaction); CLAUDE.md reference-map row; two P3 follow-up TODOs (measured chunk-count gating, per-source defer granularity). llms bundles regenerated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(spend): SSRF-harden estimator fetch + complete off/uncapped across reindex/enrich/onboard (#2139) Ship-stage codex pre-landing review caught four P1s in the secondary cost gates: - The delta estimator's fetch-first ran `git fetch` through the plain git() helper, bypassing the GIT_SSRF_FLAGS + GIT_TERMINAL_PROMPT=0 hardening that real sync uses. Added `fetchRemote()` to git-remote.ts (same flags as pullRepo) and route the estimator through it — a cost preview / dry-run can no longer hit a remote through a less-protected path. - `reindex --max-cost off`, `enrich --max-usd off`, `onboard --auto --max-usd off` were parsed but didn't actually proceed/uncap. Now: explicit off (and spend.posture=tokenmax) proceed past the confirmation/missing-cap refusal AND run uncapped. enrich threads an Infinity sentinel mapped to "no BudgetTracker ceiling" (never raw Infinity → no null in audit rows); reindex/onboard use their native undefined=uncapped path. Spend still ledgered. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.42.45.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(KEY_FILES): update sync/embedding/git-remote/reindex entries to post-#2139 truth document-release pass: the cost-gate entries described the pre-#2139 behavior (full-tree-ceiling estimator, --skip-failed-rejects-under-parallel, exit-2 confirmation gate). Updated to current truth — delta-aware estimator via the shared computeSyncDelta, per-source failure acks under parallel, non-TTY auto-defer (no exit 2), posture-aware shouldBlockSync. Added entries for the two new core modules (sync-delta.ts, spend-posture.ts) + fetchRemote on git-remote.ts + reindex --max-cost off. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
616 lines
28 KiB
TypeScript
616 lines
28 KiB
TypeScript
/**
|
|
* Bug 9 regression — sync silently drops files with broken YAML.
|
|
*
|
|
* Before the fix, sync.ts caught per-file parse errors, printed a warning,
|
|
* and still advanced sync.last_commit. The failed file was never retried
|
|
* because it was behind the bookmark. Silent data loss.
|
|
*
|
|
* After the fix:
|
|
* - failures append to ~/.gbrain/sync-failures.jsonl (with dedup)
|
|
* - incremental + full-sync + import git-continuity paths gate the
|
|
* sync.last_commit advance on "no failures"
|
|
* - `gbrain sync --skip-failed` acknowledges the current set
|
|
* - `gbrain doctor` surfaces unacknowledged failures
|
|
*
|
|
* This suite exercises the helper + the dedup behavior. The full CLI
|
|
* round-trip is covered by E2E tests.
|
|
*/
|
|
|
|
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
|
import { mkdtempSync, rmSync, readFileSync, existsSync, writeFileSync } from 'fs';
|
|
import { join } from 'path';
|
|
import { tmpdir } from 'os';
|
|
|
|
// Point HOME at a tmpdir so we don't stomp the real ~/.gbrain/sync-failures.jsonl
|
|
let tmpHome: string;
|
|
const originalHome = process.env.HOME;
|
|
|
|
beforeEach(async () => {
|
|
tmpHome = mkdtempSync(join(tmpdir(), 'gbrain-sync-failures-'));
|
|
process.env.HOME = tmpHome;
|
|
// Belt-and-suspenders: explicitly clear the jsonl at the resolved path.
|
|
const { syncFailuresPath } = await import('../src/core/sync.ts');
|
|
try { rmSync(syncFailuresPath(), { force: true }); } catch { /* none */ }
|
|
});
|
|
|
|
afterEach(() => {
|
|
if (originalHome) process.env.HOME = originalHome;
|
|
else delete process.env.HOME;
|
|
try { rmSync(tmpHome, { recursive: true, force: true }); } catch { /* ignore */ }
|
|
});
|
|
|
|
describe('Bug 9 — sync-failures JSONL helpers', () => {
|
|
// issue #1939: recordSyncFailures now upserts by (source_id, path) and
|
|
// increments `attempts` (consecutive failed runs) instead of appending a row
|
|
// per (path, commit, error). One row per failing path; the attempt counter
|
|
// drives the bounded auto-skip valve.
|
|
test('recordSyncFailures upserts per path, incrementing attempts', async () => {
|
|
const { recordSyncFailures, loadSyncFailures, syncFailuresPath } = await import('../src/core/sync.ts');
|
|
|
|
recordSyncFailures([
|
|
{ path: 'people/alice.md', error: 'YAML: unexpected colon in title' },
|
|
{ path: 'notes/broken.md', error: 'YAML: duplicated key' },
|
|
], 'abc123def456');
|
|
|
|
expect(existsSync(syncFailuresPath())).toBe(true);
|
|
const entries = loadSyncFailures();
|
|
expect(entries.length).toBe(2);
|
|
const alice = entries.find(e => e.path === 'people/alice.md')!;
|
|
expect(alice.commit).toBe('abc123def456');
|
|
expect(alice.state).toBe('open');
|
|
expect(alice.attempts).toBe(1);
|
|
expect(alice.acknowledged).toBe(false);
|
|
|
|
// Same path again (same commit) → still ONE row, attempts climbs.
|
|
recordSyncFailures([
|
|
{ path: 'people/alice.md', error: 'YAML: unexpected colon in title' },
|
|
], 'abc123def456');
|
|
expect(loadSyncFailures().length).toBe(2);
|
|
expect(loadSyncFailures().find(e => e.path === 'people/alice.md')!.attempts).toBe(2);
|
|
|
|
// Different commit → still upserted (not appended), attempts keeps climbing.
|
|
recordSyncFailures([
|
|
{ path: 'people/alice.md', error: 'YAML: unexpected colon in title' },
|
|
], 'zzz999');
|
|
const after = loadSyncFailures();
|
|
expect(after.length).toBe(2);
|
|
const alice2 = after.find(e => e.path === 'people/alice.md')!;
|
|
expect(alice2.attempts).toBe(3);
|
|
expect(alice2.commit).toBe('zzz999');
|
|
});
|
|
|
|
test('acknowledgeSyncFailures marks unacked entries, leaves acked alone', async () => {
|
|
const { recordSyncFailures, acknowledgeSyncFailures, loadSyncFailures } = await import('../src/core/sync.ts');
|
|
|
|
recordSyncFailures([
|
|
{ path: 'a.md', error: 'err1' },
|
|
{ path: 'b.md', error: 'err2' },
|
|
], 'commit1');
|
|
|
|
const result = acknowledgeSyncFailures();
|
|
expect(result.count).toBe(2);
|
|
expect(result.summary.length).toBeGreaterThan(0);
|
|
const after = loadSyncFailures();
|
|
expect(after.every(e => e.acknowledged === true)).toBe(true);
|
|
expect(after.every(e => typeof e.acknowledged_at === 'string')).toBe(true);
|
|
|
|
// Second ack: nothing new to mark.
|
|
expect(acknowledgeSyncFailures().count).toBe(0);
|
|
|
|
// Adding a fresh failure then ack: only the new one flips.
|
|
recordSyncFailures([{ path: 'c.md', error: 'err3' }], 'commit2');
|
|
expect(acknowledgeSyncFailures().count).toBe(1);
|
|
expect(loadSyncFailures().length).toBe(3);
|
|
expect(loadSyncFailures().every(e => e.acknowledged === true)).toBe(true);
|
|
});
|
|
|
|
test('unacknowledgedSyncFailures filters correctly', async () => {
|
|
const { recordSyncFailures, acknowledgeSyncFailures, unacknowledgedSyncFailures } = await import('../src/core/sync.ts');
|
|
|
|
recordSyncFailures([{ path: 'a.md', error: 'err1' }], 'c1');
|
|
acknowledgeSyncFailures();
|
|
recordSyncFailures([{ path: 'b.md', error: 'err2' }], 'c2');
|
|
|
|
const unacked = unacknowledgedSyncFailures();
|
|
expect(unacked.length).toBe(1);
|
|
expect(unacked[0].path).toBe('b.md');
|
|
});
|
|
|
|
test('loadSyncFailures returns [] when file is missing', async () => {
|
|
const { loadSyncFailures } = await import('../src/core/sync.ts');
|
|
expect(loadSyncFailures()).toEqual([]);
|
|
});
|
|
|
|
test('loadSyncFailures tolerates malformed lines', async () => {
|
|
const { loadSyncFailures, syncFailuresPath, recordSyncFailures } = await import('../src/core/sync.ts');
|
|
// Seed one valid entry.
|
|
recordSyncFailures([{ path: 'a.md', error: 'err1' }], 'c1');
|
|
// Append garbage.
|
|
writeFileSync(syncFailuresPath(), readFileSync(syncFailuresPath(), 'utf-8') + 'NOT-JSON\n', { flag: 'w' });
|
|
const out = loadSyncFailures();
|
|
expect(out.length).toBe(1);
|
|
expect(out[0].path).toBe('a.md');
|
|
});
|
|
});
|
|
|
|
describe('Bug 9 — doctor surfaces sync failures', () => {
|
|
test('doctor source contains sync_failures check', async () => {
|
|
const source = await Bun.file(new URL('../src/commands/doctor.ts', import.meta.url)).text();
|
|
expect(source).toContain('sync_failures');
|
|
expect(source).toContain('unacknowledgedSyncFailures');
|
|
expect(source).toContain("'gbrain sync --skip-failed'");
|
|
});
|
|
|
|
// issue #1939: BOTH doctor surfaces (local buildChecks + remote
|
|
// doctorReportRemote) must decide severity through the one shared helper so
|
|
// they can never drift. The remote surface must no longer hand-roll an
|
|
// `acknowledged_at` count (the old field-split that caused drift).
|
|
test('both doctor surfaces use the shared decideSyncFailureSeverity', async () => {
|
|
const source = await Bun.file(new URL('../src/commands/doctor.ts', import.meta.url)).text();
|
|
const occurrences = source.split('decideSyncFailureSeverity').length - 1;
|
|
expect(occurrences).toBeGreaterThanOrEqual(2); // local + remote
|
|
// remote no longer counts `!entry.acknowledged_at` by hand.
|
|
expect(source).not.toContain('if (!entry.acknowledged_at) unacked++');
|
|
});
|
|
});
|
|
|
|
describe('Bug 9 — sync.ts CLI flag wiring', () => {
|
|
test('runSync parses --skip-failed and --retry-failed flags', async () => {
|
|
const source = await Bun.file(new URL('../src/commands/sync.ts', import.meta.url)).text();
|
|
expect(source).toContain("args.includes('--skip-failed')");
|
|
expect(source).toContain("args.includes('--retry-failed')");
|
|
expect(source).toContain('skipFailed');
|
|
expect(source).toContain('retryFailed');
|
|
});
|
|
|
|
test('runSync acks pre-existing unacked failures up-front when --skip-failed is set', async () => {
|
|
// Without this gate, a user who fixes their broken YAML, re-runs sync
|
|
// (which finds nothing new and prints "Already up to date."), and then
|
|
// runs `gbrain sync --skip-failed` to clear the log gets a no-op —
|
|
// performSync's inner ack path only fires when failedFiles.length > 0
|
|
// in the current run. This test pins the up-front ack at the top of
|
|
// runSync so the flag means "ack whatever is currently flagged".
|
|
// v0.42.42.0 (#2139, D13C): the pre-ack is now scoped PER SOURCE — `--all`
|
|
// acks every source, single-source acks only its own.
|
|
const source = await Bun.file(new URL('../src/commands/sync.ts', import.meta.url)).text();
|
|
expect(source).toMatch(/if \(skipFailed\) \{[\s\S]*?syncAll \? acknowledgeFailures\(\) : acknowledgeFailures\(sourceId\)/);
|
|
});
|
|
|
|
test('acknowledgeSyncFailures clears stale failures end-to-end', async () => {
|
|
// Behavioral pin: the helper that --skip-failed delegates to must
|
|
// clear failures regardless of any current-run state. Mirrors the
|
|
// recovery flow: file fixed → sync clean → user wants log cleared.
|
|
const { recordSyncFailures, acknowledgeSyncFailures, unacknowledgedSyncFailures } = await import('../src/core/sync.ts');
|
|
recordSyncFailures([
|
|
{ path: 'people/old-broken.md', error: 'YAML: bad block mapping' },
|
|
{ path: 'people/old-broken.md', error: 'YAML: bad block mapping' }, // dup, dedup'd by recordSyncFailures
|
|
{ path: 'meetings/stale.md', error: 'YAML: multiline key' },
|
|
], 'old-commit');
|
|
expect(unacknowledgedSyncFailures().length).toBe(2);
|
|
const result = acknowledgeSyncFailures();
|
|
expect(result.count).toBe(2);
|
|
expect(unacknowledgedSyncFailures().length).toBe(0);
|
|
});
|
|
|
|
test('performSync gates the bookmark through the shared failure ledger', async () => {
|
|
const source = await Bun.file(new URL('../src/commands/sync.ts', import.meta.url)).text();
|
|
// issue #1939: the gate is now the shared applySyncFailureGate orchestrator.
|
|
expect(source).toContain('applySyncFailureGate');
|
|
expect(source).toContain('blocked_by_failures');
|
|
});
|
|
|
|
test('performFullSync routes failures through the same shared gate', async () => {
|
|
const source = await Bun.file(new URL('../src/commands/sync.ts', import.meta.url)).text();
|
|
expect(source).toContain('result.failures');
|
|
expect(source).toContain('applySyncFailureGate');
|
|
});
|
|
|
|
test('runImport returns RunImportResult and records via the ledger', async () => {
|
|
const source = await Bun.file(new URL('../src/commands/import.ts', import.meta.url)).text();
|
|
expect(source).toContain('RunImportResult');
|
|
expect(source).toContain('failures: Array<{ path: string; error: string }>');
|
|
// issue #1939: import records source-scoped via recordFailures (sourceId, …).
|
|
expect(source).toContain('recordFailures');
|
|
});
|
|
});
|
|
|
|
describe('classifyErrorCode — error message to code mapping', () => {
|
|
test('classifies SLUG_MISMATCH from error message', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode(
|
|
'Frontmatter slug "my-friend-mike" does not match path-derived slug "2008-03-20-my-friend-mike"'
|
|
)).toBe('SLUG_MISMATCH');
|
|
});
|
|
|
|
test('classifies YAML_PARSE from error message', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('YAML parse failed: unexpected colon in title')).toBe('YAML_PARSE');
|
|
});
|
|
|
|
test('classifies YAML_DUPLICATE_KEY', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('YAMLException: duplicated mapping key')).toBe('YAML_DUPLICATE_KEY');
|
|
});
|
|
|
|
test('classifies STATEMENT_TIMEOUT', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('canceling statement due to statement timeout')).toBe('STATEMENT_TIMEOUT');
|
|
});
|
|
|
|
test('classifies NULL_BYTES', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('invalid UTF-8: null byte at position 3770')).toBe('NULL_BYTES');
|
|
});
|
|
|
|
test('classifies INVALID_UTF8', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('invalid UTF-8 sequence at position 500')).toBe('INVALID_UTF8');
|
|
});
|
|
|
|
test('classifies FILE_TOO_LARGE across all three production sites', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
// src/core/import-file.ts:352 — OS-level file size on disk
|
|
expect(classifyErrorCode('File too large (8432105 bytes)')).toBe('FILE_TOO_LARGE');
|
|
// src/core/import-file.ts:199 — content size limit (5MB cap)
|
|
expect(classifyErrorCode('Content too large (6000000 bytes, max 5000000). Split the content into smaller files or remove large embedded assets.')).toBe('FILE_TOO_LARGE');
|
|
// src/core/import-file.ts:401 — code file size cap
|
|
expect(classifyErrorCode('Code file too large (8000000 bytes)')).toBe('FILE_TOO_LARGE');
|
|
});
|
|
|
|
test('classifies SYMLINK_NOT_ALLOWED from import-file.ts symlink rejection', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('Skipping symlink: /path/to/link.md')).toBe('SYMLINK_NOT_ALLOWED');
|
|
});
|
|
|
|
test('returns UNKNOWN for unrecognized errors', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('something completely different')).toBe('UNKNOWN');
|
|
});
|
|
});
|
|
|
|
describe('summarizeFailuresByCode — grouped summary', () => {
|
|
test('groups failures by classified code', async () => {
|
|
const { summarizeFailuresByCode } = await import('../src/core/sync.ts');
|
|
const summary = summarizeFailuresByCode([
|
|
{ error: 'Frontmatter slug "a" does not match path-derived slug "b"' },
|
|
{ error: 'Frontmatter slug "c" does not match path-derived slug "d"' },
|
|
{ error: 'YAML parse failed: bad colon' },
|
|
{ error: 'something unknown' },
|
|
]);
|
|
expect(summary).toEqual([
|
|
{ code: 'SLUG_MISMATCH', count: 2 },
|
|
{ code: 'YAML_PARSE', count: 1 },
|
|
{ code: 'UNKNOWN', count: 1 },
|
|
]);
|
|
});
|
|
|
|
test('respects pre-classified code field', async () => {
|
|
const { summarizeFailuresByCode } = await import('../src/core/sync.ts');
|
|
const summary = summarizeFailuresByCode([
|
|
{ error: 'anything', code: 'SLUG_MISMATCH' },
|
|
{ error: 'anything', code: 'SLUG_MISMATCH' },
|
|
{ error: 'anything', code: 'YAML_PARSE' },
|
|
]);
|
|
expect(summary).toEqual([
|
|
{ code: 'SLUG_MISMATCH', count: 2 },
|
|
{ code: 'YAML_PARSE', count: 1 },
|
|
]);
|
|
});
|
|
|
|
test('returns empty array for no failures', async () => {
|
|
const { summarizeFailuresByCode } = await import('../src/core/sync.ts');
|
|
expect(summarizeFailuresByCode([])).toEqual([]);
|
|
});
|
|
});
|
|
|
|
describe('acknowledgeSyncFailures — structured return', () => {
|
|
test('returns count and code summary', async () => {
|
|
const { recordSyncFailures, acknowledgeSyncFailures } = await import('../src/core/sync.ts');
|
|
recordSyncFailures([
|
|
{ path: 'a.md', error: 'Frontmatter slug "x" does not match path-derived slug "y"' },
|
|
{ path: 'b.md', error: 'Frontmatter slug "p" does not match path-derived slug "q"' },
|
|
{ path: 'c.md', error: 'YAML parse failed: bad' },
|
|
], 'commit1');
|
|
|
|
const result = acknowledgeSyncFailures();
|
|
expect(result.count).toBe(3);
|
|
expect(result.summary).toEqual([
|
|
{ code: 'SLUG_MISMATCH', count: 2 },
|
|
{ code: 'YAML_PARSE', count: 1 },
|
|
]);
|
|
});
|
|
});
|
|
|
|
describe('recordSyncFailures — code field', () => {
|
|
test('records classified code alongside error message', async () => {
|
|
const { recordSyncFailures, loadSyncFailures } = await import('../src/core/sync.ts');
|
|
recordSyncFailures([
|
|
{ path: 'a.md', error: 'Frontmatter slug "x" does not match path-derived slug "y"' },
|
|
], 'commit1');
|
|
|
|
const entries = loadSyncFailures();
|
|
expect(entries[0].code).toBe('SLUG_MISMATCH');
|
|
});
|
|
});
|
|
|
|
// classifyErrorCode disambiguates Postgres unique-constraint errors from
|
|
// YAML duplicate-key errors. Pre-fix, every "duplicate.*key" string mapped
|
|
// to YAML_DUPLICATE_KEY, which mislabels DB-layer failures during sync.
|
|
describe('classifyErrorCode — DB vs YAML duplicate-key disambiguation', () => {
|
|
test('Postgres unique-constraint violation classifies as DB_DUPLICATE_KEY', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode(
|
|
'duplicate key value violates unique constraint "pages_slug_key"'
|
|
)).toBe('DB_DUPLICATE_KEY');
|
|
});
|
|
|
|
test('YAML duplicated mapping key still classifies as YAML_DUPLICATE_KEY', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('YAMLException: duplicated mapping key "title"'))
|
|
.toBe('YAML_DUPLICATE_KEY');
|
|
});
|
|
|
|
test('DB pattern is checked BEFORE YAML so DB errors are not mislabeled', async () => {
|
|
// Both patterns historically matched /duplicate.*key/i — order matters now.
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode(
|
|
'duplicate key value violates unique constraint on table "pages"'
|
|
)).toBe('DB_DUPLICATE_KEY');
|
|
expect(classifyErrorCode(
|
|
'duplicate key value violates unique constraint on table "pages"'
|
|
)).not.toBe('YAML_DUPLICATE_KEY');
|
|
});
|
|
});
|
|
|
|
// classifyErrorCode matches the canonical messages emitted by
|
|
// collectValidationErrors() in src/core/markdown.ts. Pre-fix, the regexes
|
|
// keyed off "missing open" / "missing close" / "empty frontmatter" — none
|
|
// of which are produced upstream. Today these all classify correctly.
|
|
describe('classifyErrorCode — canonical message coverage', () => {
|
|
test('MISSING_OPEN matches "File is empty or whitespace-only"', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode(
|
|
'File is empty or whitespace-only; expected frontmatter starting with ---'
|
|
)).toBe('MISSING_OPEN');
|
|
});
|
|
|
|
test('MISSING_OPEN matches "Frontmatter must start with ---"', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode(
|
|
'Frontmatter must start with --- on the first non-empty line'
|
|
)).toBe('MISSING_OPEN');
|
|
});
|
|
|
|
test('MISSING_CLOSE matches "No closing --- delimiter"', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('No closing --- delimiter found')).toBe('MISSING_CLOSE');
|
|
});
|
|
|
|
test('MISSING_CLOSE matches "Heading at line N found inside frontmatter"', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode(
|
|
'Heading at line 5 found inside frontmatter zone (closing --- comes after)'
|
|
)).toBe('MISSING_CLOSE');
|
|
});
|
|
|
|
test('EMPTY_FRONTMATTER matches "Frontmatter block is empty"', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('Frontmatter block is empty')).toBe('EMPTY_FRONTMATTER');
|
|
});
|
|
|
|
test('NULL_BYTES matches "Content contains null bytes"', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('Content contains null bytes (likely binary corruption)'))
|
|
.toBe('NULL_BYTES');
|
|
});
|
|
|
|
test('NESTED_QUOTES matches "Nested double quotes"', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('Nested double quotes in YAML value at line 3'))
|
|
.toBe('NESTED_QUOTES');
|
|
});
|
|
});
|
|
|
|
// acknowledgeSyncFailures backfills `code` on legacy entries that were
|
|
// recorded before the code field existed (~/.gbrain/sync-failures.jsonl
|
|
// from pre-PR brains). Without this branch, upgraded users see "UNKNOWN"
|
|
// for every previously-recorded failure even when the message is parseable.
|
|
describe('acknowledgeSyncFailures — backfill on legacy entries', () => {
|
|
test('backfills code on entries that predate the code field', async () => {
|
|
const { acknowledgeSyncFailures, loadSyncFailures, syncFailuresPath } =
|
|
await import('../src/core/sync.ts');
|
|
|
|
// Hand-write a legacy entry with no `code` field. Mimics a pre-PR
|
|
// ~/.gbrain/sync-failures.jsonl row that exists on real upgrades.
|
|
const { mkdirSync } = await import('fs');
|
|
const { dirname } = await import('path');
|
|
mkdirSync(dirname(syncFailuresPath()), { recursive: true });
|
|
writeFileSync(
|
|
syncFailuresPath(),
|
|
JSON.stringify({
|
|
path: 'a.md',
|
|
error: 'Frontmatter slug "x" does not match path-derived slug "y"',
|
|
commit: 'old',
|
|
ts: '2025-01-01T00:00:00Z',
|
|
}) + '\n',
|
|
);
|
|
|
|
const result = acknowledgeSyncFailures();
|
|
expect(result.count).toBe(1);
|
|
expect(result.summary).toEqual([{ code: 'SLUG_MISMATCH', count: 1 }]);
|
|
|
|
const after = loadSyncFailures();
|
|
expect(after).toHaveLength(1);
|
|
expect(after[0].code).toBe('SLUG_MISMATCH');
|
|
expect(after[0].acknowledged).toBe(true);
|
|
});
|
|
|
|
test('preserves existing code field; never reclassifies', async () => {
|
|
const { acknowledgeSyncFailures, loadSyncFailures, syncFailuresPath } =
|
|
await import('../src/core/sync.ts');
|
|
|
|
const { mkdirSync } = await import('fs');
|
|
const { dirname } = await import('path');
|
|
mkdirSync(dirname(syncFailuresPath()), { recursive: true });
|
|
// Pre-classified entry — should NOT be re-run through classifier.
|
|
writeFileSync(
|
|
syncFailuresPath(),
|
|
JSON.stringify({
|
|
path: 'a.md',
|
|
error: 'some message that would otherwise classify as UNKNOWN',
|
|
code: 'CUSTOM_CODE',
|
|
commit: 'x',
|
|
ts: '2025-01-01T00:00:00Z',
|
|
}) + '\n',
|
|
);
|
|
|
|
const result = acknowledgeSyncFailures();
|
|
expect(result.summary).toEqual([{ code: 'CUSTOM_CODE', count: 1 }]);
|
|
expect(loadSyncFailures()[0].code).toBe('CUSTOM_CODE');
|
|
});
|
|
});
|
|
|
|
// formatCodeBreakdown is the DRY helper used by both the failures-array
|
|
// path (sync.ts blocked-by-failures + full-sync stderr) and the pre-summarized
|
|
// AcknowledgeResult.summary path (--skip-failed ack message). One renderer,
|
|
// two input shapes.
|
|
describe('formatCodeBreakdown — dual input shape', () => {
|
|
test('renders raw failures by classifying internally', async () => {
|
|
const { formatCodeBreakdown } = await import('../src/core/sync.ts');
|
|
const out = formatCodeBreakdown([
|
|
{ error: 'Frontmatter slug "a" does not match path-derived slug "b"' },
|
|
{ error: 'Frontmatter slug "c" does not match path-derived slug "d"' },
|
|
{ error: 'YAML parse failed: bad' },
|
|
]);
|
|
expect(out).toBe(' SLUG_MISMATCH: 2\n YAML_PARSE: 1');
|
|
});
|
|
|
|
test('renders pre-summarized {code, count} input directly', async () => {
|
|
const { formatCodeBreakdown } = await import('../src/core/sync.ts');
|
|
const out = formatCodeBreakdown([
|
|
{ code: 'SLUG_MISMATCH', count: 5 },
|
|
{ code: 'YAML_PARSE', count: 2 },
|
|
]);
|
|
expect(out).toBe(' SLUG_MISMATCH: 5\n YAML_PARSE: 2');
|
|
});
|
|
|
|
test('returns empty string for empty input', async () => {
|
|
const { formatCodeBreakdown } = await import('../src/core/sync.ts');
|
|
expect(formatCodeBreakdown([])).toBe('');
|
|
});
|
|
});
|
|
|
|
// v0.41.6.0 D2 — embedding error classifier patterns.
|
|
// Verbatim provider error strings extracted from:
|
|
// src/core/ai/gateway.ts:973-988 (native-openai / native-google)
|
|
// src/core/ai/gateway.ts:995-997 (native-anthropic — no-touchpoint shape)
|
|
// src/core/ai/gateway.ts:250 (defaultResolveAuth — openai-compatible)
|
|
// Each test pins a real-shaped message so a future provider-rename
|
|
// (recipe.name change) will fail loudly here instead of silently
|
|
// re-bucketing to UNKNOWN.
|
|
describe('v0.41.6.0 D2 — embedding error classification', () => {
|
|
test('EMBEDDING_NO_CREDS matches native-openai verbatim throw', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('OpenAI embedding requires OPENAI_API_KEY.')).toBe('EMBEDDING_NO_CREDS');
|
|
});
|
|
|
|
test('EMBEDDING_NO_CREDS matches native-google verbatim throw', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode(
|
|
'Google embedding requires GOOGLE_GENERATIVE_AI_API_KEY.'
|
|
)).toBe('EMBEDDING_NO_CREDS');
|
|
});
|
|
|
|
test('EMBEDDING_NO_CREDS matches Voyage AI openai-compat shape', async () => {
|
|
// defaultResolveAuth template: "${recipe.name} embedding requires ${REQUIRED_ENV}."
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('Voyage AI embedding requires VOYAGE_API_KEY.')).toBe('EMBEDDING_NO_CREDS');
|
|
});
|
|
|
|
test('EMBEDDING_NO_CREDS matches ZeroEntropy openai-compat shape', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('ZeroEntropy embedding requires ZEROENTROPY_API_KEY.')).toBe('EMBEDDING_NO_CREDS');
|
|
});
|
|
|
|
test('EMBEDDING_NO_CREDS matches DeepSeek openai-compat shape', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('DeepSeek embedding requires DEEPSEEK_API_KEY.')).toBe('EMBEDDING_NO_CREDS');
|
|
});
|
|
|
|
test('EMBEDDING_NO_CREDS matches the literal token (back-compat)', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('EMBEDDING_NO_CREDS — VOYAGE_API_KEY missing')).toBe('EMBEDDING_NO_CREDS');
|
|
});
|
|
|
|
test('EMBEDDING_NO_TOUCHPOINT matches anthropic-as-embed-provider misconfig', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode(
|
|
'Anthropic has no embedding model. Use openai or google for embeddings.'
|
|
)).toBe('EMBEDDING_NO_TOUCHPOINT');
|
|
});
|
|
|
|
test('EMBEDDING_RATE_LIMIT matches HTTP 429 phrasing', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('Request failed with status 429: rate limit exceeded')).toBe('EMBEDDING_RATE_LIMIT');
|
|
});
|
|
|
|
test('EMBEDDING_RATE_LIMIT matches OpenAI "too many requests" phrasing', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('OpenAIRateLimitError: too many requests in 1m'))
|
|
.toBe('EMBEDDING_RATE_LIMIT');
|
|
});
|
|
|
|
test('EMBEDDING_QUOTA matches OpenAI insufficient_quota error', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode(
|
|
'You exceeded your current quota, please check your plan and billing details. error code: insufficient_quota'
|
|
)).toBe('EMBEDDING_QUOTA');
|
|
});
|
|
|
|
test('EMBEDDING_QUOTA matches Anthropic credit-balance message', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('Your credit balance is too low to continue'))
|
|
.toBe('EMBEDDING_QUOTA');
|
|
});
|
|
|
|
test('EMBEDDING_OVERSIZE matches OpenAI max-context-length error', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode(
|
|
"This model's maximum context length is 8192 tokens, however you requested 9001 tokens"
|
|
)).toBe('EMBEDDING_OVERSIZE');
|
|
});
|
|
|
|
test('EMBEDDING_OVERSIZE matches max_tokens shape', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('max_tokens exceeded for embedding input')).toBe('EMBEDDING_OVERSIZE');
|
|
});
|
|
|
|
test('EMBEDDING_OVERSIZE matches Voyage "input length exceeds" shape', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('input length exceeds maximum')).toBe('EMBEDDING_OVERSIZE');
|
|
});
|
|
|
|
// Negative regression cases — make sure new patterns don't steal existing patterns' messages.
|
|
test('FILE_TOO_LARGE still classifies correctly (not overridden by new patterns)', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('File too large: 5242881 bytes')).toBe('FILE_TOO_LARGE');
|
|
});
|
|
|
|
test('STATEMENT_TIMEOUT still classifies correctly', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('canceling statement due to statement timeout')).toBe('STATEMENT_TIMEOUT');
|
|
});
|
|
|
|
test('SLUG_MISMATCH still classifies correctly', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode(
|
|
'Frontmatter slug "x" does not match path-derived slug "y"'
|
|
)).toBe('SLUG_MISMATCH');
|
|
});
|
|
|
|
test('UNKNOWN still fires when no pattern matches', async () => {
|
|
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
|
expect(classifyErrorCode('some random unmatched error message')).toBe('UNKNOWN');
|
|
});
|
|
});
|