mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
* feat(extract): link/timeline extraction freshness watermark (#1696) Closes the "imported != curated" gap: plain `gbrain sync` only extracts CHANGED pages, so a brain with autopilot off accumulated a links table that was ~99.7% untyped `mentions` with nothing surfacing it. Adds a per-page freshness watermark (pages.links_extracted_at, migration v112) and three things built on it: - `gbrain extract --stale [--source-id] [--catch-up] [--dry-run] [--json]`: incremental DB-source link+timeline sweep over pages whose extraction is stale (never extracted, edited since, or extractor version bumped). Small byte-bounded batches, non-swallowing flush, stamp-after-flush so a crash re-extracts idempotently. Stamps with the row's READ updated_at (not now()) so a concurrent edit during the sweep stays stale instead of being lost. - `links_extraction_lag` doctor check (local + remote): warn-only by default (>20%), hard-fail only via GBRAIN_EXTRACTION_LAG_FAIL_PCT. Vacuous-skip <100 pages; pre-v112 brains graceful-skip. - `gbrain sync --no-extract` flag + end-of-sync nudge (fires on synced|first_sync|up_to_date so the initial import surfaces its backlog). Three new BrainEngine methods (countStalePagesForExtraction / listStalePagesForExtraction / markPagesExtractedBatch) with Postgres<->PGLite parity + bootstrap probes. Schema parity: schema.sql + regenerated pglite-schema.ts + schema-embedded.ts + bootstrap-coverage test. Migration v112 (composite (source_id, links_extracted_at) index, no backfill so the real backlog surfaces on first doctor run). * test(audit): hermetic GBRAIN_AUDIT_DIR override for prune ENOENT case The "no-op when audit dir does not exist (ENOENT)" case called pruneOldBatchRetryAuditFiles without a GBRAIN_AUDIT_DIR override, so it read the developer's real ~/.gbrain/audit and flaked (kept>0) on any machine with prior gbrain audit history. Point it at a guaranteed-nonexistent temp path so it tests the real missing-dir branch hermetically — matching the file header's "never touches ~/.gbrain/audit" contract. Pre-existing flake (introduced by v0.41.19.0 #1537), unrelated to #1696. * chore: bump version and changelog (v0.42.2.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: CLAUDE.md key-files entry for the #1696 extract-stale wave + regen llms-full --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
41 lines
1.5 KiB
TypeScript
41 lines
1.5 KiB
TypeScript
/**
|
|
* v0.42.7 (#1696, D5) — the end-of-sync extraction-lag nudge status gate.
|
|
*
|
|
* Codex caught that the single-source nudge was gated `=== 'synced'`, which
|
|
* skips `first_sync` (a fresh / --full import — the BIGGEST un-extracted
|
|
* backlog, the exact 280K-page scenario #1696 exists for) and `up_to_date`.
|
|
* `shouldNudgeAfterSync` is the pure predicate the call site now uses; this
|
|
* pins its contract so the gate can't silently narrow back to `synced` only.
|
|
*
|
|
* Pure predicate test — no PGLite, no env mutation, no module stubbing (R1-R4 compliant).
|
|
*/
|
|
|
|
import { describe, test, expect } from 'bun:test';
|
|
import { shouldNudgeAfterSync } from '../src/commands/sync.ts';
|
|
|
|
describe('shouldNudgeAfterSync (D5 status gate)', () => {
|
|
test('fires on first_sync — the biggest-backlog initial-import case', () => {
|
|
expect(shouldNudgeAfterSync('first_sync')).toBe(true);
|
|
});
|
|
|
|
test('fires on synced (incremental)', () => {
|
|
expect(shouldNudgeAfterSync('synced')).toBe(true);
|
|
});
|
|
|
|
test('fires on up_to_date (no-op sync over a pre-existing backlog)', () => {
|
|
expect(shouldNudgeAfterSync('up_to_date')).toBe(true);
|
|
});
|
|
|
|
test('does NOT fire on dry_run (preview, no real sync)', () => {
|
|
expect(shouldNudgeAfterSync('dry_run')).toBe(false);
|
|
});
|
|
|
|
test('does NOT fire on blocked_by_failures (inconsistent state)', () => {
|
|
expect(shouldNudgeAfterSync('blocked_by_failures')).toBe(false);
|
|
});
|
|
|
|
test('does NOT fire on partial (inconsistent state)', () => {
|
|
expect(shouldNudgeAfterSync('partial')).toBe(false);
|
|
});
|
|
});
|