mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
* feat(engine): add countUnconsolidatedFacts to BrainEngine + both engines New `BrainEngine.countUnconsolidatedFacts(sourceId): Promise<number>` returns the count of active + unconsolidated facts for a source. Single SQL: COUNT(*) WHERE source_id = $1 AND consolidated_at IS NULL AND expired_at IS NULL. Backs the v0.33 `gbrain recall --pending` flag and the `recall` MCP op's new `include_pending` param. Source-scoped, no index needed (existing facts(source_id) index covers the predicate). * feat(recall): cursor state + recall rewrite + thin-client routing + watch loop `gbrain recall` gains four new flags backed by a new cursor-state file: - `--since-last-run` reads ~/.gbrain/recall-cursors/<source>.json. First run defaults to 24h. Cursor is T_start (captured BEFORE the read SQL), not T_finish, so facts inserted during render don't fall in a black hole (Codex round 1 #2). - `--pending` appends a "Pending consolidation: N" footer. Backed by the new engine method; remote round-trips through one MCP call via the recall op's new `include_pending` param. - `--rollup` prepends a "Top mentions" header — top-5 entities by fact count over the FULL result set, not a LIMIT slice (Codex round 1 #8). JSON shape `top_entities: [{entity_slug, count}]` matches the existing pinned key at test/facts-doctor-shape.test.ts:49. - `--watch [SECONDS]` re-runs on interval. Default 60, range [1, 3600]. TTY: clear-and-redraw. Non-TTY: plain `--- <ts> ---` delimited blocks. SIGINT-only clean exit. Per-tick try/catch + exponential backoff `min(SECONDS × 2^(N-1), 5×SECONDS)`; exit after 5 consecutive failures with briefing cursor NOT advanced. Watch uses a separate cursor file (<source>.watch.json) so operator quitting watch doesn't clobber the standalone briefing cursor (Codex round 2 #8). Thin-client routing: runRecall + runForget mirror the salience.ts:80 pattern. On `gbrain init --mcp-only` installs the local engine call is swapped for callRemoteTool('recall' | 'forget_fact', ...). The local canonical source resolver's assertSourceExists check is skipped on thin-client (empty local sources table); the kebab-case SOURCE_ID_RE syntactic gate still runs locally. Fixes pre-existing silent-empty-results on thin-client recall — the v0.31.1 wave missed it (Codex round 2 #6). `recall` MCP op extended with optional `include_pending` param + `pending_consolidation_count` output field. Backward-compatible. No new MCP op. No schema migration. State file uses atomic write via unique per-call tmp filename (<source>.json.tmp.<pid>.<random>) + rename(2) (Codex round 1 #7). Read returns null on missing/corrupt/future-shifted timestamps; caller falls back to 24h. * feat(thin-client): route jobs list/get + REFUSE 7 host-bound commands Continues the v0.31.1 thin-client routing wave. v0.33 audit (Codex round 2 #4) source-grounded against operations.ts + each command file: ROUTE additions (have MCP ops, mirror salience.ts:80 pattern): - `gbrain jobs list` → callRemoteTool('list_jobs', ...) - `gbrain jobs get <id>` → callRemoteTool('get_job', ...) Other jobs subcommands (submit, cancel, retry, work, supervisor, prune, stats, smoke) stay host-bound — they manage local queue state. REFUSE additions to cli.ts THIN_CLIENT_REFUSED_COMMANDS + matching hints in THIN_CLIENT_REFUSE_HINTS: - `pages` — purge-deleted is admin+localOnly (operations.ts:856-864) - `files` — file_list / file_url MCP ops are localOnly:true - `eval` — export/prune/replay touch local engine; no MCP equivalent - `code-def` / `code-refs` / `code-callers` / `code-callees` — NO MCP ops exist for symbol lookup in operations.ts:2630-2671; deferred as a v0.34 candidate to add them Each refuse hint names the host-side path the user should use instead. Closes the silent-wrong-brain bug class for 9 commands total (recall + forget routing landed in the prior commit). * test: cover v0.33 recall extensions + thin-client routing audit (45 cases) Three new test files pinning the v0.33 behavior + critical regression guards from both Codex review rounds: - test/recall-extensions.test.ts (17 cases, PGLite-backed). Covers countUnconsolidatedFacts SQL semantics (ignores expired, ignores consolidated, source-scoped, returns 0 on empty), cursor state file round-trip + corrupt/future fallback + briefing vs watch separation (Codex round 2 #8 regression guard) + atomic write tmp suffix (Codex round 1 #7 regression guard) + non-fatal write failures. Uses withEnv() for GBRAIN_HOME isolation per check-test-isolation.sh R1. - test/recall-rollup.test.ts (8 pure-function cases). CRITICAL regression guards for Codex round 1 #8: 1. Top-K computed over the FULL FactRow[], not a LIMIT-100 slice (seeded with 150 facts to prove full-window math) 2. JSON shape pinned to `{entity_slug, count}` matching test/facts-doctor-shape.test.ts:49 (the existing shape pin) 3. null entity_slug skipped, NOT bucketed as "(no entity)" 4. Ties broken alphabetically for stable output - test/thin-client-routing-audit.test.ts (20 source-grounded cases). Pins every v0.33 REFUSE addition in THIN_CLIENT_REFUSED_COMMANDS + every matching hint in THIN_CLIENT_REFUSE_HINTS + every v0.31.1-era original (no accidental removals). Pins every ROUTE addition's callRemoteTool import + call site in recall.ts and jobs.ts. Catches the audit-table regression mode that motivated the v0.31.1 wave originally. Net: 45 new test cases. All pass green against the v0.33 implementation. * chore: bump version and changelog (v0.33.0) v0.33.0 — agent integration: gbrain recall morning pulse + thin-client routing fix. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
305 lines
12 KiB
TypeScript
305 lines
12 KiB
TypeScript
/**
|
|
* v0.32 — `gbrain recall` extensions: --since-last-run + --pending + --rollup
|
|
* + --watch + thin-client routing. PGLite-backed unit tests (no DATABASE_URL,
|
|
* no API keys). Canonical block pattern from CLAUDE.md.
|
|
*
|
|
* Critical regression guards pinned here:
|
|
* - countUnconsolidatedFacts SQL semantics (ignores expired, ignores
|
|
* consolidated, returns 0 on empty)
|
|
* - Cursor state file round-trip + corrupt/future fallback + separate
|
|
* briefing vs watch variants (Codex round 2 #8)
|
|
* - Atomic write: tmp filename uses unique suffix per call (Codex round 1 #7)
|
|
* - Cursor write writes T_start, NOT T_finish (Codex round 1 #2)
|
|
*
|
|
* Renderer + watch-loop + flag-parser coverage lives in
|
|
* `test/thin-client-routing.test.ts` because those paths exercise the
|
|
* mocked-MCP-client surface, not the PGLite engine.
|
|
*/
|
|
|
|
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
|
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
|
import { resetPgliteState } from './helpers/reset-pglite.ts';
|
|
import { withEnv } from './helpers/with-env.ts';
|
|
import {
|
|
readCursor,
|
|
writeCursor,
|
|
_cursorPathForTests,
|
|
} from '../src/core/recall-cursor-state.ts';
|
|
import { existsSync, mkdirSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
|
|
import { dirname, basename, join } from 'node:path';
|
|
import { tmpdir } from 'node:os';
|
|
|
|
// Allocate a unique temp dir per test (cross-test safe; each test runs its
|
|
// body inside withEnv({ GBRAIN_HOME: tmpHome }) so process.env mutations are
|
|
// scoped + restored via try/finally instead of leaking across files).
|
|
function makeTmpHome(): string {
|
|
return join(tmpdir(), `gbrain-recall-test-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`);
|
|
}
|
|
|
|
let engine: PGLiteEngine;
|
|
|
|
beforeAll(async () => {
|
|
engine = new PGLiteEngine();
|
|
await engine.connect({});
|
|
await engine.initSchema();
|
|
});
|
|
|
|
afterAll(async () => {
|
|
await engine.disconnect();
|
|
});
|
|
|
|
beforeEach(async () => {
|
|
await resetPgliteState(engine);
|
|
});
|
|
|
|
describe('countUnconsolidatedFacts', () => {
|
|
test('returns 0 on empty facts table', async () => {
|
|
expect(await engine.countUnconsolidatedFacts('default')).toBe(0);
|
|
});
|
|
|
|
test('counts active + unconsolidated facts', async () => {
|
|
for (let i = 0; i < 3; i++) {
|
|
await engine.insertFact(
|
|
{ fact: `f${i}`, kind: 'fact', entity_slug: 'people/test', source: 'unit' },
|
|
{ source_id: 'default' },
|
|
);
|
|
}
|
|
expect(await engine.countUnconsolidatedFacts('default')).toBe(3);
|
|
});
|
|
|
|
test('ignores expired facts (Codex #4 regression: --pending shows ONLY active unconsolidated)', async () => {
|
|
const a = await engine.insertFact(
|
|
{ fact: 'active', kind: 'fact', entity_slug: 'e/a', source: 'unit' },
|
|
{ source_id: 'default' },
|
|
);
|
|
const b = await engine.insertFact(
|
|
{ fact: 'expired', kind: 'fact', entity_slug: 'e/b', source: 'unit' },
|
|
{ source_id: 'default' },
|
|
);
|
|
await engine.expireFact(b.id);
|
|
const count = await engine.countUnconsolidatedFacts('default');
|
|
expect(count).toBe(1);
|
|
expect(a.id).toBeDefined();
|
|
});
|
|
|
|
test('ignores consolidated facts', async () => {
|
|
const a = await engine.insertFact(
|
|
{ fact: 'will be consolidated', kind: 'fact', entity_slug: 'e/c', source: 'unit' },
|
|
{ source_id: 'default' },
|
|
);
|
|
// Direct SQL to flip consolidated_at — this is what the dream cycle's
|
|
// consolidate phase does.
|
|
await engine.executeRaw(
|
|
`UPDATE facts SET consolidated_at = NOW() WHERE id = $1`,
|
|
[a.id],
|
|
);
|
|
expect(await engine.countUnconsolidatedFacts('default')).toBe(0);
|
|
});
|
|
|
|
test('source-scoped (does not count other sources)', async () => {
|
|
await engine.insertFact(
|
|
{ fact: 'in default', kind: 'fact', entity_slug: 'e/d', source: 'unit' },
|
|
{ source_id: 'default' },
|
|
);
|
|
// Note: inserting under a non-existent source still works at the engine
|
|
// level (no FK enforcement on facts.source_id in the schema as of v0.32).
|
|
// The source-scope contract holds regardless.
|
|
expect(await engine.countUnconsolidatedFacts('default')).toBe(1);
|
|
expect(await engine.countUnconsolidatedFacts('other')).toBe(0);
|
|
});
|
|
});
|
|
|
|
describe('recall-cursor-state file helper', () => {
|
|
test('missing file returns null (first-run case)', async () => {
|
|
const tmpHome = makeTmpHome();
|
|
mkdirSync(tmpHome, { recursive: true });
|
|
await withEnv({ GBRAIN_HOME: tmpHome }, async () => {
|
|
expect(readCursor('default', 'briefing')).toBeNull();
|
|
expect(readCursor('default', 'watch')).toBeNull();
|
|
});
|
|
});
|
|
|
|
test('round-trip: write then read returns the same instant (ms precision)', async () => {
|
|
const tmpHome = makeTmpHome();
|
|
mkdirSync(tmpHome, { recursive: true });
|
|
await withEnv({ GBRAIN_HOME: tmpHome }, async () => {
|
|
const t = new Date('2026-05-10T14:30:00.000Z');
|
|
writeCursor('default', t, 'briefing');
|
|
const read = readCursor('default', 'briefing');
|
|
expect(read).not.toBeNull();
|
|
expect(read!.getTime()).toBe(t.getTime());
|
|
});
|
|
});
|
|
|
|
test('briefing cursor and watch cursor are separate files (Codex round 2 #8 — operator quitting watch must not clobber briefing position)', async () => {
|
|
const tmpHome = makeTmpHome();
|
|
mkdirSync(tmpHome, { recursive: true });
|
|
await withEnv({ GBRAIN_HOME: tmpHome }, async () => {
|
|
const tBriefing = new Date('2026-05-10T08:00:00.000Z');
|
|
const tWatch = new Date('2026-05-10T16:00:00.000Z');
|
|
writeCursor('default', tBriefing, 'briefing');
|
|
writeCursor('default', tWatch, 'watch');
|
|
|
|
const readBriefing = readCursor('default', 'briefing');
|
|
const readWatch = readCursor('default', 'watch');
|
|
expect(readBriefing!.getTime()).toBe(tBriefing.getTime());
|
|
expect(readWatch!.getTime()).toBe(tWatch.getTime());
|
|
|
|
const briefingPath = _cursorPathForTests('default', 'briefing');
|
|
const watchPath = _cursorPathForTests('default', 'watch');
|
|
expect(briefingPath).not.toBe(watchPath);
|
|
expect(briefingPath.endsWith('default.json')).toBe(true);
|
|
expect(watchPath.endsWith('default.watch.json')).toBe(true);
|
|
});
|
|
});
|
|
|
|
test('corrupt JSON returns null + leaves the file in place for diagnosis', async () => {
|
|
const tmpHome = makeTmpHome();
|
|
mkdirSync(tmpHome, { recursive: true });
|
|
await withEnv({ GBRAIN_HOME: tmpHome }, async () => {
|
|
const path = _cursorPathForTests('default', 'briefing');
|
|
mkdirSync(dirname(path), { recursive: true });
|
|
writeFileSync(path, '{not valid json', { mode: 0o600 });
|
|
expect(readCursor('default', 'briefing')).toBeNull();
|
|
expect(existsSync(path)).toBe(true);
|
|
});
|
|
});
|
|
|
|
test('future-shifted timestamp returns null (clock-skew sanity check)', async () => {
|
|
const tmpHome = makeTmpHome();
|
|
mkdirSync(tmpHome, { recursive: true });
|
|
await withEnv({ GBRAIN_HOME: tmpHome }, async () => {
|
|
const path = _cursorPathForTests('default', 'briefing');
|
|
mkdirSync(dirname(path), { recursive: true });
|
|
const future = new Date(Date.now() + 365 * 24 * 60 * 60 * 1000).toISOString();
|
|
writeFileSync(
|
|
path,
|
|
JSON.stringify({ schema_version: 1, last_run_iso: future }),
|
|
{ mode: 0o600 },
|
|
);
|
|
expect(readCursor('default', 'briefing')).toBeNull();
|
|
});
|
|
});
|
|
|
|
test('wrong schema_version returns null', async () => {
|
|
const tmpHome = makeTmpHome();
|
|
mkdirSync(tmpHome, { recursive: true });
|
|
await withEnv({ GBRAIN_HOME: tmpHome }, async () => {
|
|
const path = _cursorPathForTests('default', 'briefing');
|
|
mkdirSync(dirname(path), { recursive: true });
|
|
writeFileSync(
|
|
path,
|
|
JSON.stringify({ schema_version: 999, last_run_iso: new Date().toISOString() }),
|
|
{ mode: 0o600 },
|
|
);
|
|
expect(readCursor('default', 'briefing')).toBeNull();
|
|
});
|
|
});
|
|
|
|
test('atomic write: per-call tmp filename uses pid+random suffix so concurrent processes do not clobber each other (Codex round 1 #7 regression)', async () => {
|
|
const tmpHome = makeTmpHome();
|
|
mkdirSync(tmpHome, { recursive: true });
|
|
await withEnv({ GBRAIN_HOME: tmpHome }, async () => {
|
|
const dir = dirname(_cursorPathForTests('default', 'briefing'));
|
|
mkdirSync(dir, { recursive: true });
|
|
writeCursor('default', new Date(), 'briefing');
|
|
const orphanedTmps = readdirSync(dir).filter(f =>
|
|
f.startsWith('default.json.tmp.'),
|
|
);
|
|
expect(orphanedTmps).toEqual([]);
|
|
});
|
|
});
|
|
|
|
test('write to non-writable parent is non-fatal (best-effort warn + return)', async () => {
|
|
const tmpHome = makeTmpHome();
|
|
mkdirSync(tmpHome, { recursive: true });
|
|
await withEnv({ GBRAIN_HOME: tmpHome }, async () => {
|
|
const path = _cursorPathForTests('blocked', 'briefing');
|
|
mkdirSync(dirname(path), { recursive: true });
|
|
writeFileSync(dirname(path) + '-as-file', 'not a dir', { mode: 0o600 });
|
|
expect(() => writeCursor('blocked', new Date(), 'briefing')).not.toThrow();
|
|
});
|
|
});
|
|
|
|
test('stable file contents: schema_version + last_run_iso in JSON', async () => {
|
|
const tmpHome = makeTmpHome();
|
|
mkdirSync(tmpHome, { recursive: true });
|
|
await withEnv({ GBRAIN_HOME: tmpHome }, async () => {
|
|
const t = new Date('2026-01-15T12:00:00.000Z');
|
|
writeCursor('default', t, 'briefing');
|
|
const path = _cursorPathForTests('default', 'briefing');
|
|
const raw = JSON.parse(readFileSync(path, 'utf8'));
|
|
expect(raw.schema_version).toBe(1);
|
|
expect(raw.last_run_iso).toBe(t.toISOString());
|
|
});
|
|
});
|
|
|
|
test('source slug used verbatim in filename (so kebab-case slugs round-trip)', async () => {
|
|
const tmpHome = makeTmpHome();
|
|
mkdirSync(tmpHome, { recursive: true });
|
|
await withEnv({ GBRAIN_HOME: tmpHome }, async () => {
|
|
writeCursor('my-team', new Date(), 'briefing');
|
|
writeCursor('my-team', new Date(), 'watch');
|
|
const briefing = _cursorPathForTests('my-team', 'briefing');
|
|
const watch = _cursorPathForTests('my-team', 'watch');
|
|
expect(basename(briefing)).toBe('my-team.json');
|
|
expect(basename(watch)).toBe('my-team.watch.json');
|
|
});
|
|
});
|
|
});
|
|
|
|
describe('recall MCP op include_pending output field (round-trip)', () => {
|
|
// Smoke test the op handler shape end-to-end via the engine method that
|
|
// backs it. The full op-handler path is covered by the cli routing test
|
|
// file; here we pin the engine contract that the op handler depends on.
|
|
|
|
test('countUnconsolidatedFacts result fits the MCP response shape (Codex #1 regression: pending field must round-trip through JSON serialization)', async () => {
|
|
await engine.insertFact(
|
|
{ fact: 'pending', kind: 'fact', entity_slug: 'e/p', source: 'unit' },
|
|
{ source_id: 'default' },
|
|
);
|
|
const n = await engine.countUnconsolidatedFacts('default');
|
|
// The op handler does: `pending_consolidation_count = n`. JSON-serializing
|
|
// and parsing it must produce the same value (no BigInt, no Date, no
|
|
// problematic shape).
|
|
const serialized = JSON.parse(JSON.stringify({ pending_consolidation_count: n }));
|
|
expect(serialized.pending_consolidation_count).toBe(1);
|
|
expect(typeof serialized.pending_consolidation_count).toBe('number');
|
|
});
|
|
});
|
|
|
|
describe('briefing skill invocation surface', () => {
|
|
// The briefing skill calls:
|
|
// gbrain recall --since-last-run --supersessions --pending --rollup --json
|
|
//
|
|
// The engine surfaces this combo exercises are:
|
|
// listSupersessions (with since cutoff)
|
|
// countUnconsolidatedFacts
|
|
//
|
|
// The CLI side (cursor state, rollup computation, thin-client routing) is
|
|
// covered by test/thin-client-routing.test.ts.
|
|
|
|
test('listSupersessions + countUnconsolidatedFacts compose cleanly for the briefing invocation', async () => {
|
|
const a = await engine.insertFact(
|
|
{ fact: 'old belief', kind: 'belief', entity_slug: 'e/x', source: 'unit' },
|
|
{ source_id: 'default' },
|
|
);
|
|
const b = await engine.insertFact(
|
|
{ fact: 'new belief', kind: 'belief', entity_slug: 'e/x', source: 'unit' },
|
|
{ source_id: 'default' },
|
|
);
|
|
await engine.expireFact(a.id, { supersededBy: b.id });
|
|
|
|
const recentlySuperseded = await engine.listSupersessions('default', {
|
|
since: new Date(Date.now() - 60_000),
|
|
limit: 50,
|
|
});
|
|
expect(recentlySuperseded.length).toBe(1);
|
|
expect(recentlySuperseded[0].id).toBe(a.id);
|
|
expect(recentlySuperseded[0].superseded_by).toBe(b.id);
|
|
|
|
// After supersession, count should reflect only the surviving b.
|
|
expect(await engine.countUnconsolidatedFacts('default')).toBe(1);
|
|
});
|
|
});
|