Files
gbrain/test/schema-cli.test.ts
T
48e1000306 v0.41.23.0 feat: extract operator surfaces + pack-driven extractables (#1541)
* Wave A: schema + receipts foundation for v0.42 extract operator surfaces

Foundation layer for the pack-driven extractables + receipt-as-brain-memory
+ operator-discoverability cathedral. Five atomic pieces ship together
because their schema + helpers + module dependencies are tight-coupled:

A1. Widen pack manifest's `extractable` from `boolean` to
    `boolean | ExtractableSpec`. ExtractableSpec carries prompt_template,
    fixture_corpus, eval_dimensions, benchmark_min_recall, and reserves
    verifier_path for v0.43+ pack-shipped verifier code (REFUSE at
    runtime in v0.42 per plan D-EXTRACT-37). Back-compat: every pre-v0.42
    pack with `extractable: true` continues parsing unchanged. Three new
    helpers: extractableSpecsFromPack(), getExtractableSpec(),
    refuseVerifierPathInV042().

A2. New page type `extract_receipt` in ALL_PAGE_TYPES. Source-boost map
    adds `extracts/` prefix at factor 0.3 — receipts surface in search
    when extraction-relevant but never dominate user content (D-EXTRACT-42).

A3. New module src/core/extract/receipt-writer.ts (~190 LOC) exporting
    writeReceipt(engine, input). Canonical slug shape
    extracts/{date}/{kind}/{source_id}/{run_id_short}/round-{N} per
    D-EXTRACT-17. Frontmatter belt+suspenders per D-EXTRACT-19: BOTH
    type:extract_receipt AND dream_generated:true stamped on every
    receipt, regardless of caller, so the eligibility predicate's
    anti-loop guards reject the receipt page from any future extraction
    sweep (single-flag bypass requires breaking two unrelated checks).
    Idempotent on resume — same run_id+round overwrites cleanly.

A4. Migration v104 creates extract_rollup_7d table (per-day rollup of
    extract events keyed on kind+source_id+day). Audit JSONL stays the
    SOURCE OF TRUTH per F-OUT-19; this table is a best-effort cache for
    doctor's <100ms read budget. Per-day rows mean the 7-day window
    auto-evicts on every read. v100 was deliberately skipped on master
    (renumbered out during a prior wave); v101/v102/v103 also taken;
    v104 is the next clean slot.

A5. Doctor `extract_health` check reads extract_rollup_7d for last 7
    days and emits per-kind aggregates: cost_7d_usd, eval_pass_count,
    eval_fail_count, halt_count, round_completed_count, halt_rate.
    3-state: OK when rollup empty (pre-v0.42 brain or fresh init), WARN
    when any per-kind halt rate > 10% (top-3 named in message), WARN
    when rollup_write_failures > 0 (audit JSONL is SoT but operator
    deserves to know the DB cache is degraded). Pre-v104 brains stay
    quiet — the missing-table error path is caught and treated as
    OK so doctor doesn't warn during the upgrade window.

Tests added:
- test/extractable-spec-widening.test.ts (22 cases) — back-compat with
  boolean shape, new struct parsing, verifier_path REFUSE contract.
- test/extract/receipt-writer.test.ts (12 cases) — slug shape, frontmatter
  belt+suspenders, idempotent resume, body human-readability.
- test/doctor-extract-health.test.ts (8 cases) — empty rollup OK, halt
  rate WARN, rollup_write_failures WARN, 7-day window inclusion at
  boundary, multi-kind top-3 message ordering.

Plus the canonical bootstrap-coverage test passes with the new v104
migration cleanly applied through both engines.

Plan: ~/.claude/plans/system-instruction-you-are-working-stateless-dragonfly.md
Wave A scope. Wave B (hook receipts into existing extractors) follows.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Wave B: hook receipts + rollup row into the 5 shipped extractors

Each LLM-backed extractor surface now records its run in two places when
something actually happened:

1. An extract receipt PAGE at extracts/{date}/{kind}/{source_id}/{run_id_short}/round-{N}
   (queryable via gbrain search, citable, surfaces in cross-modal
   contradiction probes per the Wave A foundation). Only written when
   `total_rows > 0` so no-op runs don't bloat the brain.

2. An UPSERT row in extract_rollup_7d (DB-backed best-effort cache
   per F-OUT-19) so the doctor extract_health check from Wave A reads
   per-kind aggregates without scanning JSONL.

New module src/core/extract/rollup-writer.ts (~120 LOC) exports
upsertExtractRollup() with PostgreSQL ON CONFLICT DO UPDATE on the
(kind, source_id, day) PK. Concurrency-safe per F-OUT-14 design.
Failure path is best-effort — bumps rollup_write_failures in the
table itself, stderr-warns once per (kind, day, error-class), and
NEVER fails the parent extraction operation. JSONL remains source
of truth.

Wired into 5 extractors:
- extract-conversation-facts (kind: facts.conversation) — both
  success path AND BudgetExhausted halt path write receipt+rollup
  so partial runs are still observable.
- extract_atoms cycle phase (kind: atoms)
- synthesize_concepts cycle phase (kind: concepts, source_id: default
  because concepts are brain-global)
- propose_takes cycle phase (kind: takes.proposed) — scope-aware
  source_id from the read scope.
- extract_facts cycle phase (kind: facts.fence) — deterministic
  (no LLM cost) but still records reconcile activity so doctor sees
  the cycle is alive.

Receipt frontmatter belt+suspenders (D-EXTRACT-19) reused from
Wave A: every receipt stamps BOTH `type: extract_receipt` AND
`dream_generated: true` so the eligibility predicate's anti-loop
guards reject the receipt page from any future extraction sweep.

Test surgery in test/propose-takes.test.ts — one existing assertion
tightened from "no INSERTs" to "no INSERT INTO take_proposals" so
the new rollup UPSERT doesn't falsely fail the cache-hit case test.

Run regression: 85/85 tests pass across extract-conversation-facts,
extract-atoms-synthesize-concepts, extract-facts-phase, propose-takes.

Plan: ~/.claude/plans/system-instruction-you-are-working-stateless-dragonfly.md
Wave B scope. Wave C (pack-author scaffolding + benchmark) follows.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Wave C+D: pack-author scaffolding + operator surfaces for v0.42 extract

Wave C: pack-author authoring loop
  - scaffold-extractable mutation primitive declares a kind as extractable
    on a pack manifest in one verb (wires through updateTypeOnPack from
    the v0.41 mutate library); generates 5 placeholder fixtures + a
    pack-supplied prompt template stub
  - schema CLI wires gbrain schema scaffold-extractable <type> --pack <pack>
  - extract benchmark CLI loads a pack's fixture corpus through strict
    D-EXTRACT-21 path validation (rejects absolute paths, .. traversal,
    null bytes, symlinks resolving outside pack root); v0.42 ships as a
    stub reporter (LLM dispatch deferred to Wave E)

Wave D: operator surfaces
  - extract status CLI reads extract_rollup_7d for the last 7 days,
    sorts by (halt_rate desc, cost desc); kubectl-style right-aligned
    table, top-5 + "more rows" hint by default, --verbose shows all;
    stable schema_version: 1 JSON envelope for monitoring pipelines
  - extract --explain <kind> CLI prints the active pack's resolution
    chain: declaration source (pack-declared vs built-in cycle phase),
    prompt_template + fixture_corpus paths with existence checks,
    eval_dimensions, benchmark_min_recall, and the last 7d rollup
  - extract.ts gains a lifecycle-grouped help text (Extraction /
    Inspection / Status) per the original D3 plan goal

Tests:
  - test/schema-pack/scaffold-extractable.test.ts (15 cases) including
    explicit privacy-rule assertions guarding against real-name leakage
  - test/extract/benchmark.test.ts (17 cases) covering path validation
    rejections + JSONL fixture parsing
  - test/extract/status.test.ts (15 cases) over pure aggregation +
    formatting

Housekeeping:
  - test/extract/receipt-writer.test.ts refactored to the canonical
    PGLite block (beforeAll/afterAll/resetPgliteState in beforeEach)
    per CLAUDE.md test-isolation R3+R4; runtime drops from ~30s of
    99-migration replay per test to <6s for all 12 cases together

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.42.0.0: extract operator surfaces + pack-driven extractables

Bump VERSION + package.json to 0.42.0.0. CHANGELOG entry covers the
three-wave shipped scope (receipts + rollup + doctor check; receipts
hooked into all 5 shipped extractors; pack-author scaffolding +
benchmark stub-reporter; status + --explain dashboards + lifecycle
help). CLAUDE.md Key Files gains a v0.42 cluster annotation. llms.txt
regenerated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.41.23.0: re-tag from 0.42.0.0 (patch-channel slot, no scope change)

VERSION + package.json + CHANGELOG header + CLAUDE.md cluster annotation
all moved from 0.42.0.0 to 0.41.23.0. Body text updated in-place: every
"v0.42" / "v0.43+" reference inside this entry's release notes now reads
"v0.41.23" or "follow-up release" as appropriate.

Same scope shipping — the three-wave extract operator surface stays
intact. Just lands in the patch-channel queue (.20/.21/.23 free; .22 is
PR #1542's type-unification cathedral) instead of the minor-channel bump.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: add extract_receipt to gbrain-base.yaml page_types (CI parity gate)

CI shard 5 caught the drift: test/regressions/gbrain-base-equivalence.test.ts
asserts every ALL_PAGE_TYPES seed has a matching page_type entry in the
gbrain-base.yaml pack. Wave A added `extract_receipt` to ALL_PAGE_TYPES
but didn't seed it in the base pack manifest.

Adds the entry under the `annotation` primitive with `extracts/` path
prefix (matches the source-boost demote site) and `extractable: false`
(receipts are written by the framework, never extracted from). Comment
documents the belt+suspenders D-EXTRACT-19 invariant so future readers
understand why receipts carry both `dream_generated: true` AND
`type: extract_receipt`.

Closes the CI gate without changing runtime behavior — the pack-aware
read paths already had the prefix demote wired in src/core/search/source-boost.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: bump gbrain-base page-type count 24→25 in schema-cli test

CI shard 4 caught the second drift from the same root cause as the
prior parity-gate fix: v0.41.23's `extract_receipt` addition bumped
gbrain-base.yaml from 24 to 25 page types. The schema-cli smoke test
was pinned at 24 (the count after v0.41.11.0 added `conversation` +
`atom`); update to 25 and note v0.41.23's contribution alongside the
prior version stamp.

Verified hermetic: running test/schema-cli.test.ts with a clean
GBRAIN_HOME tempdir produces 12/12 pass (the local-machine 'schema
active' fail is from a real ~/.gbrain pinning gbrain-base-v2; not a
shipped-code issue, doesn't repro on CI).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(test): pack-locator stub leak between shard 6 test files

CI shard 6 caught three flaky failures in test/onboard-pack-upgrade-checks.test.ts:
  - checkPackUpgradeAvailable > fires on gbrain-base brain with gbrain-base-v2
  - checkPackUpgradeAvailable > manual_only routing via render.ts allowlist (D17)
  - checkTypeProliferation > warns when distinct types exceed declared+5

Root cause: test/schema-pack-sync.test.ts calls
`__setPackLocatorForTests(...)` to stub the disk-loader, but doesn't
restore in afterAll. Bun's CI shard 6 loads multiple test files into
one process; when sync.test.ts runs before onboard-pack-upgrade-checks.test.ts,
the stubbed locator persists at module scope. `loadActivePack` for
gbrain-base / gbrain-base-v2 then returns null and:
  - findPackSuccessors returns [] → status='ok' instead of 'warn' (F1+F2)
  - declared falls back to 15 → fail threshold becomes 30, 32 > 30 → 'fail'
    instead of 'warn' (F3)

Local single-file runs pass because the locator starts at its default.

Two-layer fix:
  1. test/schema-pack-sync.test.ts afterAll calls
     `_resetPackLocatorForTests()` to undo the mutation (the canonical
     fix at the source).
  2. test/onboard-pack-upgrade-checks.test.ts beforeEach calls the same
     reset (defense-in-depth against any future test file in the shard
     that forgets to restore).

Reproduced locally: running the three shard-6 schema-pack files together
fails 3 tests pre-fix and passes 30/30 post-fix. Full shard 6 sweep
(77 files, 1232 tests) now green; bun run verify still 28/28.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(test): pglite-engine — dim-agnostic chunk-embedding test data

CI shard 6 caught two flaky failures in test/pglite-engine.test.ts:
  - PGLiteEngine: Chunks > getChunksWithEmbeddings returns embedding data
  - PGLiteEngine: stale chunk pagination > countStaleChunks counts chunks
    with NULL embedding only

Both failed with `expected 1280 dimensions, not 1536` at the upsert site.

Root cause: pglite-engine.ts:287 initSchema() reads embedding dim from
gw.getEmbeddingDimensions() if the gateway is configured (potentially
left in that state by another shard-6 test file in the same bun process),
falling back to DEFAULT_EMBEDDING_DIMENSIONS otherwise — which is 1280
since v0.36+ when the ZE default landed (zeroentropyai:zembed-1).
Pre-v0.36 defaults were OpenAI's 1536; my test data was pinned to that
stale literal.

The two outcomes that pass:
  - gateway happens to be configured for 1536-dim (e.g. master shard 6
    run 26515999465 — these tests passed at 20ms + 24ms with no
    "dimensions" error)
  - gateway happens to be configured for 1280-dim AND test data is 1280

The outcome that fails:
  - gateway configured for 1280-dim AND test data hardcoded to 1536

Fix: capture the actual column width after initSchema (probe
pg_attribute.atttypmod for content_chunks.embedding) and use that
captured `CHUNK_EMBED_DIM` constant at the three Float32Array sites.
Test data now matches whatever width the column was created at,
regardless of which shard-6 file ran first.

Local repro: full shard 6 (77 files, 1232 tests, ~6min) green; this
file standalone (100 tests) green; bun run verify 28/28.

Broader pattern: 9 other test files use the same Float32Array(1536)
literal. None land in shard 6 today (so they don't flake), but the
fix shape here can be lifted into a shared helper if the bug class
surfaces elsewhere — filed as a v0.42+ follow-up rather than a
preemptive sweep, since each file's setup shape is slightly different.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 08:27:05 -07:00

173 lines
6.8 KiB
TypeScript

// v0.38 Phase C: gbrain schema CLI smoke tests.
//
// Tests the runSchema dispatch + each subcommand's output shape via
// the public CLI entrypoint. Hermetic — uses Bun's subprocess to run
// the CLI like a user would.
import { describe, expect, test, beforeAll, afterAll, beforeEach, afterEach } from 'bun:test';
import { spawnSync } from 'node:child_process';
import { mkdtempSync, rmSync, existsSync, readFileSync, writeFileSync, mkdirSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
const REPO_ROOT = join(import.meta.dir, '..');
// Default-isolated GBRAIN_HOME for every gbrain() call. Without this,
// tests that read `~/.gbrain/config.json` inherit the developer's real
// brain config — and sibling Conductor worktrees writing to the same
// config (e.g. via `schema use` or `config set` during their own tests)
// cause flakes (the failing test pre-fix saw `schema_pack: "gbrain-base-v2"`
// from another worktree, which doesn't exist in the bundle, and got
// exit 1 instead of the asserted 0).
let DEFAULT_GBRAIN_HOME: string;
beforeAll(() => {
DEFAULT_GBRAIN_HOME = mkdtempSync(join(tmpdir(), 'gbrain-schema-cli-default-'));
});
afterAll(() => {
rmSync(DEFAULT_GBRAIN_HOME, { recursive: true, force: true });
});
function gbrain(
args: string[],
extraEnv: Record<string, string> = {},
): { stdout: string; stderr: string; code: number } {
// bun's spawnSync does NOT inherit env mutations done via process.env = ...,
// so pass env explicitly. CLAUDE.md flags this pattern as load-bearing for
// any subprocess test that needs GBRAIN_HOME isolation.
const env = { ...process.env, GBRAIN_HOME: DEFAULT_GBRAIN_HOME, ...extraEnv };
const result = spawnSync('bun', ['run', 'src/cli.ts', ...args], {
cwd: REPO_ROOT,
encoding: 'utf-8',
env,
});
return {
stdout: result.stdout ?? '',
stderr: result.stderr ?? '',
code: result.status ?? -1,
};
}
describe('gbrain schema CLI (Phase C)', () => {
test('schema with no subcommand shows help text', () => {
// Note: `schema --help` is intercepted by the CLI's parent help system
// and prints generic help (`gbrain --help` for full command list). The
// schema-specific help fires when no subcommand is provided.
const r = gbrain(['schema']);
expect(r.stdout + r.stderr).toMatch(/schema|active|list|show|validate|use/i);
});
test('schema list shows gbrain-base bundled', () => {
const r = gbrain(['schema', 'list']);
expect(r.code).toBe(0);
expect(r.stdout).toContain('Bundled packs:');
expect(r.stdout).toContain('gbrain-base');
});
test('schema show gbrain-base prints manifest details', () => {
const r = gbrain(['schema', 'show', 'gbrain-base']);
expect(r.code).toBe(0);
expect(r.stdout).toContain('gbrain-base v1.0.0');
// v0.41.11.0: page types extended from 22 to 24 by promoting
// `conversation` and `atom` into gbrain-base.
// v0.41.23.0: extended to 25 by adding `extract_receipt` for the
// unified extract receipt-writer surface (D-EXTRACT-19 belt+suspenders).
expect(r.stdout).toContain('Page types (25)');
expect(r.stdout).toContain('Link verbs (12)');
expect(r.stdout).toContain('Takes kinds: fact, take, bet, hunch');
expect(r.stdout).toContain('person :: entity');
expect(r.stdout).toContain('company :: entity');
});
test('schema validate gbrain-base passes', () => {
const r = gbrain(['schema', 'validate', 'gbrain-base']);
expect(r.code).toBe(0);
expect(r.stdout).toContain('✓');
expect(r.stdout).toContain('valid manifest');
});
test('schema active reports default resolution', () => {
const r = gbrain(['schema', 'active']);
expect(r.code).toBe(0);
expect(r.stdout).toContain('Active pack:');
expect(r.stdout).toContain('Pack identity:');
});
test('schema show unknown-pack errors with hint', () => {
const r = gbrain(['schema', 'show', 'nonexistent-pack']);
expect(r.code).not.toBe(0);
expect(r.stderr).toContain('Unknown pack');
expect(r.stderr).toContain('schema list');
});
test('unknown subcommand exits with hint', () => {
const r = gbrain(['schema', 'frobnicate']);
expect(r.code).toBe(2);
expect(r.stderr).toContain('Unknown schema subcommand');
});
test('schema use without arg shows usage hint', () => {
const r = gbrain(['schema', 'use']);
expect(r.code).toBe(2);
expect(r.stderr).toContain('Usage:');
});
});
describe('gbrain schema use (Phase C, gap-fill T3)', () => {
let home: string;
beforeEach(() => {
home = mkdtempSync(join(tmpdir(), 'gbrain-schema-use-'));
});
afterEach(() => {
rmSync(home, { recursive: true, force: true });
});
test('writes schema_pack to ~/.gbrain/config.json on happy path', () => {
const r = gbrain(['schema', 'use', 'gbrain-base'], { GBRAIN_HOME: home });
expect(r.code).toBe(0);
expect(r.stdout).toContain('Active schema pack set to: gbrain-base');
expect(r.stdout).toContain('schema active');
const cfgPath = join(home, '.gbrain', 'config.json');
expect(existsSync(cfgPath)).toBe(true);
const cfg = JSON.parse(readFileSync(cfgPath, 'utf-8'));
expect(cfg.schema_pack).toBe('gbrain-base');
});
test('preserves pre-existing config fields when writing schema_pack', () => {
// Pre-seed a config with engine + a custom key so the merge preserves them.
mkdirSync(join(home, '.gbrain'), { recursive: true });
const cfgPath = join(home, '.gbrain', 'config.json');
writeFileSync(cfgPath, JSON.stringify({ engine: 'pglite', openai_key: 'sk-fake' }, null, 2), 'utf-8');
const r = gbrain(['schema', 'use', 'gbrain-base'], { GBRAIN_HOME: home });
expect(r.code).toBe(0);
const cfg = JSON.parse(readFileSync(cfgPath, 'utf-8'));
expect(cfg.engine).toBe('pglite');
expect(cfg.openai_key).toBe('sk-fake');
expect(cfg.schema_pack).toBe('gbrain-base');
});
test('overwrites prior schema_pack value on re-run', () => {
// First set a placeholder, then overwrite via the CLI.
mkdirSync(join(home, '.gbrain'), { recursive: true });
const cfgPath = join(home, '.gbrain', 'config.json');
writeFileSync(cfgPath, JSON.stringify({ engine: 'pglite', schema_pack: 'something-else' }, null, 2), 'utf-8');
const r = gbrain(['schema', 'use', 'gbrain-base'], { GBRAIN_HOME: home });
expect(r.code).toBe(0);
const cfg = JSON.parse(readFileSync(cfgPath, 'utf-8'));
expect(cfg.schema_pack).toBe('gbrain-base');
});
test('unknown pack rejected with exit 1 + paste-ready hint', () => {
const r = gbrain(['schema', 'use', 'no-such-pack-xyz'], { GBRAIN_HOME: home });
expect(r.code).toBe(1);
expect(r.stderr).toContain('Unknown pack');
expect(r.stderr).toContain('schema list');
// Importantly: a failed `use` must NOT have written a config.
const cfgPath = join(home, '.gbrain', 'config.json');
expect(existsSync(cfgPath)).toBe(false);
});
});