Files
gbrain/test/brain-registry.serial.test.ts
T
d97f159793 v0.26.4 test: parallel unit-test loop (12x speedup, failure-first logging) (#605)
* test: parallel unit-test wrapper + failure-first logging (commit 1/8)

Lay foundation for v0.26.4 parallel test loop:

- scripts/run-unit-parallel.sh: spawns N shards (default min(8, cpu_count))
  via run-unit-shard.sh, captures per-shard logs, post-shard single-writer
  failure-log aggregation at .context/test-failures.log, 10s heartbeat to
  stderr, per-shard 600s timeout (gtimeout/timeout/bg-pid fallback chain),
  loud final banner with absolute path + tail-30 of failures, summary file
  for at-a-glance status. Single writer eliminates concurrent-write hazards
  on the failure log.
- scripts/run-serial-tests.sh: discovers *.serial.test.ts files (concurrency-
  unsafe by design), runs them with --max-concurrency=1. Invoked after the
  parallel pass.
- scripts/run-unit-shard.sh: now accepts --max-concurrency=N (forwarded to
  bun test); --dry-run-list moved into argv parsing alongside; excludes
  *.serial.test.ts in addition to *.slow.test.ts.
- bunfig.toml: trim stale comment about typecheck-chained timeout.
- .gitignore: add .context/ (Conductor workspace artifacts directory; the
  failure log + summary + per-shard logs all live here).

No package.json changes yet (commit 2). No test reorganization yet
(commits 4-7).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: split package.json scripts; bun run test = parallel fast loop (commit 2/8)

Per Codex Tension #4 (verify scope), distinguish three tiers cleanly:

- `bun run test` = fast loop, file-level parallel fan-out via the new wrapper
  (scripts/run-unit-parallel.sh). No pre-checks, no typecheck, no wasm
  compile in the hot path. ~15s of pre-test gates removed.
- `bun run verify` = CI's authoritative gate set: check:jsonb +
  check:progress + check:wasm + typecheck. Matches what
  .github/workflows/test.yml runs on shard 1, no scope drift. The 4
  checks not in CI (privacy, no-legacy-getconnection, trailing-newline,
  exports-count) move to `bun run check:all` for opt-in local use.
- `bun run test:full` = verify + parallel + slow + smart e2e (runs e2e
  only if DATABASE_URL is set; else loud skip notice to stderr per Open
  Item #7). The local equivalent of "everything CI runs."

Adds `bun run test:serial` for the *.serial.test.ts subset (concurrency-
unsafe files run with --max-concurrency=1).

Bumps VERSION + package.json to 0.26.4. Both move together per the CI
version-gate contract in CLAUDE.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: fix-wave for parallel wrapper + tighten privacy gate (commit 3/5)

Wave: makes the new wrapper actually green and tightens the CI gate it
exposed.

Wrapper bug fixes (scripts/run-unit-parallel.sh):
- grep_count helper: avoids the `grep -c | echo 0` double-output bug
  where 0 matches yields a 2-line "0\n0" string and breaks arithmetic.
- bun_summary_count helper: parses Bun's actual end-of-shard summary
  format (`N pass` / `N fail` / `N skip`), not the per-test markers
  (which are `✓` / `(fail)`, never `(pass)` / `(skip)`).
- Heartbeat now reads `^\s+✓` (Bun's per-test pass marker) for live
  progress mid-run; final summary still uses the summary-line counts
  for accuracy.

Privacy gate tightening:
- Move scripts/check-privacy.sh into `bun run verify` (was previously
  only in the now-removed `bun run test` chain). Without this, after
  commit 2 the privacy check ran in nothing automatic.
- .github/workflows/test.yml now calls `bun run verify` instead of
  inlining the gate list. Single source of truth for "what's the ship
  gate." This is what verify == CI was supposed to mean per Codex T#4.
- Pre-existing `Wintermute` references in src/core/mounts-cache.ts:6
  and :324 caught by the now-running gate; replaced with `your OpenClaw`
  per CLAUDE.md privacy rule (verify gate now passes on master HEAD).
- test/privacy-script-wired.test.ts updated: regression guard now
  asserts verify includes check:privacy AND that test.yml runs
  `bun run verify`, replacing the obsolete "test script includes
  check-privacy.sh" assertion.

Quarantine 2 cross-file-contention flakes:
- test/brain-registry.test.ts: 28 tests pass alone (41ms); 1 test
  ("empty/null/undefined id routes to host") fails when run alongside
  other files in the same shard. Renamed → *.serial.test.ts so it
  runs in scripts/run-serial-tests.sh's serial pass after the parallel
  pass completes.
- test/reconcile-links.test.ts: 6 tests pass alone (1s); a beforeEach
  hook times out (~896s) under cross-file contention. Same treatment.

Both flakes are bun-process-level shared-state leaks (PGLite singletons
or top-level imports). Fixing them properly is the v0.27.0+ intra-file
parallelism project (TODO P0 — see commit 5).

Measurement after this commit:
  bun run test = 94s (was 18 min sequential)
  3639 pass, 0 fail, 0 skip across 8 parallel shards + 34 serial tests
  Failure-log + heartbeat + summary all working

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: regression tests for parallel wrapper + serial-test contracts (commit 4/5)

Three regression suites pin the v0.26.4 contracts. Without these,
future refactors of the wrapper or shard scripts could silently
regress the work in commits 1-3.

test/scripts/run-unit-shard.test.ts (4 cases — gap b):
- Asserts the unit-shard `--dry-run-list` output excludes every
  *.slow.test.ts and *.serial.test.ts file, plus the test/e2e/ subtree.
- Catches a future `find` expression that drops one of the `-not -name`
  clauses and silently un-quarantines slow/serial files into the
  parallel pass.

test/scripts/serial-files.test.ts (3 cases — gap e):
- Every checked-in *.serial.test.ts (via `git ls-files`) is listed by
  scripts/run-serial-tests.sh's `--dry-run-list`.
- The script's source contains `bun test --max-concurrency=1` (the
  serial-pass guarantee that quarantined files don't run intra-file
  concurrent and reintroduce the contention they were quarantined for).
- Disjoint set: a file is never in both the unit-shard list AND the
  serial list — pins the carve-out contract.

test/scripts/run-unit-parallel.test.ts (6 cases — gaps a + d):
- Exit-code propagation (a): wrapper exits non-zero when ANY shard
  has a failing test; exits zero when all pass. The hardest contract
  to silently break in a fan-out wrapper (`for ... &; wait` returns
  the LAST child's status, not any failure's).
- Failure-log contract (d): on failure, .context/test-failures.log
  exists, is non-empty, contains the `--- shard N:` prefix and the
  failing test's describe text. Stderr banner contains the absolute
  log path. On success, the log is cleared (no stale content).
- Summary file format: `shard N/M: pass=X fail=Y skip=Z rc=W` per
  shard, machine-parseable for future tooling.

The wrapper test runs against a 4-file tempdir (3 pass + 1 fail) so
it executes in ~500ms; spawning the wrapper against the real test
suite would take ~90s and isn't worth the cost in a regression suite.

All 13 cases pass on first run.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(v0.26.4): testing tier docs + CHANGELOG + intra-file P0 TODO (commit 5/5)

Closes the v0.26.4 ship.

CLAUDE.md Testing section rewritten:
- New tier table: test (fast loop, 85s) / verify (CI gates, 12s) /
  test:full (everything local) / test:slow / test:serial / test:e2e /
  check:all. Each row names its scope, wallclock, and when to use.
- Intentional CI vs local divergence section: CI matrix (test-shard.sh,
  hash-bucketed, includes slow) vs local fast loop (run-unit-shard.sh,
  round-robin, excludes slow + serial). Codex correctly flagged that a
  parity test would always fail by design — this is the documentation
  that explains why.
- Failure-first logging contract: .context/test-failures.log format,
  stderr banner, summary file, wedge handling.
- File taxonomy: *.test.ts / *.slow.test.ts / *.serial.test.ts /
  test/e2e/. Names the two currently-quarantined files and points at the
  intra-file P0 TODO for the proper fix.

CHANGELOG.md `## [0.26.4]` entry per voice rules:
- Two-line headline: "bun run test finishes in 85 seconds. Was 18
  minutes." + failure-log directive.
- Lead paragraph names what shipped and why.
- Numbers-that-matter table: BEFORE / AFTER / Δ for wallclock, pre-test
  gates, failure visibility, shards, pipe-survival.
- "What this means for you" closing tied to the inner-loop user.
- "To take advantage of v0.26.4" block per the v0.13+ self-repair
  template (gbrain upgrade + contributor steps).
- Itemized changes by area (new scripts, script extensions, package.json
  tier split, CI tightening, failure-first logging, quarantine, regression
  tests, bunfig).
- "What did NOT ship" section names the intra-file project + E2E
  template-DB project as P0/P1 follow-ups with concrete acceptance
  criteria.
- Process section names the codex review + scope-correction loop
  honestly: "snapped back to ship today once empirical measurement showed
  Bun's --max-concurrency does nothing on tests not marked
  test.concurrent()."
- For-contributors note on portability + single-writer + fallback paths.

TODOS.md adds two P-rated entries:
- P0: intra-file parallelism via --concurrent flag. Sweep ~58 PGLite
  sites + ~40 env mutations + 2 mock.module sites. Target: bun run test
  < 30s. ~1-2 weeks. Detailed acceptance criteria. References Codex
  findings and plan-file rationale.
- P1: E2E parallelism via Postgres template databases. CREATE DATABASE
  TEMPLATE gbrain_template per test file. ~1-2 days.

llms.txt + llms-full.txt regenerated via `bun run build:llms` to absorb
the CLAUDE.md changes (per CLAUDE.md's "After any release ship that
touches the Key Files annotations in CLAUDE.md, run bun run build:llms"
rule). The build-llms regression test was firing in shard 7 of the
parallel pass — caught the drift, regeneration cleared it. Final
measurement after fix: 94s wallclock, 3652 pass, 0 fail across 8
parallel shards + 34 serial tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 20:16:15 -07:00

282 lines
10 KiB
TypeScript

import { describe, test, expect, afterEach } from 'bun:test';
import { mkdtempSync, writeFileSync, rmSync } from 'fs';
import { join } from 'path';
import { tmpdir } from 'os';
import {
loadMounts,
validateMountId,
BrainRegistry,
DuplicateMountPathError,
UnknownBrainError,
HOST_BRAIN_ID,
type MountsFile,
type MountEntry,
} from '../src/core/brain-registry.ts';
import { GBrainError } from '../src/core/types.ts';
/** Create a temp dir + write a mounts.json into it. Returns the path. */
function tempMountsFile(contents: unknown): string {
const dir = mkdtempSync(join(tmpdir(), 'brain-registry-'));
const path = join(dir, 'mounts.json');
writeFileSync(path, typeof contents === 'string' ? contents : JSON.stringify(contents));
return path;
}
const toCleanup: string[] = [];
function track(p: string): string {
toCleanup.push(p);
return p;
}
afterEach(() => {
while (toCleanup.length > 0) {
const p = toCleanup.pop();
if (!p) continue;
try {
rmSync(p, { recursive: true, force: true });
} catch { /* best effort */ }
}
});
describe('validateMountId', () => {
test('accepts kebab-case ids', () => {
expect(validateMountId('yc-media')).toBe('yc-media');
expect(validateMountId('garrys-list')).toBe('garrys-list');
expect(validateMountId('a')).toBe('a');
expect(validateMountId('yc1')).toBe('yc1');
});
test('rejects empty / non-string', () => {
expect(() => validateMountId('')).toThrow(GBrainError);
expect(() => validateMountId(null as unknown as string)).toThrow(GBrainError);
expect(() => validateMountId(undefined as unknown as string)).toThrow(GBrainError);
expect(() => validateMountId(42 as unknown as string)).toThrow(GBrainError);
});
test('rejects reserved host id', () => {
expect(() => validateMountId('host')).toThrow(/Reserved/);
});
test('rejects invalid patterns', () => {
expect(() => validateMountId('UPPER')).toThrow();
expect(() => validateMountId('has space')).toThrow();
expect(() => validateMountId('-leading')).toThrow();
expect(() => validateMountId('trailing-')).toThrow();
expect(() => validateMountId('has_underscore')).toThrow();
expect(() => validateMountId('a'.repeat(33))).toThrow();
});
});
describe('loadMounts — file-level parsing', () => {
test('returns empty array when mounts.json is absent', () => {
const path = join(tmpdir(), `nonexistent-${Date.now()}.json`);
expect(loadMounts(path)).toEqual([]);
});
test('throws on malformed JSON', () => {
const path = track(tempMountsFile('{ not valid json'));
expect(() => loadMounts(path)).toThrow(/Malformed mounts.json/);
});
test('throws on unsupported version', () => {
const path = track(tempMountsFile({ version: 99, mounts: [] }));
expect(() => loadMounts(path)).toThrow(/Unsupported mounts.json version: 99/);
});
test('throws when mounts is not an array', () => {
const path = track(tempMountsFile({ version: 1, mounts: 'not-an-array' }));
expect(() => loadMounts(path)).toThrow(/must be an array/);
});
test('throws when top-level is not an object', () => {
const path = track(tempMountsFile([1, 2, 3]));
expect(() => loadMounts(path)).toThrow(/must be a JSON object/);
});
});
describe('loadMounts — entry validation', () => {
test('accepts a minimal pglite entry', () => {
const path = track(tempMountsFile({
version: 1,
mounts: [{ id: 'yc-media', path: '/tmp/yc-media', engine: 'pglite', database_path: '/tmp/yc-media/.pg' }],
}));
const mounts = loadMounts(path);
expect(mounts).toHaveLength(1);
expect(mounts[0].id).toBe('yc-media');
expect(mounts[0].engine).toBe('pglite');
expect(mounts[0].enabled).toBe(true); // default
});
test('accepts a postgres entry', () => {
const path = track(tempMountsFile({
version: 1,
mounts: [{
id: 'yc-politics', path: '/tmp/yc-politics', engine: 'postgres',
database_url: 'postgresql://localhost/luther',
}],
}));
const mounts = loadMounts(path);
expect(mounts[0].database_url).toBe('postgresql://localhost/luther');
});
test('resolves paths to absolute form', () => {
const path = track(tempMountsFile({
version: 1,
mounts: [{ id: 'a', path: '/tmp/relative-test', engine: 'pglite', database_path: '/tmp/a/.pg' }],
}));
const mounts = loadMounts(path);
expect(mounts[0].path.startsWith('/')).toBe(true);
});
test('enabled=false is preserved', () => {
const path = track(tempMountsFile({
version: 1,
mounts: [{
id: 'disabled-mount', path: '/tmp/disabled', engine: 'pglite',
database_path: '/tmp/disabled/.pg', enabled: false,
}],
}));
const mounts = loadMounts(path);
expect(mounts[0].enabled).toBe(false);
});
test('rejects duplicate ids', () => {
const path = track(tempMountsFile({
version: 1,
mounts: [
{ id: 'dup', path: '/tmp/a', engine: 'pglite', database_path: '/tmp/a/.pg' },
{ id: 'dup', path: '/tmp/b', engine: 'pglite', database_path: '/tmp/b/.pg' },
],
}));
expect(() => loadMounts(path)).toThrow(/duplicate id "dup"/);
});
test('rejects duplicate paths (Codex finding #9)', () => {
const path = track(tempMountsFile({
version: 1,
mounts: [
{ id: 'first', path: '/tmp/shared', engine: 'pglite', database_path: '/tmp/shared/.pg' },
{ id: 'second', path: '/tmp/shared', engine: 'pglite', database_path: '/tmp/shared/.pg2' },
],
}));
expect(() => loadMounts(path)).toThrow(DuplicateMountPathError);
});
test('rejects entry missing id', () => {
const path = track(tempMountsFile({
version: 1, mounts: [{ path: '/tmp/x', engine: 'pglite', database_path: '/tmp/x/.pg' }],
}));
expect(() => loadMounts(path)).toThrow(/Invalid mounts\[0\].id/);
});
test('rejects entry missing path', () => {
const path = track(tempMountsFile({
version: 1, mounts: [{ id: 'no-path', engine: 'pglite', database_path: '/tmp/x/.pg' }],
}));
expect(() => loadMounts(path)).toThrow(/path is required/);
});
test('rejects invalid engine kind', () => {
const path = track(tempMountsFile({
version: 1, mounts: [{ id: 'bad', path: '/tmp/b', engine: 'sqlite' }],
}));
expect(() => loadMounts(path)).toThrow(/engine must be "postgres" or "pglite"/);
});
test('rejects postgres without database_url', () => {
const path = track(tempMountsFile({
version: 1, mounts: [{ id: 'pg-no-url', path: '/tmp/p', engine: 'postgres' }],
}));
expect(() => loadMounts(path)).toThrow(/postgres mount requires database_url/);
});
test('rejects pglite without database_path or url', () => {
const path = track(tempMountsFile({
version: 1, mounts: [{ id: 'pgl-no-path', path: '/tmp/p', engine: 'pglite' }],
}));
expect(() => loadMounts(path)).toThrow(/pglite mount requires database_path/);
});
test('rejects reserved host id', () => {
const path = track(tempMountsFile({
version: 1, mounts: [{ id: 'host', path: '/tmp/h', engine: 'pglite', database_path: '/tmp/h/.pg' }],
}));
expect(() => loadMounts(path)).toThrow(/Reserved/);
});
});
describe('BrainRegistry — resolution', () => {
test('listBrainIds includes host + enabled mounts only', () => {
const mounts: MountEntry[] = [
{ id: 'a', path: '/tmp/a', engine: 'pglite', database_path: '/tmp/a/.pg', enabled: true },
{ id: 'b', path: '/tmp/b', engine: 'pglite', database_path: '/tmp/b/.pg', enabled: false },
{ id: 'c', path: '/tmp/c', engine: 'pglite', database_path: '/tmp/c/.pg', enabled: true },
];
const reg = new BrainRegistry(mounts);
expect(reg.listBrainIds()).toEqual([HOST_BRAIN_ID, 'a', 'c']);
});
test('disabled mount → UnknownBrainError', async () => {
const reg = new BrainRegistry([
{ id: 'disabled', path: '/tmp/d', engine: 'pglite', database_path: '/tmp/d/.pg', enabled: false },
]);
await expect(reg.getBrain('disabled')).rejects.toBeInstanceOf(UnknownBrainError);
});
test('unknown id → UnknownBrainError with available list', async () => {
const reg = new BrainRegistry([
{ id: 'yc-media', path: '/tmp/m', engine: 'pglite', database_path: '/tmp/m/.pg', enabled: true },
]);
try {
await reg.getBrain('nonexistent');
throw new Error('expected throw');
} catch (e) {
expect(e).toBeInstanceOf(UnknownBrainError);
if (e instanceof GBrainError) {
expect(e.cause_description).toContain('yc-media');
}
}
});
test('listMounts returns only enabled mounts', () => {
const reg = new BrainRegistry([
{ id: 'on', path: '/tmp/on', engine: 'pglite', database_path: '/tmp/on/.pg', enabled: true },
{ id: 'off', path: '/tmp/off', engine: 'pglite', database_path: '/tmp/off/.pg', enabled: false },
]);
const listed = reg.listMounts();
expect(listed).toHaveLength(1);
expect(listed[0].id).toBe('on');
});
test('disconnectAll on empty registry is idempotent', async () => {
const reg = new BrainRegistry([]);
await reg.disconnectAll();
await reg.disconnectAll(); // second call must not throw
});
});
describe('BrainRegistry — lazy init', () => {
test('getBrain does not connect until called', () => {
const reg = new BrainRegistry([
{ id: 'lazy', path: '/tmp/lazy', engine: 'pglite', database_path: '/tmp/lazy/.pg', enabled: true },
]);
// No assertion on engine state: we just prove the constructor returned
// without attempting to touch the filesystem. If init were eager, the
// constructor would throw on the missing database_path.
expect(reg.listBrainIds()).toContain('lazy');
});
test('empty/null/undefined id routes to host', async () => {
// We can't actually call getBrain('') without a host config, so we just
// verify the routing logic by observing the default-branch path. This
// test proves the fall-through to HOST_BRAIN_ID happens before any
// lookup, not that host init actually succeeds.
const reg = new BrainRegistry([]);
// Expect the host-init path to be attempted (it'll fail on missing
// ~/.gbrain/config.json in test env, but the error will come from
// initHostBrain, not UnknownBrainError — proving routing hit host).
await expect(reg.getBrain(null)).rejects.not.toBeInstanceOf(UnknownBrainError);
await expect(reg.getBrain(undefined)).rejects.not.toBeInstanceOf(UnknownBrainError);
await expect(reg.getBrain('')).rejects.not.toBeInstanceOf(UnknownBrainError);
});
});