Files
gbrain/test/progress.test.ts
T
90e22c22e2 v0.23.1 feat: local CI gate + 4-tier wall-time optimization (~13x faster) (#528)
* feat: diff-aware E2E test selector

Adds scripts/select-e2e.ts: reads git diff vs origin/master, classifies
the change set (EMPTY/DOC_ONLY/SRC), and emits the relevant E2E test files
on stdout. Fail-closed by design: any unmapped src/ change runs all E2E.

- scripts/e2e-test-map.ts: hand-tuned path-glob -> test files map
- scripts/select-e2e.ts: pure-function selector with three explicit cases
- scripts/run-e2e.sh: accepts optional file list from argv + --dry-run-list
- test/select-e2e.test.ts: 24 cases including 3 codex regression guards
  (skills/, untracked files, unmapped src/)

* feat: local CI gate via docker compose

Adds bun run ci:local — runs every check GH Actions runs (gitleaks +
unit + 29 E2E files) inside a Docker container that bind-mounts the
repo. Pure bind-mount + named volumes (gbrain-ci-node-modules,
gbrain-ci-bun-cache, gbrain-ci-pg-data) for fast warm restarts.

- docker-compose.ci.yml: pgvector/pgvector:pg16 + oven/bun:1
- scripts/ci-local.sh: orchestrator with --diff, --no-pull, --clean
- gitleaks runs on host (scoped to working dir + branch commits)
- DATABASE_URL unset for unit phase (matches GH Actions split)
- git installed in container at startup (oven/bun:1 omits it)
- Postgres host port via GBRAIN_CI_PG_PORT env (default 5434)

Stronger than PR CI: runs all 29 E2E files vs CI's 2-file Tier 1.

* chore: bump version and changelog (v0.23.1)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: document local CI gate for v0.23.1

CLAUDE.md gains key-files entries for docker-compose.ci.yml,
scripts/ci-local.sh, scripts/select-e2e.ts + e2e-test-map.ts, and the
scripts/run-e2e.sh argv tweak. Pre-ship requirements section now lists
the Docker-based local gate as Path A alongside the manual lifecycle.

CONTRIBUTING.md tests section adds the bun run ci:local / ci:local:diff /
ci:select-e2e block with prerequisites (Docker engine + gitleaks) and the
GBRAIN_CI_PG_PORT override.

AGENTS.md "Before shipping" promotes ci:local as the easiest path and
keeps the manual lifecycle as a fallback.

README.md Contributing section points to ci:local for the full gate.

CHANGELOG.md untouched — v0.23.1 entry already finalized.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: SHARD=N/M env support in scripts/run-e2e.sh

Filters the E2E file list to every M-th file starting at index N (1-indexed).
Sequential execution within a shard preserves the TRUNCATE CASCADE no-race
property documented at the top of the file. Empty-shard handling under
`set -u` uses ${arr[@]:-} fallback.

Standalone change; not yet wired up in ci-local.sh.

* feat: 4-way parallel E2E shards in ci:local

Replaces the single postgres service with 4 (postgres-1..4) on host ports
5434-5437. scripts/ci-local.sh fans 4 workers via xargs -P4 inside the
runner container; each pinned to its own DATABASE_URL via SHARD=N/4.

Wall-time on a 16-core host: ~6 min sequential -> ~1.5-2 min sharded.
Total full-gate wall-time goes from ~25 min to ~3-5 min warm.

Also handles git-worktree (Conductor) layouts: when /app/.git is a file
instead of a directory, parse the gitdir + commondir and bind-mount the
shared host gitdir at its absolute path. Without this, in-container
`git ls-files` (used by scripts/check-trailing-newline.sh and friends)
exits 128 with "not a git repository". Also runs
`git config --global --add safe.directory '*'` inside the container so
the root-uid container can read host-uid gitdir without "dubious
ownership" rejection.

CHANGELOG entry updated to cover the speedup.

- docker-compose.ci.yml: 4 pgvector services + per-shard named volumes
- scripts/ci-local.sh: parallel xargs orchestration + worktree mount fix
- CHANGELOG.md v0.23.1: 4-way sharded wall-time, 36 E2E files, --no-shard flag

* chore: regenerate llms-full.txt for v0.23.1 doc updates

Required by test/build-llms.test.ts case 4 — committed llms-full.txt
must match `bun run build:llms` output. The CHANGELOG + CLAUDE.md
updates in this branch shifted bytes; regen catches up.

* feat: scripts/run-unit-shard.sh + slow-test convention

Tier 1 + Tier 4 plumbing:
- scripts/run-unit-shard.sh: SHARD=N/M filter for unit files (excludes
  test/e2e/*). Excludes *.slow.test.ts (Tier 4 convention) so the fast
  shard fan-out skips known-slow files; CI's `bun run test` still includes
  them via default discovery.
- scripts/run-slow-tests.sh: companion that runs ONLY *.slow.test.ts.
  Wired as `bun run test:slow`.
- scripts/profile-tests.sh: portable awk parser that extracts the top-N
  slowest tests from any captured `bun test` output. Wired as
  `bun run test:profile`. Use it to pick demotion candidates.

* feat: PGLite snapshot fixture for ~4.5x faster cold init (Tier 3)

scripts/build-pglite-snapshot.ts boots a fresh PGLite, runs the full
initSchema() (forward bootstrap + 30 migrations), and dumps the post-init
state to test/fixtures/pglite-snapshot.tar plus a SHA-256 schema hash
sidecar (.version). Both gitignored — built on demand via
`bun run build:pglite-snapshot`.

PGLiteEngine.connect() reads GBRAIN_PGLITE_SNAPSHOT env: validates the
sidecar hash against the in-process MIGRATIONS hash, loads via PGLite's
loadDataDir blob, sets _snapshotLoaded so initSchema() short-circuits.
Measured per-file cold init drops from 828ms → 181ms.

Bootstrap-correctness tests (bootstrap.test.ts,
schema-bootstrap-coverage.test.ts) explicitly delete the env at file
top so they keep exercising the cold path they verify.

* feat: --classify-only + heartbeat tolerance fix (Tiers 2 + flake fix)

- scripts/select-e2e.ts: --classify-only flag emits EMPTY|DOC_ONLY|SRC.
  Used by ci-local.sh's --diff fast-path to skip the heavy gate when
  only docs changed.
- test/progress.test.ts: startHeartbeat tolerance widened to 1-20 over
  200ms (was 2-6 over 85ms). Under 4-way parallel shard load on a
  contended host, setTimeout's effective quantum balloons and the tight
  bound flakes. The test still verifies "fires multiple times, stops
  cleanly" — exact count was never load-bearing.

* feat: 4-way unit + E2E sharding in ci-local.sh + CHANGELOG (Tiers 1-4)

ci-local.sh ties the four tiers together:
- Tier 2: pre-flight diff classification on host. DOC_ONLY exits in ~5s
  (gitleaks only, no postgres, no container).
- Tier 1: guards + typecheck run ONCE before fan-out. xargs -P4 then
  spawns 4 shards inside the runner container, each running unit phase
  (env -u DATABASE_URL bash run-unit-shard.sh) followed by E2E phase
  (DATABASE_URL=postgres-N bash run-e2e.sh) — both sharded N/4. Per-shard
  logs in /tmp/shard-logs/shard-N.log; printed in shard order at the end.
- Tier 3: snapshot fixture built once at runner startup if missing,
  GBRAIN_PGLITE_SNAPSHOT exported so all shards inherit.
- Tier 4: run-unit-shard.sh excludes *.slow.test.ts; run-slow-tests.sh
  + test:slow npm script handle the demoted set.
- --no-shard preserves the legacy single-process flow for debug.

package.json: build:pglite-snapshot, test:slow, test:profile scripts.

Measured wall-time on 16-core host: 100s warm (down from ~22 min cold
single-process). 4 shards × ~640-1024 unit tests each, plus 9 E2E
files each. PGLite snapshot saves 4.5× per cold init (828ms → 181ms).

CHANGELOG.md updated with measured numbers + four-tier breakdown.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 20:47:32 -07:00

264 lines
10 KiB
TypeScript

import { describe, test, expect } from 'bun:test';
import { PassThrough } from 'node:stream';
import { createProgress, startHeartbeat, __liveReporterCountForTest, __signalHandlerInstalledForTest } from '../src/core/progress.ts';
/** Collect everything a reporter writes into a string. */
function sink(isTTY = false): { stream: PassThrough & { isTTY?: boolean }; read: () => string } {
const s = new PassThrough() as PassThrough & { isTTY?: boolean };
s.isTTY = isTTY;
const chunks: string[] = [];
s.on('data', (c) => chunks.push(c.toString('utf8')));
return { stream: s, read: () => chunks.join('') };
}
function parseJsonl(raw: string): Record<string, unknown>[] {
return raw
.split('\n')
.filter((l) => l.length > 0)
.map((l) => JSON.parse(l));
}
describe('progress reporter', () => {
test('auto mode: non-TTY → human-plain (NOT JSON)', () => {
const { stream, read } = sink(false);
const p = createProgress({ mode: 'auto', stream, minIntervalMs: 0, minItems: 1 });
p.start('scan', 3);
p.tick();
p.tick();
p.tick();
p.finish();
const out = read();
// plain lines, no JSON
expect(out).not.toContain('"event"');
expect(out).toContain('[scan]');
expect(out).toContain('1/3');
expect(out).toContain('3/3');
});
test('auto mode: TTY → human-\\r (carriage return, no newline between ticks)', () => {
const { stream, read } = sink(true);
const p = createProgress({ mode: 'auto', stream, minIntervalMs: 0, minItems: 1 });
p.start('scan', 2);
p.tick();
p.tick();
p.finish();
const out = read();
// TTY path uses \r + clear-line escape; final newline on finish.
expect(out).toContain('\r');
expect(out).toContain('[scan]');
});
test('json mode emits one JSON object per line with schema', () => {
const { stream, read } = sink(false);
const p = createProgress({ mode: 'json', stream, minIntervalMs: 0, minItems: 1 });
p.start('doctor.jsonb_integrity', 4);
p.tick(1, 'pages.frontmatter');
p.tick(1, 'raw_data.data');
p.finish();
const events = parseJsonl(read());
expect(events.length).toBeGreaterThanOrEqual(3);
expect(events[0]).toMatchObject({ event: 'start', phase: 'doctor.jsonb_integrity', total: 4 });
expect(events[0].ts).toMatch(/^\d{4}-\d{2}-\d{2}T/);
expect(events[1]).toMatchObject({ event: 'tick', phase: 'doctor.jsonb_integrity', done: 1, total: 4 });
expect(events[1].pct).toBe(25);
expect(typeof events[1].elapsed_ms).toBe('number');
expect(events[events.length - 1]).toMatchObject({ event: 'finish', phase: 'doctor.jsonb_integrity' });
});
test('quiet mode emits nothing', () => {
const { stream, read } = sink(false);
const p = createProgress({ mode: 'quiet', stream });
p.start('scan', 10);
p.tick();
p.heartbeat('hello');
p.finish();
expect(read()).toBe('');
});
test('tick() time-gated: calls inside minIntervalMs collapse to one emit', () => {
const { stream, read } = sink(false);
const p = createProgress({ mode: 'json', stream, minIntervalMs: 5000, minItems: 999999 });
p.start('scan', 100);
// Rapid ticks — should not emit intermediate 'tick' events (only the final one if eq total).
for (let i = 0; i < 10; i++) p.tick();
const events = parseJsonl(read());
const ticks = events.filter((e) => e.event === 'tick');
// 10 ticks, total=100, final-tick-on-complete heuristic doesn't apply (done < total).
// Time-gated + item-gated should suppress all.
expect(ticks.length).toBe(0);
p.finish();
});
test('tick() item-gated: minItems threshold emits after N items', () => {
const { stream, read } = sink(false);
const p = createProgress({ mode: 'json', stream, minIntervalMs: 999999, minItems: 50 });
p.start('scan', 1000);
for (let i = 0; i < 100; i++) p.tick();
p.finish();
const events = parseJsonl(read());
const ticks = events.filter((e) => e.event === 'tick');
// 100 ticks with minItems=50 ⇒ expect ~2 emits
expect(ticks.length).toBeGreaterThanOrEqual(1);
expect(ticks.length).toBeLessThanOrEqual(3);
});
test('final tick emits regardless of gating when done === total', () => {
const { stream, read } = sink(false);
const p = createProgress({ mode: 'json', stream, minIntervalMs: 999999, minItems: 999999 });
p.start('scan', 3);
p.tick();
p.tick();
p.tick(); // this one hits done===total, must emit
p.finish();
const events = parseJsonl(read());
const ticks = events.filter((e) => e.event === 'tick');
expect(ticks.length).toBe(1);
expect(ticks[0]).toMatchObject({ done: 3, total: 3 });
});
test('start(phase) with no total → ticks omit pct/eta_ms', () => {
const { stream, read } = sink(false);
const p = createProgress({ mode: 'json', stream, minIntervalMs: 0, minItems: 1 });
p.start('unknown_size_scan'); // no total
p.tick();
p.finish();
const events = parseJsonl(read());
const tick = events.find((e) => e.event === 'tick')!;
expect(tick).toBeDefined();
expect(tick.total).toBeUndefined();
expect(tick.pct).toBeUndefined();
expect(tick.eta_ms).toBeUndefined();
expect(tick.done).toBe(1);
});
test('heartbeat() emits without bumping done', () => {
const { stream, read } = sink(false);
const p = createProgress({ mode: 'json', stream, minIntervalMs: 0, minItems: 1 });
p.start('slow_query');
p.heartbeat('still scanning…');
p.heartbeat('still scanning…');
p.finish();
const events = parseJsonl(read());
const hb = events.filter((e) => e.event === 'heartbeat');
expect(hb.length).toBe(2);
expect(hb[0]).toMatchObject({ phase: 'slow_query', note: 'still scanning…' });
// No 'done' field on heartbeat.
expect(hb[0].done).toBeUndefined();
});
test('child() composes phase path with dots', () => {
const { stream, read } = sink(false);
const p = createProgress({ mode: 'json', stream, minIntervalMs: 0, minItems: 1 });
p.start('sync');
const c = p.child('import');
c.start('file1', 1);
c.tick();
c.finish();
p.finish();
const events = parseJsonl(read());
const startEvents = events.filter((e) => e.event === 'start');
const phases = startEvents.map((e) => e.phase);
expect(phases).toContain('sync');
expect(phases).toContain('sync.import.file1');
});
test('child.finish() does not close parent', () => {
const { stream, read } = sink(false);
const p = createProgress({ mode: 'json', stream, minIntervalMs: 0, minItems: 1 });
p.start('sync');
const c = p.child('import');
c.start('batch1', 1);
c.tick();
c.finish();
// Parent still alive — another tick should work.
// (parent.tick requires a started phase; start was called on 'sync'.)
p.tick(1, 'after-child');
p.finish();
const events = parseJsonl(read());
const finishes = events.filter((e) => e.event === 'finish');
const finishPhases = finishes.map((e) => e.phase);
expect(finishPhases).toContain('sync.import.batch1');
expect(finishPhases).toContain('sync');
});
test('EPIPE sync throw is swallowed; subsequent writes are no-ops', () => {
const brokenStream = {
isTTY: false,
write: () => {
throw Object.assign(new Error('write EPIPE'), { code: 'EPIPE' });
},
on: () => {},
} as unknown as NodeJS.WritableStream;
const p = createProgress({ mode: 'json', stream: brokenStream, minIntervalMs: 0, minItems: 1 });
// Must not throw.
expect(() => {
p.start('scan', 3);
p.tick();
p.tick();
p.finish();
}).not.toThrow();
});
test("EPIPE stream 'error' event marks stream broken", () => {
const { stream, read } = sink(false);
const p = createProgress({ mode: 'json', stream, minIntervalMs: 0, minItems: 1 });
p.start('scan', 2);
p.tick();
// Simulate async EPIPE via error event.
stream.emit('error', Object.assign(new Error('EPIPE'), { code: 'EPIPE' }));
// Subsequent calls must not throw.
expect(() => {
p.tick();
p.finish();
}).not.toThrow();
// We did get at least the pre-error emissions.
expect(read()).toContain('"event":"start"');
});
test('only one process-level signal handler installed across many reporters', () => {
// Baseline: one handler already installed by prior tests in this file.
const installedBefore = __signalHandlerInstalledForTest();
const { stream } = sink(false);
for (let i = 0; i < 50; i++) {
const p = createProgress({ mode: 'json', stream, minIntervalMs: 0, minItems: 1 });
p.start(`phase_${i}`, 1);
p.finish();
}
// After 50 reporter lifecycles, still exactly one handler and zero leaked live entries.
expect(__signalHandlerInstalledForTest()).toBe(installedBefore || true);
expect(__liveReporterCountForTest()).toBe(0);
});
test('startHeartbeat() fires heartbeats and stop() clears', async () => {
const { stream, read } = sink(false);
const p = createProgress({ mode: 'json', stream, minIntervalMs: 0, minItems: 1 });
p.start('slow_query');
// Larger window + wider tolerance: under 4-way parallel CI shards on a
// contended host, setTimeout's effective quantum can balloon and a tight
// 85ms/2-6 bound flakes. We just need to confirm "fires multiple times,
// stops cleanly" — exact count isn't load-bearing.
const stop = startHeartbeat(p, 'still running…', 20);
await new Promise((r) => setTimeout(r, 200));
stop();
p.finish();
const events = parseJsonl(read());
const hb = events.filter((e) => e.event === 'heartbeat');
expect(hb.length).toBeGreaterThanOrEqual(1);
expect(hb.length).toBeLessThanOrEqual(20);
});
test('finish without prior start is a no-op (no crash)', () => {
const { stream, read } = sink(false);
const p = createProgress({ mode: 'json', stream });
expect(() => p.finish()).not.toThrow();
expect(read()).toBe('');
});
test('tick without prior start is a no-op (no crash)', () => {
const { stream, read } = sink(false);
const p = createProgress({ mode: 'json', stream });
expect(() => p.tick()).not.toThrow();
expect(read()).toBe('');
});
});