Files
gbrain/test/select-e2e.test.ts
T
90e22c22e2 v0.23.1 feat: local CI gate + 4-tier wall-time optimization (~13x faster) (#528)
* feat: diff-aware E2E test selector

Adds scripts/select-e2e.ts: reads git diff vs origin/master, classifies
the change set (EMPTY/DOC_ONLY/SRC), and emits the relevant E2E test files
on stdout. Fail-closed by design: any unmapped src/ change runs all E2E.

- scripts/e2e-test-map.ts: hand-tuned path-glob -> test files map
- scripts/select-e2e.ts: pure-function selector with three explicit cases
- scripts/run-e2e.sh: accepts optional file list from argv + --dry-run-list
- test/select-e2e.test.ts: 24 cases including 3 codex regression guards
  (skills/, untracked files, unmapped src/)

* feat: local CI gate via docker compose

Adds bun run ci:local — runs every check GH Actions runs (gitleaks +
unit + 29 E2E files) inside a Docker container that bind-mounts the
repo. Pure bind-mount + named volumes (gbrain-ci-node-modules,
gbrain-ci-bun-cache, gbrain-ci-pg-data) for fast warm restarts.

- docker-compose.ci.yml: pgvector/pgvector:pg16 + oven/bun:1
- scripts/ci-local.sh: orchestrator with --diff, --no-pull, --clean
- gitleaks runs on host (scoped to working dir + branch commits)
- DATABASE_URL unset for unit phase (matches GH Actions split)
- git installed in container at startup (oven/bun:1 omits it)
- Postgres host port via GBRAIN_CI_PG_PORT env (default 5434)

Stronger than PR CI: runs all 29 E2E files vs CI's 2-file Tier 1.

* chore: bump version and changelog (v0.23.1)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: document local CI gate for v0.23.1

CLAUDE.md gains key-files entries for docker-compose.ci.yml,
scripts/ci-local.sh, scripts/select-e2e.ts + e2e-test-map.ts, and the
scripts/run-e2e.sh argv tweak. Pre-ship requirements section now lists
the Docker-based local gate as Path A alongside the manual lifecycle.

CONTRIBUTING.md tests section adds the bun run ci:local / ci:local:diff /
ci:select-e2e block with prerequisites (Docker engine + gitleaks) and the
GBRAIN_CI_PG_PORT override.

AGENTS.md "Before shipping" promotes ci:local as the easiest path and
keeps the manual lifecycle as a fallback.

README.md Contributing section points to ci:local for the full gate.

CHANGELOG.md untouched — v0.23.1 entry already finalized.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: SHARD=N/M env support in scripts/run-e2e.sh

Filters the E2E file list to every M-th file starting at index N (1-indexed).
Sequential execution within a shard preserves the TRUNCATE CASCADE no-race
property documented at the top of the file. Empty-shard handling under
`set -u` uses ${arr[@]:-} fallback.

Standalone change; not yet wired up in ci-local.sh.

* feat: 4-way parallel E2E shards in ci:local

Replaces the single postgres service with 4 (postgres-1..4) on host ports
5434-5437. scripts/ci-local.sh fans 4 workers via xargs -P4 inside the
runner container; each pinned to its own DATABASE_URL via SHARD=N/4.

Wall-time on a 16-core host: ~6 min sequential -> ~1.5-2 min sharded.
Total full-gate wall-time goes from ~25 min to ~3-5 min warm.

Also handles git-worktree (Conductor) layouts: when /app/.git is a file
instead of a directory, parse the gitdir + commondir and bind-mount the
shared host gitdir at its absolute path. Without this, in-container
`git ls-files` (used by scripts/check-trailing-newline.sh and friends)
exits 128 with "not a git repository". Also runs
`git config --global --add safe.directory '*'` inside the container so
the root-uid container can read host-uid gitdir without "dubious
ownership" rejection.

CHANGELOG entry updated to cover the speedup.

- docker-compose.ci.yml: 4 pgvector services + per-shard named volumes
- scripts/ci-local.sh: parallel xargs orchestration + worktree mount fix
- CHANGELOG.md v0.23.1: 4-way sharded wall-time, 36 E2E files, --no-shard flag

* chore: regenerate llms-full.txt for v0.23.1 doc updates

Required by test/build-llms.test.ts case 4 — committed llms-full.txt
must match `bun run build:llms` output. The CHANGELOG + CLAUDE.md
updates in this branch shifted bytes; regen catches up.

* feat: scripts/run-unit-shard.sh + slow-test convention

Tier 1 + Tier 4 plumbing:
- scripts/run-unit-shard.sh: SHARD=N/M filter for unit files (excludes
  test/e2e/*). Excludes *.slow.test.ts (Tier 4 convention) so the fast
  shard fan-out skips known-slow files; CI's `bun run test` still includes
  them via default discovery.
- scripts/run-slow-tests.sh: companion that runs ONLY *.slow.test.ts.
  Wired as `bun run test:slow`.
- scripts/profile-tests.sh: portable awk parser that extracts the top-N
  slowest tests from any captured `bun test` output. Wired as
  `bun run test:profile`. Use it to pick demotion candidates.

* feat: PGLite snapshot fixture for ~4.5x faster cold init (Tier 3)

scripts/build-pglite-snapshot.ts boots a fresh PGLite, runs the full
initSchema() (forward bootstrap + 30 migrations), and dumps the post-init
state to test/fixtures/pglite-snapshot.tar plus a SHA-256 schema hash
sidecar (.version). Both gitignored — built on demand via
`bun run build:pglite-snapshot`.

PGLiteEngine.connect() reads GBRAIN_PGLITE_SNAPSHOT env: validates the
sidecar hash against the in-process MIGRATIONS hash, loads via PGLite's
loadDataDir blob, sets _snapshotLoaded so initSchema() short-circuits.
Measured per-file cold init drops from 828ms → 181ms.

Bootstrap-correctness tests (bootstrap.test.ts,
schema-bootstrap-coverage.test.ts) explicitly delete the env at file
top so they keep exercising the cold path they verify.

* feat: --classify-only + heartbeat tolerance fix (Tiers 2 + flake fix)

- scripts/select-e2e.ts: --classify-only flag emits EMPTY|DOC_ONLY|SRC.
  Used by ci-local.sh's --diff fast-path to skip the heavy gate when
  only docs changed.
- test/progress.test.ts: startHeartbeat tolerance widened to 1-20 over
  200ms (was 2-6 over 85ms). Under 4-way parallel shard load on a
  contended host, setTimeout's effective quantum balloons and the tight
  bound flakes. The test still verifies "fires multiple times, stops
  cleanly" — exact count was never load-bearing.

* feat: 4-way unit + E2E sharding in ci-local.sh + CHANGELOG (Tiers 1-4)

ci-local.sh ties the four tiers together:
- Tier 2: pre-flight diff classification on host. DOC_ONLY exits in ~5s
  (gitleaks only, no postgres, no container).
- Tier 1: guards + typecheck run ONCE before fan-out. xargs -P4 then
  spawns 4 shards inside the runner container, each running unit phase
  (env -u DATABASE_URL bash run-unit-shard.sh) followed by E2E phase
  (DATABASE_URL=postgres-N bash run-e2e.sh) — both sharded N/4. Per-shard
  logs in /tmp/shard-logs/shard-N.log; printed in shard order at the end.
- Tier 3: snapshot fixture built once at runner startup if missing,
  GBRAIN_PGLITE_SNAPSHOT exported so all shards inherit.
- Tier 4: run-unit-shard.sh excludes *.slow.test.ts; run-slow-tests.sh
  + test:slow npm script handle the demoted set.
- --no-shard preserves the legacy single-process flow for debug.

package.json: build:pglite-snapshot, test:slow, test:profile scripts.

Measured wall-time on 16-core host: 100s warm (down from ~22 min cold
single-process). 4 shards × ~640-1024 unit tests each, plus 9 E2E
files each. PGLite snapshot saves 4.5× per cold init (828ms → 181ms).

CHANGELOG.md updated with measured numbers + four-tier breakdown.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 20:47:32 -07:00

212 lines
6.7 KiB
TypeScript

// test/select-e2e.test.ts
//
// Unit tests for the diff-based E2E selector. Pure-function tests — no git,
// no filesystem. The 3 codex regression guards (skills/, untracked,
// unmapped src/) are explicitly named.
import { describe, expect, test } from "bun:test";
import {
E2E_TEST_MAP,
} from "../scripts/e2e-test-map.ts";
import {
classify,
matchGlob,
selectTests,
} from "../scripts/select-e2e.ts";
const ALL_E2E = [
"test/e2e/cycle.test.ts",
"test/e2e/dream.test.ts",
"test/e2e/code-indexing.test.ts",
"test/e2e/engine-parity.test.ts",
"test/e2e/graph-quality.test.ts",
"test/e2e/http-transport.test.ts",
"test/e2e/integrity-batch.test.ts",
"test/e2e/jsonb-roundtrip.test.ts",
"test/e2e/mcp.test.ts",
"test/e2e/mechanical.test.ts",
"test/e2e/migrate-chain.test.ts",
"test/e2e/migration-flow.test.ts",
"test/e2e/minions-concurrency.test.ts",
"test/e2e/minions-resilience.test.ts",
"test/e2e/minions-shell-pglite.test.ts",
"test/e2e/minions-shell.test.ts",
"test/e2e/multi-source.test.ts",
"test/e2e/postgres-bootstrap.test.ts",
"test/e2e/postgres-jsonb.test.ts",
"test/e2e/search-exclude.test.ts",
"test/e2e/search-quality.test.ts",
"test/e2e/search-swamp.test.ts",
"test/e2e/skills.test.ts",
"test/e2e/sync.test.ts",
"test/e2e/upgrade.test.ts",
"test/e2e/worker-abort-recovery.test.ts",
"test/e2e/doctor-progress.test.ts",
"test/e2e/frontmatter-migration.test.ts",
"test/e2e/openclaw-reference-compat.test.ts",
];
function select(changedFiles: string[]): string[] {
return selectTests({
changedFiles,
allE2ETests: ALL_E2E,
map: E2E_TEST_MAP,
});
}
describe("matchGlob", () => {
test("** matches any path segments", () => {
expect(matchGlob("src/core/search/**", "src/core/search/intent.ts")).toBe(
true
);
expect(
matchGlob("src/core/search/**", "src/core/search/sub/dir/file.ts")
).toBe(true);
});
test("* matches one segment, no /", () => {
expect(matchGlob("src/*.ts", "src/cli.ts")).toBe(true);
expect(matchGlob("src/*.ts", "src/core/cli.ts")).toBe(false);
});
test("literal path matches itself", () => {
expect(matchGlob("src/core/cycle.ts", "src/core/cycle.ts")).toBe(true);
expect(matchGlob("src/core/cycle.ts", "src/core/cycle.test.ts")).toBe(false);
});
test("throws on unsupported glob syntax", () => {
expect(() => matchGlob("src/[abc].ts", "src/a.ts")).toThrow();
expect(() => matchGlob("src/{foo,bar}.ts", "src/foo.ts")).toThrow();
});
});
describe("classify", () => {
test("empty -> EMPTY", () => {
expect(classify([])).toBe("EMPTY");
});
test("only doc paths -> DOC_ONLY", () => {
expect(classify(["README.md", "docs/foo.md", "CHANGELOG.md"])).toBe(
"DOC_ONLY"
);
});
test("any non-doc path -> SRC", () => {
expect(classify(["README.md", "src/cli.ts"])).toBe("SRC");
});
test("skills/ is NOT doc-only (Codex F4)", () => {
expect(classify(["skills/RESOLVER.md"])).toBe("SRC");
});
});
describe("selectTests", () => {
test("case 1: empty diff -> all E2E", () => {
expect(select([])).toEqual(ALL_E2E.slice().sort());
});
test("case 2: doc-only -> nothing", () => {
expect(select(["README.md", "docs/guides/foo.md", "CHANGELOG.md"])).toEqual(
[]
);
});
test("case 3: single mapped src -> only mapped tests", () => {
expect(select(["src/core/search/intent.ts"])).toEqual([
"test/e2e/search-exclude.test.ts",
"test/e2e/search-quality.test.ts",
"test/e2e/search-swamp.test.ts",
]);
});
test("case 4: multiple mapped srcs -> union, no duplicates", () => {
const result = select([
"src/core/search/intent.ts",
"src/core/minions/queue.ts",
]);
expect(result).toContain("test/e2e/search-quality.test.ts");
expect(result).toContain("test/e2e/minions-concurrency.test.ts");
// Determinism: dedup preserved
const set = new Set(result);
expect(set.size).toBe(result.length);
});
test("case 5: schema escape-hatch -> all", () => {
expect(select(["src/schema.sql"])).toEqual(ALL_E2E.slice().sort());
});
test("case 6 (Codex F4 regression): skills/ -> all", () => {
expect(select(["skills/RESOLVER.md"])).toEqual(ALL_E2E.slice().sort());
expect(select(["skills/migrations/v0.22.4.md"])).toEqual(
ALL_E2E.slice().sort()
);
});
test("case 7 (Codex F5 regression): untracked file -> fail-closed -> all", () => {
// The selector receives the union of (committed, unstaged, untracked).
// We simulate "untracked" by passing the path in the changed list with
// no map entry — should fail-closed to ALL.
expect(select(["src/foo-new.ts"])).toEqual(ALL_E2E.slice().sort());
});
test("case 8 (Codex F1 headline): unmapped src/ -> fail-closed -> all", () => {
// src/core/utils.ts is not in the map; must fail-closed.
expect(select(["src/core/utils.ts"])).toEqual(ALL_E2E.slice().sort());
// src/cli.ts is also not in the map.
expect(select(["src/cli.ts"])).toEqual(ALL_E2E.slice().sort());
});
test("case 9: directly-modified test file is included", () => {
// Touching a test file directly with no other src changes:
// - test/e2e/foo.test.ts is in changedFiles
// - it gets added to result
// - no other map entries match
// - result has 1 entry, so NOT fail-closed
expect(select(["test/e2e/sync.test.ts"])).toEqual([
"test/e2e/sync.test.ts",
]);
});
test("case 10: mixed doc + mapped-src -> only src-relevant", () => {
const result = select([
"README.md",
"docs/foo.md",
"src/core/search/intent.ts",
]);
expect(result).toEqual([
"test/e2e/search-exclude.test.ts",
"test/e2e/search-quality.test.ts",
"test/e2e/search-swamp.test.ts",
]);
});
test("escape-hatch: package.json -> all", () => {
expect(select(["package.json"])).toEqual(ALL_E2E.slice().sort());
});
test("escape-hatch: bun.lock -> all", () => {
expect(select(["bun.lock"])).toEqual(ALL_E2E.slice().sort());
});
test("escape-hatch: .github/workflows/** -> all", () => {
expect(select([".github/workflows/test.yml"])).toEqual(
ALL_E2E.slice().sort()
);
});
test("escape-hatch: src/commands/migrations/** -> all", () => {
expect(select(["src/commands/migrations/v0_22_8.ts"])).toEqual(
ALL_E2E.slice().sort()
);
});
test("escape-hatch: test/e2e/helpers.ts -> all", () => {
expect(select(["test/e2e/helpers.ts"])).toEqual(ALL_E2E.slice().sort());
});
test("escape-hatch beats narrow map: schema + search both touched", () => {
// schema.sql is escape-hatch; should win over search narrow match.
expect(select(["src/schema.sql", "src/core/search/intent.ts"])).toEqual(
ALL_E2E.slice().sort()
);
});
});