2 Commits

Include renames

Author	SHA1	Message	Date
Garry TanandClaude Fable 5	d26bcf12ec	test(e2e): PgBouncer transaction-mode pooler in CI + teardown e2e (#2084) ... Three consecutive waves (#1972 → #2015 → #2084) fixed pooler-teardown bugs verified only against one production deployment — CI had no transaction-mode pooler and could never see the class. Now it can: - docker-compose.ci.yml: `pgbouncer` service (transaction pooling) fronting postgres-1, mirroring the production split-pool topology (direct :5432 + pooled :6543). AUTH_TYPE=plain (pg16 SCRAM verifiers need the plaintext password in the userlist) + IGNORE_STARTUP_PARAMETERS for the statement_timeout/idle_in_transaction_session_timeout startup params gbrain's client sets (the Supabase pooler whitelists the same). - test/e2e/pgbouncer-teardown.test.ts: schema + fixture via the DIRECT url into a dedicated `gbrain_pgbouncer` database (never races shard TRUNCATEs), then spawns the real CLI against the POOLED url and asserts: exit 0, stdout intact (the #1959 truncation class), and NO "did not return within 10000ms — force-exiting" banner (pre-#2084 it printed on 100% of query-shaped ops on this topology). Class bound, not exact timing. Skips gracefully without GBRAIN_PGBOUNCER_URL. - scripts/ci-local.sh: threads GBRAIN_PGBOUNCER_URL + GBRAIN_PGBOUNCER_DIRECT_URL into all three e2e phases. Verified live: both tests green against pgbouncer 1.25.2 in transaction mode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 21:23:37 -07:00
3 people	90e22c22e2	v0.23.1 feat: local CI gate + 4-tier wall-time optimization (~13x faster) (#528) ... * feat: diff-aware E2E test selector Adds scripts/select-e2e.ts: reads git diff vs origin/master, classifies the change set (EMPTY/DOC_ONLY/SRC), and emits the relevant E2E test files on stdout. Fail-closed by design: any unmapped src/ change runs all E2E. - scripts/e2e-test-map.ts: hand-tuned path-glob -> test files map - scripts/select-e2e.ts: pure-function selector with three explicit cases - scripts/run-e2e.sh: accepts optional file list from argv + --dry-run-list - test/select-e2e.test.ts: 24 cases including 3 codex regression guards (skills/, untracked files, unmapped src/) * feat: local CI gate via docker compose Adds bun run ci:local — runs every check GH Actions runs (gitleaks + unit + 29 E2E files) inside a Docker container that bind-mounts the repo. Pure bind-mount + named volumes (gbrain-ci-node-modules, gbrain-ci-bun-cache, gbrain-ci-pg-data) for fast warm restarts. - docker-compose.ci.yml: pgvector/pgvector:pg16 + oven/bun:1 - scripts/ci-local.sh: orchestrator with --diff, --no-pull, --clean - gitleaks runs on host (scoped to working dir + branch commits) - DATABASE_URL unset for unit phase (matches GH Actions split) - git installed in container at startup (oven/bun:1 omits it) - Postgres host port via GBRAIN_CI_PG_PORT env (default 5434) Stronger than PR CI: runs all 29 E2E files vs CI's 2-file Tier 1. * chore: bump version and changelog (v0.23.1) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: document local CI gate for v0.23.1 CLAUDE.md gains key-files entries for docker-compose.ci.yml, scripts/ci-local.sh, scripts/select-e2e.ts + e2e-test-map.ts, and the scripts/run-e2e.sh argv tweak. Pre-ship requirements section now lists the Docker-based local gate as Path A alongside the manual lifecycle. CONTRIBUTING.md tests section adds the bun run ci:local / ci:local:diff / ci:select-e2e block with prerequisites (Docker engine + gitleaks) and the GBRAIN_CI_PG_PORT override. AGENTS.md "Before shipping" promotes ci:local as the easiest path and keeps the manual lifecycle as a fallback. README.md Contributing section points to ci:local for the full gate. CHANGELOG.md untouched — v0.23.1 entry already finalized. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: SHARD=N/M env support in scripts/run-e2e.sh Filters the E2E file list to every M-th file starting at index N (1-indexed). Sequential execution within a shard preserves the TRUNCATE CASCADE no-race property documented at the top of the file. Empty-shard handling under `set -u` uses ${arr[@]:-} fallback. Standalone change; not yet wired up in ci-local.sh. * feat: 4-way parallel E2E shards in ci:local Replaces the single postgres service with 4 (postgres-1..4) on host ports 5434-5437. scripts/ci-local.sh fans 4 workers via xargs -P4 inside the runner container; each pinned to its own DATABASE_URL via SHARD=N/4. Wall-time on a 16-core host: ~6 min sequential -> ~1.5-2 min sharded. Total full-gate wall-time goes from ~25 min to ~3-5 min warm. Also handles git-worktree (Conductor) layouts: when /app/.git is a file instead of a directory, parse the gitdir + commondir and bind-mount the shared host gitdir at its absolute path. Without this, in-container `git ls-files` (used by scripts/check-trailing-newline.sh and friends) exits 128 with "not a git repository". Also runs `git config --global --add safe.directory '*'` inside the container so the root-uid container can read host-uid gitdir without "dubious ownership" rejection. CHANGELOG entry updated to cover the speedup. - docker-compose.ci.yml: 4 pgvector services + per-shard named volumes - scripts/ci-local.sh: parallel xargs orchestration + worktree mount fix - CHANGELOG.md v0.23.1: 4-way sharded wall-time, 36 E2E files, --no-shard flag * chore: regenerate llms-full.txt for v0.23.1 doc updates Required by test/build-llms.test.ts case 4 — committed llms-full.txt must match `bun run build:llms` output. The CHANGELOG + CLAUDE.md updates in this branch shifted bytes; regen catches up. * feat: scripts/run-unit-shard.sh + slow-test convention Tier 1 + Tier 4 plumbing: - scripts/run-unit-shard.sh: SHARD=N/M filter for unit files (excludes test/e2e/*). Excludes *.slow.test.ts (Tier 4 convention) so the fast shard fan-out skips known-slow files; CI's `bun run test` still includes them via default discovery. - scripts/run-slow-tests.sh: companion that runs ONLY *.slow.test.ts. Wired as `bun run test:slow`. - scripts/profile-tests.sh: portable awk parser that extracts the top-N slowest tests from any captured `bun test` output. Wired as `bun run test:profile`. Use it to pick demotion candidates. * feat: PGLite snapshot fixture for ~4.5x faster cold init (Tier 3) scripts/build-pglite-snapshot.ts boots a fresh PGLite, runs the full initSchema() (forward bootstrap + 30 migrations), and dumps the post-init state to test/fixtures/pglite-snapshot.tar plus a SHA-256 schema hash sidecar (.version). Both gitignored — built on demand via `bun run build:pglite-snapshot`. PGLiteEngine.connect() reads GBRAIN_PGLITE_SNAPSHOT env: validates the sidecar hash against the in-process MIGRATIONS hash, loads via PGLite's loadDataDir blob, sets _snapshotLoaded so initSchema() short-circuits. Measured per-file cold init drops from 828ms → 181ms. Bootstrap-correctness tests (bootstrap.test.ts, schema-bootstrap-coverage.test.ts) explicitly delete the env at file top so they keep exercising the cold path they verify. * feat: --classify-only + heartbeat tolerance fix (Tiers 2 + flake fix) - scripts/select-e2e.ts: --classify-only flag emits EMPTY|DOC_ONLY|SRC. Used by ci-local.sh's --diff fast-path to skip the heavy gate when only docs changed. - test/progress.test.ts: startHeartbeat tolerance widened to 1-20 over 200ms (was 2-6 over 85ms). Under 4-way parallel shard load on a contended host, setTimeout's effective quantum balloons and the tight bound flakes. The test still verifies "fires multiple times, stops cleanly" — exact count was never load-bearing. * feat: 4-way unit + E2E sharding in ci-local.sh + CHANGELOG (Tiers 1-4) ci-local.sh ties the four tiers together: - Tier 2: pre-flight diff classification on host. DOC_ONLY exits in ~5s (gitleaks only, no postgres, no container). - Tier 1: guards + typecheck run ONCE before fan-out. xargs -P4 then spawns 4 shards inside the runner container, each running unit phase (env -u DATABASE_URL bash run-unit-shard.sh) followed by E2E phase (DATABASE_URL=postgres-N bash run-e2e.sh) — both sharded N/4. Per-shard logs in /tmp/shard-logs/shard-N.log; printed in shard order at the end. - Tier 3: snapshot fixture built once at runner startup if missing, GBRAIN_PGLITE_SNAPSHOT exported so all shards inherit. - Tier 4: run-unit-shard.sh excludes *.slow.test.ts; run-slow-tests.sh + test:slow npm script handle the demoted set. - --no-shard preserves the legacy single-process flow for debug. package.json: build:pglite-snapshot, test:slow, test:profile scripts. Measured wall-time on 16-core host: 100s warm (down from ~22 min cold single-process). 4 shards × ~640-1024 unit tests each, plus 9 E2E files each. PGLite snapshot saves 4.5× per cold init (828ms → 181ms). CHANGELOG.md updated with measured numbers + four-tier breakdown. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-30 20:47:32 -07:00