mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
814258dda67945ffec9457a1e73980e947b7e462
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
3a2605e9a0 |
v0.41.6.0 feat(ci): CI test speedup — 23min → ~9min via matrix 4→6 + weight-aware sharding + auto SHA cache + parallel verify (#1444)
* feat(ci): scripts/run-verify-parallel.sh — parallel verify dispatcher Fans out the 21 pre-test grep guards via & + wait, captures per-check exit codes in a tempdir, aggregates failures with named check + log tail to stderr on miss. Wallclock 27s sequential → 13s parallel locally (2x). Bigger CI win is shard 1 deload (workflow restructure in a later commit). Pinned by test/scripts/run-verify-parallel.test.ts (6 cases: CLI contract + synthetic dispatcher failure-surfacing). * feat(ci): weight-aware LPT bin-packer + auto SHA cache hash scripts/sharding.ts (NEW) — pure TypeScript LPT bin-packer. Sort weights desc, assign each file to the shard with current minimum total. Worst-case makespan within 4/3 of optimal, O(n log n). Missing weights fall back to corpus median (not 0). New test file → ships immediately without regenerating weights. Pinned by test/scripts/sharding.test.ts (23 cases). scripts/mine-shard-weights.ts (NEW) — scrapes per-file timing from gh run view --log via timestamp delta between ##[group]test/foo.test.ts: headers within a shard. Three input modes: --run <ID>, --from-file <PATH>, stdin. Stable JSON output (sorted keys). Initial weights mined from run 26398061007. Pinned by test/scripts/mine-shard-weights.test.ts (15 cases). scripts/ci-cache-hash.sh (NEW) — deterministic 16-char sha256 over git ls-files -s minus deny-list (CHANGELOG/TODOS/README/LICENSE/ docs/**/*.md). CLAUDE.md, AGENTS.md, skills/**/* deliberately INCLUDED (8+ test files read them; deny-listing would create false-pass holes). ~40ms on 1891 files. Pinned by test/scripts/ci-cache-hash.test.ts (24 cases: 8 CRITICAL false-pass guards + 7 SAFE deny-list invariants + 9 edge cases). scripts/test-weights.json (NEW) — 712 weights. Total 3306s observed runtime; median 30ms; max 6 min outlier. * chore: bump version and changelog (v0.41.6.0) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
|
d97f159793 |
v0.26.4 test: parallel unit-test loop (12x speedup, failure-first logging) (#605)
* test: parallel unit-test wrapper + failure-first logging (commit 1/8) Lay foundation for v0.26.4 parallel test loop: - scripts/run-unit-parallel.sh: spawns N shards (default min(8, cpu_count)) via run-unit-shard.sh, captures per-shard logs, post-shard single-writer failure-log aggregation at .context/test-failures.log, 10s heartbeat to stderr, per-shard 600s timeout (gtimeout/timeout/bg-pid fallback chain), loud final banner with absolute path + tail-30 of failures, summary file for at-a-glance status. Single writer eliminates concurrent-write hazards on the failure log. - scripts/run-serial-tests.sh: discovers *.serial.test.ts files (concurrency- unsafe by design), runs them with --max-concurrency=1. Invoked after the parallel pass. - scripts/run-unit-shard.sh: now accepts --max-concurrency=N (forwarded to bun test); --dry-run-list moved into argv parsing alongside; excludes *.serial.test.ts in addition to *.slow.test.ts. - bunfig.toml: trim stale comment about typecheck-chained timeout. - .gitignore: add .context/ (Conductor workspace artifacts directory; the failure log + summary + per-shard logs all live here). No package.json changes yet (commit 2). No test reorganization yet (commits 4-7). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: split package.json scripts; bun run test = parallel fast loop (commit 2/8) Per Codex Tension #4 (verify scope), distinguish three tiers cleanly: - `bun run test` = fast loop, file-level parallel fan-out via the new wrapper (scripts/run-unit-parallel.sh). No pre-checks, no typecheck, no wasm compile in the hot path. ~15s of pre-test gates removed. - `bun run verify` = CI's authoritative gate set: check:jsonb + check:progress + check:wasm + typecheck. Matches what .github/workflows/test.yml runs on shard 1, no scope drift. The 4 checks not in CI (privacy, no-legacy-getconnection, trailing-newline, exports-count) move to `bun run check:all` for opt-in local use. - `bun run test:full` = verify + parallel + slow + smart e2e (runs e2e only if DATABASE_URL is set; else loud skip notice to stderr per Open Item #7). The local equivalent of "everything CI runs." Adds `bun run test:serial` for the *.serial.test.ts subset (concurrency- unsafe files run with --max-concurrency=1). Bumps VERSION + package.json to 0.26.4. Both move together per the CI version-gate contract in CLAUDE.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: fix-wave for parallel wrapper + tighten privacy gate (commit 3/5) Wave: makes the new wrapper actually green and tightens the CI gate it exposed. Wrapper bug fixes (scripts/run-unit-parallel.sh): - grep_count helper: avoids the `grep -c | echo 0` double-output bug where 0 matches yields a 2-line "0\n0" string and breaks arithmetic. - bun_summary_count helper: parses Bun's actual end-of-shard summary format (`N pass` / `N fail` / `N skip`), not the per-test markers (which are `✓` / `(fail)`, never `(pass)` / `(skip)`). - Heartbeat now reads `^\s+✓` (Bun's per-test pass marker) for live progress mid-run; final summary still uses the summary-line counts for accuracy. Privacy gate tightening: - Move scripts/check-privacy.sh into `bun run verify` (was previously only in the now-removed `bun run test` chain). Without this, after commit 2 the privacy check ran in nothing automatic. - .github/workflows/test.yml now calls `bun run verify` instead of inlining the gate list. Single source of truth for "what's the ship gate." This is what verify == CI was supposed to mean per Codex T#4. - Pre-existing `Wintermute` references in src/core/mounts-cache.ts:6 and :324 caught by the now-running gate; replaced with `your OpenClaw` per CLAUDE.md privacy rule (verify gate now passes on master HEAD). - test/privacy-script-wired.test.ts updated: regression guard now asserts verify includes check:privacy AND that test.yml runs `bun run verify`, replacing the obsolete "test script includes check-privacy.sh" assertion. Quarantine 2 cross-file-contention flakes: - test/brain-registry.test.ts: 28 tests pass alone (41ms); 1 test ("empty/null/undefined id routes to host") fails when run alongside other files in the same shard. Renamed → *.serial.test.ts so it runs in scripts/run-serial-tests.sh's serial pass after the parallel pass completes. - test/reconcile-links.test.ts: 6 tests pass alone (1s); a beforeEach hook times out (~896s) under cross-file contention. Same treatment. Both flakes are bun-process-level shared-state leaks (PGLite singletons or top-level imports). Fixing them properly is the v0.27.0+ intra-file parallelism project (TODO P0 — see commit 5). Measurement after this commit: bun run test = 94s (was 18 min sequential) 3639 pass, 0 fail, 0 skip across 8 parallel shards + 34 serial tests Failure-log + heartbeat + summary all working Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: regression tests for parallel wrapper + serial-test contracts (commit 4/5) Three regression suites pin the v0.26.4 contracts. Without these, future refactors of the wrapper or shard scripts could silently regress the work in commits 1-3. test/scripts/run-unit-shard.test.ts (4 cases — gap b): - Asserts the unit-shard `--dry-run-list` output excludes every *.slow.test.ts and *.serial.test.ts file, plus the test/e2e/ subtree. - Catches a future `find` expression that drops one of the `-not -name` clauses and silently un-quarantines slow/serial files into the parallel pass. test/scripts/serial-files.test.ts (3 cases — gap e): - Every checked-in *.serial.test.ts (via `git ls-files`) is listed by scripts/run-serial-tests.sh's `--dry-run-list`. - The script's source contains `bun test --max-concurrency=1` (the serial-pass guarantee that quarantined files don't run intra-file concurrent and reintroduce the contention they were quarantined for). - Disjoint set: a file is never in both the unit-shard list AND the serial list — pins the carve-out contract. test/scripts/run-unit-parallel.test.ts (6 cases — gaps a + d): - Exit-code propagation (a): wrapper exits non-zero when ANY shard has a failing test; exits zero when all pass. The hardest contract to silently break in a fan-out wrapper (`for ... &; wait` returns the LAST child's status, not any failure's). - Failure-log contract (d): on failure, .context/test-failures.log exists, is non-empty, contains the `--- shard N:` prefix and the failing test's describe text. Stderr banner contains the absolute log path. On success, the log is cleared (no stale content). - Summary file format: `shard N/M: pass=X fail=Y skip=Z rc=W` per shard, machine-parseable for future tooling. The wrapper test runs against a 4-file tempdir (3 pass + 1 fail) so it executes in ~500ms; spawning the wrapper against the real test suite would take ~90s and isn't worth the cost in a regression suite. All 13 cases pass on first run. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(v0.26.4): testing tier docs + CHANGELOG + intra-file P0 TODO (commit 5/5) Closes the v0.26.4 ship. CLAUDE.md Testing section rewritten: - New tier table: test (fast loop, 85s) / verify (CI gates, 12s) / test:full (everything local) / test:slow / test:serial / test:e2e / check:all. Each row names its scope, wallclock, and when to use. - Intentional CI vs local divergence section: CI matrix (test-shard.sh, hash-bucketed, includes slow) vs local fast loop (run-unit-shard.sh, round-robin, excludes slow + serial). Codex correctly flagged that a parity test would always fail by design — this is the documentation that explains why. - Failure-first logging contract: .context/test-failures.log format, stderr banner, summary file, wedge handling. - File taxonomy: *.test.ts / *.slow.test.ts / *.serial.test.ts / test/e2e/. Names the two currently-quarantined files and points at the intra-file P0 TODO for the proper fix. CHANGELOG.md `## [0.26.4]` entry per voice rules: - Two-line headline: "bun run test finishes in 85 seconds. Was 18 minutes." + failure-log directive. - Lead paragraph names what shipped and why. - Numbers-that-matter table: BEFORE / AFTER / Δ for wallclock, pre-test gates, failure visibility, shards, pipe-survival. - "What this means for you" closing tied to the inner-loop user. - "To take advantage of v0.26.4" block per the v0.13+ self-repair template (gbrain upgrade + contributor steps). - Itemized changes by area (new scripts, script extensions, package.json tier split, CI tightening, failure-first logging, quarantine, regression tests, bunfig). - "What did NOT ship" section names the intra-file project + E2E template-DB project as P0/P1 follow-ups with concrete acceptance criteria. - Process section names the codex review + scope-correction loop honestly: "snapped back to ship today once empirical measurement showed Bun's --max-concurrency does nothing on tests not marked test.concurrent()." - For-contributors note on portability + single-writer + fallback paths. TODOS.md adds two P-rated entries: - P0: intra-file parallelism via --concurrent flag. Sweep ~58 PGLite sites + ~40 env mutations + 2 mock.module sites. Target: bun run test < 30s. ~1-2 weeks. Detailed acceptance criteria. References Codex findings and plan-file rationale. - P1: E2E parallelism via Postgres template databases. CREATE DATABASE TEMPLATE gbrain_template per test file. ~1-2 days. llms.txt + llms-full.txt regenerated via `bun run build:llms` to absorb the CLAUDE.md changes (per CLAUDE.md's "After any release ship that touches the Key Files annotations in CLAUDE.md, run bun run build:llms" rule). The build-llms regression test was firing in shard 7 of the parallel pass — caught the drift, regeneration cleared it. Final measurement after fix: 94s wallclock, 3652 pass, 0 fail across 8 parallel shards + 34 serial tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
4fc1246606 |
v0.24.0: production-hardening pass on the skillify loop (#387)
* feat: v0.19.0 — skillify loop + AGENTS.md compat + brain-first convention This is the v0.19.0 release. The branch ships four new CLI commands, a refactor to check-resolvable, and an expansion of the brain-first convention for sub-agent tool discovery. The original commit message described only the convention expansion, undercounting the scope by ~5x; this amend captures the full release. NEW COMMANDS - gbrain skillify scaffold <name> — 4 stub files + idempotent resolver row - gbrain skillify check [path] — 10-item post-task audit (promoted) - gbrain skillpack list / install — curated 25-skill bundle, atomic install - gbrain skillpack diff <name> — per-file diff preview - gbrain routing-eval — dedicated CI verb for Check 5 fixtures CHECK-RESOLVABLE REFACTOR - Accepts AGENTS.md as a resolver file alongside RESOLVER.md, at either the skills directory or one level up (workspace root layout). - Auto-derives the skill manifest by walking skills/*/SKILL.md when manifest.json is missing. - Splits ResolvableReport into errors[] + warnings[] so advisory checks (filing audit, routing gaps, DRY violations) don't break CI by default. - New --strict opt-in flag promotes warnings to exit 1. BRAIN-FIRST CONVENTION - skills/conventions/brain-first.md expanded from 5-step lookup guide to full sub-agent reference: tool inventory, lookup chain, score thresholds, authority hierarchy, sync rules, entity page conventions, sub-agent propagation rule. PRODUCTION-READINESS HARDENING (this branch's review pass) - routing-eval --llm: emits stderr placeholder notice + runs structural layer only. README, CHANGELOG, CLI help all rewritten consistently. Was a silent no-op against documented contract. - skillpack installer: receipt comment in fence (cumulative-slugs="...") preserves single-skill-install accumulation while letting install --all prune removed bundle skills cleanly. Unknown rows preserved + stderr warning for the operating agent. Pre-v0.19 fences upgrade silently. - skillify scaffold: resolver-row regex broadened to detect backticked, quoted, and bare path forms. No duplicate row on --force after the user normalizes formatting. - scripts/check-privacy.sh: now wired into package.json test chain so the wintermute-ban rule is actually enforced. New regression test. - E2E Tier 2 (LLM skills) promoted from schedule-only to required per-PR CI. Local Tier 1 + Tier 2 verified clean. - Stale v0.17/v0.18 version labels rewritten across new files. TESTS - test/routing-eval-cli.test.ts: 4 cases covering --llm warn semantics - test/privacy-script-wired.test.ts: regression guard for CI wiring - test/skillpack-install.test.ts: 4 new cases for receipt + cumulative + unknown-row preserve+warn + pre-v0.19 upgrade path - test/skillify-scaffold.test.ts: 4 new cases for broadened regex VERIFICATION - bun test: 2237 pass / 18 known PGLite-contention flakes (CI green; documented as P3 dev-experience in TODOS.md) - bun run typecheck: clean - bun run test:e2e: 18/19 files green (1 pre-existing flake on master, not caused by this branch — verified via git stash) - llms.txt + llms-full.txt regenerated to match README + CHANGELOG Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: scrub banned fork name from public artifacts The privacy guard wired into the test chain in this branch caught 5 pre-existing references to the banned OpenClaw fork name in CHANGELOG.md (2x), skills/migrations/v0.19.0.md (1x), src/cli.ts (1x), and src/commands/sync.ts (1x). All originated in master's v0.19.0 release notes and migration doc when the privacy script existed but wasn't wired into CI yet. Replacements per CLAUDE.md privacy mapping: - Origin-story copy (CHANGELOG layer narratives, code comments naming the production deployment that drove the feature) → "Garry's OpenClaw" - Reader-facing migration step → "your OpenClaw" No code semantics changed. Comments + headings only. Verification: scripts/check-privacy.sh exits 0, full CI guard chain green (privacy + jsonb + progress + wasm + typecheck). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump VERSION to 0.24.0 + new CHANGELOG entry Bump branch version above master's v0.21.0 per CLAUDE.md "CHANGELOG + VERSION are branch-scoped" rule. The new v0.24.0 entry at the top of CHANGELOG covers what THIS branch adds vs master: - routing-eval --llm honesty pass (4-surface contract drift fix) - skillpack installer cumulative-receipt + unknown-row preserve+warn (the Codex-caught regression that would have shipped in master if the original v0.19.0 had landed without this branch's review pass) - skillify scaffold resolver-row regex broadening (backtick + quoted + bare forms; idempotency contract preserved under hand-editing) - 5 banned-name leaks scrubbed from public artifacts - check-privacy.sh wired into CI test chain + regression guard test - 7 stale v0.17/v0.18 version labels rewritten across 5 files - Tier 2 (LLM-skills E2E) promoted from schedule-only to required per-PR VERSION 0.21.0 → 0.24.0 package.json version field synced. llms.txt + llms-full.txt regenerated (no content drift; sizes match). Test suite: 62/62 green across the 5 test files this branch added or extended (routing-eval-cli, privacy-script-wired, skillpack-install, skillify-scaffold, build-llms). CI guards: privacy + jsonb + progress + wasm + typecheck all clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update project documentation for v0.24.0 Auto-discovered drift via /document-release after the v0.24.0 hardening pass landed. All factual corrections clearly warranted by the diff. CLAUDE.md: - Skillpack installer: documented the cumulative-slugs receipt comment, install --all prune semantics, unknown-row preserve+warn behavior, and pre-v0.24 silent upgrade. Was previously vague about "tracks a skill manifest so install --update diffs cleanly" without explaining what the receipt is or why it matters. - routing-eval: replaced the false claim that --llm "opts into a Haiku tie-break layer for CI." Now correctly describes the placeholder semantic landed in v0.24.0 (stderr notice + structural-only run). README.md: - Skillpack section: added one paragraph on the receipt comment + the user-visible stderr message for hand-added rows. Connects the safe rerun promise to the v0.24.0 implementation that actually enforces it. CONTRIBUTING.md: - Running tests section: now recommends `bun run test` (full CI guard chain + typecheck + tests) before pushing. Names each guard so new contributors understand what catches what. The privacy guard (newly wired in v0.24.0) is one of these — without `bun run test` you'd skip it locally and find out from CI. llms-full.txt: regenerated to reflect CLAUDE.md changes. Verification: full guard chain green locally (privacy + jsonb + progress + wasm + typecheck). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Garry Tan <garry@ycombinator.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |