mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
* v0.31.4.1 chore: align VERSION/package.json with #795 + mandate MAJOR.MINOR.PATCH.MICRO
PR #795 (takes v2) landed on master with `v0.31.4` in its commit subject but
never bumped VERSION, package.json, or CHANGELOG.md. Master shipped at 0.31.3.
This corrective release:
- Bumps VERSION + package.json to 0.31.4.1 (the dot-suffix follow-up channel
documented in CLAUDE.md, so the patch number doesn't churn to 0.31.5)
- Adds the v0.31.4.1 CHANGELOG entry covering takes v2 (lessons from a 100K-take
production extraction), the auth-on-Postgres regression fix, and the new
`gbrain eval takes-quality` CLI surface
- Updates CLAUDE.md to mandate `MAJOR.MINOR.PATCH.MICRO` for every new release.
Historical 3-segment versions in git log + migration filenames stay valid;
do not rewrite. Going forward only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: regenerate llms-full.txt for v0.31.4.1 doc edits
The build-llms regen-drift guard caught that llms-full.txt was stale relative
to the CHANGELOG + CLAUDE.md edits in the prior commit. Per CLAUDE.md the
bundle is auto-derived: bump VERSION/CHANGELOG/CLAUDE.md, then run
`bun run build:llms`. Did the second part now.
llms.txt unchanged (it's just the curated index). Only llms-full.txt picks
up the v0.31.4.1 CHANGELOG entry and the new "Version format is mandatory"
section in CLAUDE.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ci): exclude *.serial.test.ts from test-shard.sh hash buckets
Root cause of test (2) failing on the v0.31.4.1 PR (and on master since
#795 landed): CI's scripts/test-shard.sh hashed every test file into 4
shards via FNV-1a, INCLUDING *.serial.test.ts files. Serial files share
file-wide state (top-level mock.module, module singletons) that's
supposed to be quarantined by the .serial.test.ts naming + local
run-serial-tests.sh running them at --max-concurrency=1.
In CI the quarantine didn't apply. eval-takes-quality-runner.serial.test.ts
(new in #795) hashes into shard 2, where it calls:
mock.module('../src/core/ai/gateway.ts', () => ({
chat: async (opts) => { ... },
configureGateway: () => undefined,
}));
That replaces every export of gateway.ts at module-load time for the
WHOLE shard process. voyage-multimodal.test.ts also lives in shard 2
(both files happen to hash there), and it imports `embedMultimodal` from
gateway.ts. After the serial file loads, `embedMultimodal` is undefined
inside the shard process, and all 18 of voyage-multimodal's
embedMultimodal tests fail. Tests still passed locally because
run-unit-shard.sh excludes .serial files from its parallel pass.
Fix:
- scripts/test-shard.sh: add `-not -name '*.serial.test.ts'` to the
find expression so serial files no longer compete for shard buckets.
Add --dry-run-list flag to mirror run-unit-shard.sh's interface so
the regression test can introspect without spawning bun test.
- .github/workflows/test.yml: add a `bun run test:serial` step that
runs on shard 1 (which already runs `bun run verify`). Uses the
existing scripts/run-serial-tests.sh which invokes bun test at
--max-concurrency=1, matching local behavior.
- test/scripts/test-shard.slow.test.ts: 4 regression cases that pin
the contract (no serial files in any shard, no e2e files in any
shard, plain files partitioned without overlap). .slow.test.ts
because it shells out 4× with pure-bash FNV-1a hashing (~14s
wallclock); excluded from the local fast loop, runs in CI via the
same hash bucketing as other slow tests.
- CLAUDE.md: update the CI vs local divergence section so this
intentional asymmetry is documented going forward.
Build-llms drift in test (1) was fixed in the prior commit (c99a4af1).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: regenerate llms-full.txt for the CI-fix CLAUDE.md edits
The prior commit updated the "CI vs local: intentionally divergent file sets"
section in CLAUDE.md, which drifted llms-full.txt. Per CLAUDE.md the bundle
is auto-derived: edit CLAUDE.md, then run `bun run build:llms`. Did the
second part now.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
7267462311
commit
943e7b9dec
@@ -40,3 +40,9 @@ jobs:
|
||||
run: bun run verify
|
||||
- name: Run test shard ${{ matrix.shard }}/4
|
||||
run: scripts/test-shard.sh ${{ matrix.shard }} 4
|
||||
- name: Run *.serial.test.ts at --max-concurrency=1 (shard 1 only)
|
||||
# Serial files share file-wide state (top-level mock.module, module
|
||||
# singletons) that leaks across files in the same bun-test process.
|
||||
# test-shard.sh excludes them; this step runs them at concurrency=1.
|
||||
if: matrix.shard == 1
|
||||
run: bun run test:serial
|
||||
|
||||
@@ -2,6 +2,103 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.31.4.1] - 2026-05-10
|
||||
|
||||
**Takes v2: lessons from a 100K-take production extraction folded back into the runtime, and `gbrain auth` works on Postgres again.**
|
||||
|
||||
The first full production takes extraction (28,256 pages, 100,720 takes, $361 on Azure GPT-5.5) plus a cross-modal eval (GPT-5.5 + Opus 4.6 against a 5-dim rubric, 6.8/10 overall) surfaced five concrete things to fix in the runtime. Holder grammar is now enforced at parse time. Weights round to a 0.05 grid on insert at all four write sites, and migration v46 backfills pre-v0.32 rows to the same grid. Synthesize auto-enables when a corpus dir is configured, so the silent footgun ("I set the dir, why is nothing happening?") is gone. `recall` and `forget` actually route now (v0.31 added them to `handleCliOnly()` but forgot the `CLI_ONLY` gate set). And the `gbrain auth` regression that crashed every OAuth subcommand on Postgres with a misleading `connect() has not been called` error is fixed.
|
||||
|
||||
This release is also the v0.31.4.1 dot-suffix slot we needed because the previous merge subject claimed `v0.31.4` without bumping VERSION or `package.json`. Going forward, gbrain versions adopt `MAJOR.MINOR.PATCH.MICRO` so dot-suffix follow-ups land cleanly without churning the patch number.
|
||||
|
||||
### What you can now do
|
||||
|
||||
**`gbrain eval takes-quality`** ships as a real CLI surface. `run` scores a sample with three frontier models against the 5-dim rubric. `replay` re-verifies a receipt without the brain (CI-friendly, no `DATABASE_URL` needed). `trend` shows quality over time segregated by `rubric_version`. `regress --against <receipt> --threshold T` exits 1 when any dim drops past the threshold, so you can gate a PR on "is the new prompt actually better, or am I overfitting to one example?". Receipts persist DB-authoritative in `eval_takes_quality_runs` (migration v47); the disk artifact is best-effort. The 4-sha receipt name (corpus + prompt + models + rubric) means a rubric tweak segregates trend rows instead of silently corrupting the graph.
|
||||
|
||||
**`gbrain doctor` warns when takes weights drift off the 0.05 grid.** Post-migration drift detector. More than 10% off-grid fails with an `apply-migrations` fix hint, 1-10% warns, less than 1% is ok. Tolerance comparison (abs > 1e-3) avoids float32 representation noise so a healthy brain doesn't look broken.
|
||||
|
||||
**Bad takes holders surface as sync failures instead of silent quality drag.** Strings like `Garry`, `people/Garry-Tan`, `world/garry-tan`, whitespace-only, embedded slashes all bucket into the new `TAKES_HOLDER_INVALID` code in `~/.gbrain/sync-failures.jsonl` and show up in `gbrain doctor`. Legacy bare-slug holders (`garry`, `alice`) are preserved as v0.32 transition compat. Dotted and underscore slugs (`companies/acme.io`, `people/foo_bar`) stay valid because `SLUG_SEGMENT_PATTERN` is now a single source of truth shared with sync.
|
||||
|
||||
**`gbrain auth register-client` works on Postgres.** Pre-fix every OAuth subcommand crashed with `Error: No database connection: connect() has not been called.` because `withConfiguredSql` created a `PostgresEngine` but never called `engine.connect()`. The error pointed users at `gbrain init` and made the regression invisible. The fix unblocks three previously-failing E2E suites (`serve-http-oauth.test.ts` 0/28 → 28/28, `sources-remote-mcp.test.ts` 0/14 → 14/14, `thin-client.test.ts` 0/7 → 6/7 + 1 documented skip).
|
||||
|
||||
### Numbers that matter
|
||||
|
||||
Cross-modal eval over the first full production extraction, 100,720 takes scored on a 5-dim rubric with GPT-5.5 and Opus 4.6 as judges. The "fix shipped" column is what v0.31.4.1 does about each dimension:
|
||||
|
||||
| Dimension | GPT-5.5 | Opus 4.6 | Avg | Fix shipped |
|
||||
|---|---|---|---|---|
|
||||
| Accuracy | 7 | 8 | 7.5 | `docs/takes-vs-facts.md` architectural split |
|
||||
| Attribution | 6 | 7 | 6.5 | Holder grammar enforced at parser |
|
||||
| Weight calibration | 7 | 7 | 7.0 | 0.05 grid on insert + v46 backfill |
|
||||
| Kind classification | 6 | 7 | 6.5 | Filing-rules anchor + JSDoc examples |
|
||||
| Signal density | 7 | 6 | 6.5 | Synthesize auto-enables on corpus dir |
|
||||
|
||||
Attribution at 6.5/10 was the dominant failure mode, and the eval flagged holder/subject confusion specifically (the model writing a fact as `holder=people/Founder-Name` when the right answer is "this person holds this belief"). v0.31.4.1 makes that a sync-failure record an operator can see instead of a silent calibration drag.
|
||||
|
||||
### What this means for you
|
||||
|
||||
If you run takes extraction in production, upgrade. The runtime fixes (weight rounding, holder validation, synthesize auto-enable) close real footgun classes, and v46 + v47 bring existing data and schema to the same bar without manual work. Run `gbrain upgrade && gbrain doctor` and watch the new `takes_weight_grid` slice confirm the backfill landed. Then capture a baseline with `gbrain eval takes-quality run` so the next prompt change has something to regress against.
|
||||
|
||||
## To take advantage of v0.31.4.1
|
||||
|
||||
`gbrain upgrade` should do this automatically. If it didn't, or if `gbrain doctor` warns about a partial migration:
|
||||
|
||||
1. **Run the orchestrator manually:**
|
||||
```bash
|
||||
gbrain apply-migrations --yes
|
||||
```
|
||||
2. **Verify the outcome:**
|
||||
```bash
|
||||
gbrain doctor # takes_weight_grid + sync_failures slices
|
||||
gbrain eval takes-quality run --limit 20 --cycles 1 # capture a baseline receipt
|
||||
```
|
||||
3. **If any step fails or the numbers look wrong,** please file an issue:
|
||||
https://github.com/garrytan/gbrain/issues with:
|
||||
- output of `gbrain doctor`
|
||||
- contents of `~/.gbrain/upgrade-errors.jsonl` if it exists
|
||||
- which step broke
|
||||
|
||||
This feedback loop is how the gbrain maintainers find fragile upgrade paths. Thank you.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
#### Takes v2 runtime
|
||||
- Holder grammar enforced at parse time. `parseTakesFence()` emits `TAKES_HOLDER_INVALID` warnings on non-matching holders; the row is preserved (markdown source-of-truth contract). `HOLDER_REGEX` wraps the shared `SLUG_SEGMENT_PATTERN` exported from `src/core/sync.ts` so it accepts every legitimate slug `slugifySegment()` produces.
|
||||
- Producer seam: `extractTakesFromDb` and `extractTakesFromFs` populate `ExtractTakesResult.failedFiles[]` from the parser warnings. Migration v0.28.0's `phaseBBackfill` calls `recordSyncFailures(result.failedFiles, 'migration:v0.28.0-backfill')` so doctor's `sync_failures` slice surfaces post-upgrade drift.
|
||||
- `classifyErrorCode` extends with `TAKES_HOLDER_INVALID`, `TAKES_TABLE_MALFORMED`, `TAKES_ROW_NUM_COLLISION`, `TAKES_FENCE_UNBALANCED`. Previously these warnings bucketed to `UNKNOWN`.
|
||||
- `normalizeWeightForStorage()` extracted to `src/core/takes-fence.ts` as the single source of truth for all four takes write sites: `addTakesBatch` + `updateTake` on PGLite + Postgres. Guards `!Number.isFinite()` BEFORE the [0,1] clamp so NaN no longer survives to the DB. Also rounds to 0.05 grid; preserves 0.0 and 1.0 exactly.
|
||||
- Migration v46 (`takes_weight_round_to_grid`) backfills pre-v0.32 rows to the 0.05 grid. Tolerance comparison (`abs > 1e-3`) avoids the float32 representation-noise re-touch loop. Runs with `transaction: false` so the migration runner doesn't hold a long-lived transaction blocking other gbrain processes.
|
||||
|
||||
#### Takes v2 documentation
|
||||
- `docs/takes-vs-facts.md` documents the two epistemological layers, why they must never be conflated, how the dream cycle bridges them, and the production extraction data (model selection, eval dimensions, key learnings for extraction prompts).
|
||||
- `skills/_brain-filing-rules.md` gains a "Takes attribution" section distilling the 6 rules into a terse contract for downstream agents.
|
||||
- `src/core/takes-fence.ts` JSDoc expanded with concrete right/wrong holder examples from the cross-modal eval (amplification ≠ endorsement, self-reported ≠ verified, founder describing company → `people/founder` not `companies/slug`).
|
||||
|
||||
#### Takes v2 fixes
|
||||
- `recall` and `forget` added to the `CLI_ONLY` gate set. v0.31 added them to `handleCliOnly()` but the gate set was missed, so both fell through to `cliOps.get()` → `'Unknown command'`.
|
||||
- `synthesize` auto-enables when `session_corpus_dir` is configured. Explicit `enabled=false` still wins.
|
||||
|
||||
#### Takes-quality eval CLI
|
||||
- `gbrain eval takes-quality {run,replay,trend,regress}` (`src/commands/eval-takes-quality.ts` + `src/core/takes-quality-eval/`).
|
||||
- Migration v47 creates `eval_takes_quality_runs`. DB-authoritative receipt persistence (full receipt JSON in a JSONB column) with a 4-sha UNIQUE constraint that supports `ON CONFLICT DO NOTHING` idempotency and rubric-version segregation.
|
||||
- `replay` is the only sub-subcommand that doesn't need a brain. Routes through `cli.ts`'s no-DB bypass directly to `runReplayNoBrain`, exits 0/1/2 cleanly without ever touching the engine.
|
||||
- Pricing fail-closed: any model not in `pricing.ts` produces a `PricingNotFoundError` BEFORE any HTTP call fires (only when `--budget-usd` is set; null budget skips the gate).
|
||||
- Aggregate requires all 5 declared rubric dimensions per model. Missing-dim drops the contribution, treated identically to a parse failure. Empty-scores PASS regression guard preserved.
|
||||
- `parseModelJSON` + `ParsedScore` + `ParsedModelResult` types hoisted from `cross-modal-eval/json-repair.ts` to `eval-shared/json-repair.ts`. The original module re-exports both function and types for compatibility.
|
||||
|
||||
#### Doctor
|
||||
- New `takes_weight_grid` slice. Pure exported `takesWeightGridCheck(engine: BrainEngine)` so tests can target the helper directly with stubbed engines and against real PGLite for the four ratio bands. Branches: 0 takes → ok, >10% off-grid → fail with `apply-migrations` fix hint, 1-10% → warn, ≤1% → ok, missing table (pre-v37) → graceful skip.
|
||||
|
||||
#### Auth (Postgres regression fix)
|
||||
- `withConfiguredSql` in `src/commands/auth.ts` now calls `engine.connect()` before invoking the SQL adapter. Pre-fix every `gbrain auth` subcommand on Postgres crashed because `PostgresEngine.sql` fell back to the module-level singleton (`db.getConnection()`) when its instance `_sql` was unset, and `db.connect()` wasn't called either. Anyone with a Postgres-backed brain hit this. Verified with `gbrain auth register-client` round-trip: before, `Error: No database connection`; after, credentials printed.
|
||||
|
||||
#### Tests
|
||||
- 200+ new cases. Highlights: takes-quality boundaries / runner / receipt-write, takes-holder-validation (26 cases including codex-review-#3 dotted-slug positives and HOLDER_REGEX anchoring guard), engine-weight-rounding integration (15 cases mirroring the real-Postgres E2E), v46 + v47 structural assertions, real-Postgres E2E for `eval_takes_quality_runs` (8 cases including the postgres.js JSONB encoding round-trip), real-Postgres takes-weight-rounding write-path (6 cases covering the `unnest()` bind shape PGLite doesn't exercise), extract-takes producer seam (7 cases).
|
||||
- `withEnv()` adoption: `test/eval-takes-quality-receipt-write.test.ts` refactored from direct `process.env.GBRAIN_HOME = ...` mutation to the helper, complying with the v0.32.0 test-isolation lint contract (`scripts/check-test-isolation.sh` R1).
|
||||
- `test/eval-takes-quality-runner.serial.test.ts` quarantined as `*.serial.test.ts` because `mock.module(...)` leaks across files in the same shard process (R2 in the lint contract).
|
||||
|
||||
#### Versioning convention
|
||||
- gbrain versions move to `MAJOR.MINOR.PATCH.MICRO`. The `.MICRO` slot lets dot-suffix follow-ups land without churning the patch number when a release like #795 ships its commit subject ahead of its VERSION bump. Existing 3-level versions remain valid in history.
|
||||
|
||||
## [0.31.3] - 2026-05-09
|
||||
|
||||
**`gbrain serve` stops leaking the PGLite write lock when the parent disconnects, and `gbrain auth` + `gbrain serve --http` work against PGLite brains.**
|
||||
|
||||
@@ -413,7 +413,7 @@ Five tiers of test commands, each with a clear scope:
|
||||
|
||||
### CI vs local: intentionally divergent file sets
|
||||
|
||||
- **CI matrix** (`.github/workflows/test.yml`) runs `scripts/test-shard.sh` 4-way, which uses FNV-1a hash bucketing and INCLUDES `*.slow.test.ts`. CI is the ground truth for "did everything pass."
|
||||
- **CI matrix** (`.github/workflows/test.yml`) runs `scripts/test-shard.sh` 4-way, which uses FNV-1a hash bucketing and INCLUDES `*.slow.test.ts`. As of v0.31.4.1, CI EXCLUDES `*.serial.test.ts` from the hash buckets and runs them on shard 1 via `bun run test:serial` at `--max-concurrency=1`. Before that, serial files were hashed in alongside parallel files, which broke the `mock.module` quarantine (top-level mocks in serial files leaked into the parallel files they shared a shard process with — most visibly, `eval-takes-quality-runner.serial.test.ts` stubbed `gateway.ts` and broke every `gateway.embedMultimodal` test in `voyage-multimodal.test.ts` on shard 2). CI is the ground truth for "did everything pass."
|
||||
- **Local fast loop** (`scripts/run-unit-shard.sh` via the parallel wrapper) uses round-robin-by-index sharding and EXCLUDES `*.slow.test.ts` AND `*.serial.test.ts`. Local trades coverage for inner-loop speed; CI catches what local skips.
|
||||
|
||||
This divergence is intentional. Don't try to make them equal — the two scripts deliberately solve different problems. The regression test at `test/scripts/run-unit-shard.test.ts` pins what the local fast loop should and shouldn't include.
|
||||
@@ -775,15 +775,26 @@ sync. `/ship` enforces this via Step 12's idempotency check (VERSION vs
|
||||
package.json drift), but the canonical list lives here so future runs and
|
||||
the auto-update agent know where to look.
|
||||
|
||||
**Version format is mandatory: `MAJOR.MINOR.PATCH.MICRO` (four numeric
|
||||
segments, dot-separated, no leading `v`).** Every new release MUST use the
|
||||
4-segment form. The `.MICRO` slot is the dot-suffix follow-up channel: when
|
||||
a release ships its commit subject ahead of its VERSION bump (e.g. PR #795
|
||||
landing as `v0.31.4` without bumping the file), the corrective ship lands
|
||||
as `0.31.4.1` rather than churning the patch number to `0.31.5`. Suffixes
|
||||
like `-fixwave` are still allowed as needed (`0.31.1.1-fixwave`), but the
|
||||
four numeric segments are required first. Historical 3-segment versions
|
||||
(`0.31.3`, `0.22.1`) remain valid in `git log` and migration filenames
|
||||
(`skills/migrations/v0.21.0.md`); do NOT rewrite them. Going forward only.
|
||||
|
||||
**Required (every release must update all five):**
|
||||
|
||||
| File | What lives there | Format |
|
||||
|---|---|---|
|
||||
| `VERSION` | The single source of truth. Read first by `/ship`, the binary, and CI version-gate. | Bare 4-digit string `MAJOR.MINOR.PATCH.MICRO` (e.g. `0.22.1`), no leading `v`, no trailing newline-sensitivity issues. |
|
||||
| `package.json` | Bun/npm package version. `gbrain --version` reads it via the compiled binary's bundled package metadata. CI version-gate cross-checks this against `VERSION` and fails if they drift. | `"version": "0.22.1"` |
|
||||
| `CHANGELOG.md` | Top entry header `## [0.22.1] - YYYY-MM-DD` plus the "To take advantage of v0.22.1" block. | Standard Keep-a-Changelog header. |
|
||||
| `TODOS.md` | Any TODO entries that mention "follow-up from vX.Y.Z" use the version of the release that filed them. Update only when filing NEW follow-up TODOs. | Inline `vX.Y.Z` references in TODO bodies. |
|
||||
| `CLAUDE.md` | The Key Files section's per-file annotations carry `vX.Y.Z (#NNN)` tags noting which release introduced a behavior. Update whenever a wave's annotations get folded in. | Inline `vX.Y.Z (#NNN, contributed by @user)` references. |
|
||||
| `VERSION` | The single source of truth. Read first by `/ship`, the binary, and CI version-gate. | Bare 4-segment string `MAJOR.MINOR.PATCH.MICRO` (e.g. `0.31.4.1`), no leading `v`. |
|
||||
| `package.json` | Bun/npm package version. `gbrain --version` reads it via the compiled binary's bundled package metadata. CI version-gate cross-checks this against `VERSION` and fails if they drift. | `"version": "0.31.4.1"` |
|
||||
| `CHANGELOG.md` | Top entry header `## [0.31.4.1] - YYYY-MM-DD` plus the "To take advantage of v0.31.4.1" block. | Standard Keep-a-Changelog header. |
|
||||
| `TODOS.md` | Any TODO entries that mention "follow-up from vX.Y.Z.W" use the version of the release that filed them. Update only when filing NEW follow-up TODOs. | Inline `vX.Y.Z.W` references in TODO bodies. |
|
||||
| `CLAUDE.md` | The Key Files section's per-file annotations carry `vX.Y.Z.W (#NNN)` tags noting which release introduced a behavior. Update whenever a wave's annotations get folded in. | Inline `vX.Y.Z.W (#NNN, contributed by @user)` references. |
|
||||
|
||||
**Auto-derived (no manual edit; refreshed by their own commands):**
|
||||
|
||||
|
||||
+17
-6
@@ -513,7 +513,7 @@ Five tiers of test commands, each with a clear scope:
|
||||
|
||||
### CI vs local: intentionally divergent file sets
|
||||
|
||||
- **CI matrix** (`.github/workflows/test.yml`) runs `scripts/test-shard.sh` 4-way, which uses FNV-1a hash bucketing and INCLUDES `*.slow.test.ts`. CI is the ground truth for "did everything pass."
|
||||
- **CI matrix** (`.github/workflows/test.yml`) runs `scripts/test-shard.sh` 4-way, which uses FNV-1a hash bucketing and INCLUDES `*.slow.test.ts`. As of v0.31.4.1, CI EXCLUDES `*.serial.test.ts` from the hash buckets and runs them on shard 1 via `bun run test:serial` at `--max-concurrency=1`. Before that, serial files were hashed in alongside parallel files, which broke the `mock.module` quarantine (top-level mocks in serial files leaked into the parallel files they shared a shard process with — most visibly, `eval-takes-quality-runner.serial.test.ts` stubbed `gateway.ts` and broke every `gateway.embedMultimodal` test in `voyage-multimodal.test.ts` on shard 2). CI is the ground truth for "did everything pass."
|
||||
- **Local fast loop** (`scripts/run-unit-shard.sh` via the parallel wrapper) uses round-robin-by-index sharding and EXCLUDES `*.slow.test.ts` AND `*.serial.test.ts`. Local trades coverage for inner-loop speed; CI catches what local skips.
|
||||
|
||||
This divergence is intentional. Don't try to make them equal — the two scripts deliberately solve different problems. The regression test at `test/scripts/run-unit-shard.test.ts` pins what the local fast loop should and shouldn't include.
|
||||
@@ -875,15 +875,26 @@ sync. `/ship` enforces this via Step 12's idempotency check (VERSION vs
|
||||
package.json drift), but the canonical list lives here so future runs and
|
||||
the auto-update agent know where to look.
|
||||
|
||||
**Version format is mandatory: `MAJOR.MINOR.PATCH.MICRO` (four numeric
|
||||
segments, dot-separated, no leading `v`).** Every new release MUST use the
|
||||
4-segment form. The `.MICRO` slot is the dot-suffix follow-up channel: when
|
||||
a release ships its commit subject ahead of its VERSION bump (e.g. PR #795
|
||||
landing as `v0.31.4` without bumping the file), the corrective ship lands
|
||||
as `0.31.4.1` rather than churning the patch number to `0.31.5`. Suffixes
|
||||
like `-fixwave` are still allowed as needed (`0.31.1.1-fixwave`), but the
|
||||
four numeric segments are required first. Historical 3-segment versions
|
||||
(`0.31.3`, `0.22.1`) remain valid in `git log` and migration filenames
|
||||
(`skills/migrations/v0.21.0.md`); do NOT rewrite them. Going forward only.
|
||||
|
||||
**Required (every release must update all five):**
|
||||
|
||||
| File | What lives there | Format |
|
||||
|---|---|---|
|
||||
| `VERSION` | The single source of truth. Read first by `/ship`, the binary, and CI version-gate. | Bare 4-digit string `MAJOR.MINOR.PATCH.MICRO` (e.g. `0.22.1`), no leading `v`, no trailing newline-sensitivity issues. |
|
||||
| `package.json` | Bun/npm package version. `gbrain --version` reads it via the compiled binary's bundled package metadata. CI version-gate cross-checks this against `VERSION` and fails if they drift. | `"version": "0.22.1"` |
|
||||
| `CHANGELOG.md` | Top entry header `## [0.22.1] - YYYY-MM-DD` plus the "To take advantage of v0.22.1" block. | Standard Keep-a-Changelog header. |
|
||||
| `TODOS.md` | Any TODO entries that mention "follow-up from vX.Y.Z" use the version of the release that filed them. Update only when filing NEW follow-up TODOs. | Inline `vX.Y.Z` references in TODO bodies. |
|
||||
| `CLAUDE.md` | The Key Files section's per-file annotations carry `vX.Y.Z (#NNN)` tags noting which release introduced a behavior. Update whenever a wave's annotations get folded in. | Inline `vX.Y.Z (#NNN, contributed by @user)` references. |
|
||||
| `VERSION` | The single source of truth. Read first by `/ship`, the binary, and CI version-gate. | Bare 4-segment string `MAJOR.MINOR.PATCH.MICRO` (e.g. `0.31.4.1`), no leading `v`. |
|
||||
| `package.json` | Bun/npm package version. `gbrain --version` reads it via the compiled binary's bundled package metadata. CI version-gate cross-checks this against `VERSION` and fails if they drift. | `"version": "0.31.4.1"` |
|
||||
| `CHANGELOG.md` | Top entry header `## [0.31.4.1] - YYYY-MM-DD` plus the "To take advantage of v0.31.4.1" block. | Standard Keep-a-Changelog header. |
|
||||
| `TODOS.md` | Any TODO entries that mention "follow-up from vX.Y.Z.W" use the version of the release that filed them. Update only when filing NEW follow-up TODOs. | Inline `vX.Y.Z.W` references in TODO bodies. |
|
||||
| `CLAUDE.md` | The Key Files section's per-file annotations carry `vX.Y.Z.W (#NNN)` tags noting which release introduced a behavior. Update whenever a wave's annotations get folded in. | Inline `vX.Y.Z.W (#NNN, contributed by @user)` references. |
|
||||
|
||||
**Auto-derived (no manual edit; refreshed by their own commands):**
|
||||
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gbrain",
|
||||
"version": "0.31.3",
|
||||
"version": "0.31.4.1",
|
||||
"description": "Postgres-native personal knowledge brain with hybrid RAG search",
|
||||
"type": "module",
|
||||
"main": "src/core/index.ts",
|
||||
|
||||
+25
-4
@@ -13,8 +13,14 @@
|
||||
# exist, so retries are reproducible. Hash is FNV-1a — pure shell, no jq.
|
||||
set -euo pipefail
|
||||
|
||||
DRY_RUN_LIST=0
|
||||
if [ "${1:-}" = "--dry-run-list" ]; then
|
||||
DRY_RUN_LIST=1
|
||||
shift
|
||||
fi
|
||||
|
||||
if [ "$#" -ne 2 ]; then
|
||||
echo "usage: scripts/test-shard.sh <shard-index> <total-shards>" >&2
|
||||
echo "usage: scripts/test-shard.sh [--dry-run-list] <shard-index> <total-shards>" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -32,12 +38,19 @@ fi
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
# Find all unit test files, deterministic order. Excludes test/e2e/.
|
||||
# Portable: avoid `mapfile` (bash 4+) so this runs on macOS bash 3.2 too.
|
||||
# Find all unit test files, deterministic order. Excludes test/e2e/ and
|
||||
# *.serial.test.ts. Serial files share file-wide state (top-level
|
||||
# mock.module, module singletons) that leaks across files in the same
|
||||
# `bun test` shard process — see scripts/check-test-isolation.sh R2.
|
||||
# CI runs them via `bun run test:serial` (scripts/run-serial-tests.sh) at
|
||||
# --max-concurrency=1 in a separate step on shard 1. Local `bun run test`
|
||||
# already excludes them from the parallel pass and runs them after the
|
||||
# same way. Portable: avoid `mapfile` (bash 4+) so this runs on macOS
|
||||
# bash 3.2 too.
|
||||
FILES=()
|
||||
while IFS= read -r line; do
|
||||
FILES+=("$line")
|
||||
done < <(find test -name '*.test.ts' -not -path 'test/e2e/*' | sort)
|
||||
done < <(find test -name '*.test.ts' -not -name '*.serial.test.ts' -not -path 'test/e2e/*' | sort)
|
||||
|
||||
if [ "${#FILES[@]}" -eq 0 ]; then
|
||||
echo "no test files found under test/" >&2
|
||||
@@ -67,6 +80,14 @@ for f in "${FILES[@]}"; do
|
||||
fi
|
||||
done
|
||||
|
||||
if [ "$DRY_RUN_LIST" = "1" ]; then
|
||||
if [ "${#SHARD_FILES[@]}" -eq 0 ]; then
|
||||
exit 0
|
||||
fi
|
||||
printf '%s\n' "${SHARD_FILES[@]}"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "shard $SHARD_INDEX/$TOTAL_SHARDS: ${#SHARD_FILES[@]}/${#FILES[@]} files"
|
||||
if [ "${#SHARD_FILES[@]}" -eq 0 ]; then
|
||||
echo "warning: shard $SHARD_INDEX has no files (rehash or reduce shard count)" >&2
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
/**
|
||||
* Regression test: scripts/test-shard.sh exclusion symmetry.
|
||||
*
|
||||
* Pins the contract that CI's hash-bucketed shard script EXCLUDES
|
||||
* *.serial.test.ts and test/e2e/ from every shard. Serial files share
|
||||
* file-wide state (top-level mock.module, module singletons) that leaks
|
||||
* across files in the same `bun test` shard process. Before v0.31.4.1
|
||||
* they were hashed into the same buckets as parallel files, which broke
|
||||
* the quarantine — `eval-takes-quality-runner.serial.test.ts` stubbed
|
||||
* `gateway.ts` and broke every `gateway.embedMultimodal` test in
|
||||
* `voyage-multimodal.test.ts` on shard 2.
|
||||
*
|
||||
* Without this guard, a future refactor that drops the `-not -name
|
||||
* '*.serial.test.ts'` clause from test-shard.sh would silently undo the
|
||||
* fix and re-introduce the contention flake.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeAll } from 'bun:test';
|
||||
import { execFileSync } from 'child_process';
|
||||
import { resolve } from 'path';
|
||||
|
||||
const REPO_ROOT = resolve(import.meta.dir, '..', '..');
|
||||
const SHARD_SH = resolve(REPO_ROOT, 'scripts/test-shard.sh');
|
||||
|
||||
// Pure-bash FNV-1a per shard takes ~4s on a M-series Mac because the script
|
||||
// computes a hash for every test file. Compute all 4 shards once in beforeAll
|
||||
// and reuse across cases so the suite finishes in one shell-out per shard.
|
||||
const shardCache: Record<number, string[]> = {};
|
||||
|
||||
function dryRunList(shard: number, total: number): string[] {
|
||||
if (shardCache[shard]) return shardCache[shard];
|
||||
const out = execFileSync('bash', [SHARD_SH, '--dry-run-list', String(shard), String(total)], {
|
||||
cwd: REPO_ROOT,
|
||||
encoding: 'utf-8',
|
||||
});
|
||||
shardCache[shard] = out.split('\n').map(s => s.trim()).filter(Boolean);
|
||||
return shardCache[shard];
|
||||
}
|
||||
|
||||
describe('test-shard.sh exclusion symmetry', () => {
|
||||
beforeAll(() => {
|
||||
for (const shard of [1, 2, 3, 4]) dryRunList(shard, 4);
|
||||
}, 60_000);
|
||||
it('includes plain *.test.ts files in at least one shard', () => {
|
||||
const allFiles = [1, 2, 3, 4].flatMap(s => dryRunList(s, 4));
|
||||
expect(allFiles.length).toBeGreaterThan(0);
|
||||
expect(allFiles.some(f => /\.test\.ts$/.test(f) && !/\.serial\.test\.ts$/.test(f))).toBe(true);
|
||||
});
|
||||
|
||||
it('excludes every *.serial.test.ts file from every shard', () => {
|
||||
for (const shard of [1, 2, 3, 4]) {
|
||||
const files = dryRunList(shard, 4);
|
||||
const leaks = files.filter(f => /\.serial\.test\.ts$/.test(f));
|
||||
expect(leaks, `shard ${shard} contains serial files`).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
it('excludes the test/e2e/ subtree from every shard', () => {
|
||||
for (const shard of [1, 2, 3, 4]) {
|
||||
const files = dryRunList(shard, 4);
|
||||
const leaks = files.filter(f => f.startsWith('test/e2e/'));
|
||||
expect(leaks, `shard ${shard} contains e2e files`).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
it('partitions plain files across shards without overlap', () => {
|
||||
const seen = new Map<string, number>();
|
||||
for (const shard of [1, 2, 3, 4]) {
|
||||
for (const f of dryRunList(shard, 4)) {
|
||||
if (seen.has(f)) {
|
||||
throw new Error(`file ${f} appears in shard ${seen.get(f)} AND shard ${shard}`);
|
||||
}
|
||||
seen.set(f, shard);
|
||||
}
|
||||
}
|
||||
expect(seen.size).toBeGreaterThan(0);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user