fix(wave): v0.15.1 - 4 hot issues + scope expansion (#248)

* fix(wave): 4 hot issues + 3 scope expansions (v0.13.1)

Addresses four user-filed regressions after v0.13.0 plus three adjacent
footgun closures.

* #170 — CREATE INDEX [CONCURRENTLY] IF NOT EXISTS idx_pages_updated_at_desc
  on pages (updated_at DESC). Engine-aware migration v12 with invalid-index
  cleanup on Postgres, plain CREATE on PGLite. ~700x on 30k+ row brains.
  Contributed by @fuleinist (#215).

* #219 — Minions schema default max_stalled 1 -> 5. v13 migration ALTERs
  the default and UPDATEs existing non-terminal rows (waiting/active/
  delayed/waiting-children/paused) so live queues get rescued on upgrade.
  Adds MinionJobInput.max_stalled with [1,100] clamp. New --max-stalled
  CLI flag on `jobs submit`. Reported by @macbotmini-eng.

* #218 — package.json postinstall surfaces errors instead of silencing.
  trustedDependencies whitelists @electric-sql/pglite. doctor
  schema_version check fails loudly when migrations never ran and links
  to #218. README + INSTALL_FOR_AGENTS warn against `bun install -g`.
  Reported by @gopalpatel.

* #223 — @electric-sql/pglite pinned to exactly 0.4.3 (was ^0.4.4).
  PGLiteEngine.connect() wraps PGlite.create() errors with a message
  pointing at the issue + gbrain doctor. Does NOT suggest 'missing
  migrations' as a cause (create-time abort happens before migrations
  run). Pin is unverified against macOS 26.3; error-wrap is the safety
  net. Reported by @AndreLYL.

* Scope: `gbrain jobs submit` gains --backoff-type/--backoff-delay/
  --backoff-jitter/--timeout-ms/--idempotency-key (MinionJobInput audit).
* Scope: `gbrain jobs smoke --sigkill-rescue` regression case (opt-in,
  CI-only) that simulates a killed worker and asserts the new default
  rescues.
* Scope: `gbrain doctor --index-audit` reports zero-scan Postgres indexes
  as drop candidates (informational; no auto-drop).

Infrastructure:
* Migration interface extended with sqlFor: { postgres?, pglite? } and
  transaction: boolean. Runner picks the engine-specific branch and
  bypasses engine.transaction() when transaction:false (required for
  CONCURRENTLY). BrainEngine.kind readonly discriminator added.
* scripts/check-jsonb-pattern.sh CI guard extended to block
  `max_stalled DEFAULT 1` from regressing.

Tests:
* 15 new unit tests: v12/v13 structural + behavioral assertions,
  max_stalled default/clamp/backfill, PGLite error-wrap source guard,
  engine kind discriminator.
* 3 regression tests pinned by IRON RULE.
* Full unit suite: 1416 pass.
* Full E2E suite against Postgres 16 + pgvector: 126 pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.13.1)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: sync documentation for v0.13.1

CLAUDE.md "Key files" and "Commands" sections refreshed to match the
v0.13.1 fix wave:

- Note `BrainEngine.kind` discriminator on engine.ts
- Document v0.13.1 connect() error-wrap on pglite-engine.ts
- Refresh src/core/minions/ layout (no shell handler, no protected-names,
  no quiet-hours/stagger — that was v0.13-development scaffolding that
  did not ship)
- Add src/core/migrate.ts entry with `Migration` interface extensions
  (`sqlFor`, `transaction: false`)
- Document new `gbrain jobs submit` flags (--max-stalled, --backoff-type,
  --backoff-delay, --backoff-jitter, --timeout-ms, --idempotency-key)
- Document `gbrain jobs smoke --sigkill-rescue` regression guard
- Document `gbrain doctor --index-audit` and the schema_version=0
  surface that catches #218 postinstall failures
- Extend check-jsonb-pattern.sh note with the max_stalled DEFAULT 1
  regression guard
- Touch up test file blurbs for migrate.test.ts, pglite-engine.test.ts,
  minions.test.ts with v0.13.1 coverage

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(e2e): run files sequentially to eliminate shared-DB race

The E2E suite was flaky. ~3 of every 5 runs had 4-10 failures clustered
in Links, Timeline, Versions, Minions resilience, Parallel Import, and
Page CRUD tests. Symptoms included "expected 16 pages, got 8" (half),
"expected 1 link inserted, got 0", timeline entries missing after
round-trip, and similar data-shape mismatches.

Root cause: bun test runs test FILES in parallel (each in a worker
process). 13 E2E files share one DATABASE_URL, and `setupDB()` in
`test/e2e/helpers.ts` does `TRUNCATE ... CASCADE` on all tables before
each file's `importFixtures()`. File A's TRUNCATE would race with file
B's in-flight INSERT stream, producing the observed half-populated or
wrong-count states.

An earlier attempt used a Postgres advisory lock held on a dedicated
single-connection client for the lifetime of each file's run. It broke
because bun's default 5000 ms hook timeout fires on queued beforeAll()
calls: with 13 files serializing through the lock, files 2-13 would
time out waiting for file 1 to finish.

This commit switches to sequential file execution at the harness level
via scripts/run-e2e.sh, which loops through test/e2e/*.test.ts one at
a time, tracks aggregate pass/fail counts, and exits non-zero on the
first failing file. No lock, no timeout issues, no changes to any test
file. package.json test:e2e points at the new script.

Verified: 5 back-to-back runs against the same Postgres container,
each completing in ~5 min. Every run: 13 files, 138 tests, 0 fails.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump version to 0.15.1 (fix wave locked to MINOR line)

Master v0.14.2 was the last /investigate root-cause wave on the
v0.14.x line. This fix wave opens v0.15.x: four hot issues (#170,
#218, #219, #223) close v0.13.x regressions that v0.14.x didn't
cover, so the MINOR bump reflects the semantic shift — new schema
migrations (v14, v15), a new CLI surface (`--max-stalled`,
`--sigkill-rescue`, `--index-audit`), a new BrainEngine contract
(`kind` discriminator + extended `Migration` interface), and a new
install-time contract (PGLite 0.4.3 pin + `trustedDependencies`).

Locked to 0.15.1 in advance: other work may land before/after this
PR, but the version is fixed so reviewers can cite a stable number.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-04-21 13:19:23 -07:00
committed by GitHub
co-authored by Claude Opus 4.7
parent 7f156c8873
commit ff10796a00
25 changed files with 797 additions and 94 deletions
+106
View File
@@ -79,6 +79,112 @@ describe('migrations v8 + v9 — structural guard for helper-index fix', () => {
});
});
// v0.14.1 — fix wave structural assertions (migrations renumbered from v12/v13 to
// v14/v15 after master merged budget_ledger (v12) + minion_quiet_hours_stagger (v13)).
describe('migrate v14 — pages_updated_at_index (handler-based, engine-aware)', () => {
const v14 = MIGRATIONS.find(m => m.version === 14);
test('v14 exists and uses a handler (not pure SQL) for engine-aware branching', () => {
expect(v14).toBeDefined();
expect(v14!.name).toBe('pages_updated_at_index');
expect(typeof v14!.handler).toBe('function');
expect(v14!.sql).toBe('');
});
test('v14 handler source contains CONCURRENTLY + invalid-index cleanup for Postgres branch', async () => {
const { readFileSync } = await import('fs');
const src = readFileSync('src/core/migrate.ts', 'utf-8');
const v14Start = src.indexOf("name: 'pages_updated_at_index'");
expect(v14Start).toBeGreaterThan(-1);
const v14Block = src.slice(v14Start, v14Start + 3000);
expect(v14Block).toContain('pg_index');
expect(v14Block).toContain('indisvalid');
expect(v14Block).toContain('DROP INDEX CONCURRENTLY IF EXISTS idx_pages_updated_at_desc');
expect(v14Block).toContain('CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_pages_updated_at_desc');
// Order within the handler body: DROP IF EXISTS must precede CREATE IF NOT EXISTS,
// so a failed prior CONCURRENTLY build is cleaned before re-create. Anchor on the
// explicit "IF EXISTS" / "IF NOT EXISTS" phrases so the header doc-comment
// (which mentions both unqualified) doesn't fool the ordering assertion.
const dropIdx = v14Block.indexOf('DROP INDEX CONCURRENTLY IF EXISTS');
const createIdx = v14Block.indexOf('CREATE INDEX CONCURRENTLY IF NOT EXISTS');
expect(dropIdx).toBeLessThan(createIdx);
expect(v14Block).toContain('engine.kind');
});
});
describe('migrate v15 — minion_jobs_max_stalled_default_5', () => {
const v15 = MIGRATIONS.find(m => m.version === 15);
test('v15 exists and alters max_stalled default to 5', () => {
expect(v15).toBeDefined();
expect(v15!.name).toBe('minion_jobs_max_stalled_default_5');
expect(v15!.sql).toContain('ALTER TABLE minion_jobs ALTER COLUMN max_stalled SET DEFAULT 5');
});
test('v15 backfill UPDATE targets the correct non-terminal statuses', () => {
const sql = v15!.sql;
expect(sql).toContain(`'waiting'`);
expect(sql).toContain(`'active'`);
expect(sql).toContain(`'delayed'`);
expect(sql).toContain(`'waiting-children'`);
expect(sql).toContain(`'paused'`);
expect(sql).not.toContain(`'completed'`);
expect(sql).not.toContain(`'dead'`);
expect(sql).not.toContain(`'cancelled'`);
expect(sql).not.toContain(`'claimed'`);
expect(sql).not.toContain(`'running'`);
expect(sql).not.toContain(`'stalled'`);
});
test('v15 UPDATE clause has the < 5 guard so idempotent re-runs are no-ops', () => {
expect(v15!.sql).toContain('max_stalled < 5');
});
});
describe('migrate — runner behavioral (v14 handler + v15 backfill)', () => {
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
});
afterAll(async () => {
await engine.disconnect();
});
test('v14 created idx_pages_updated_at_desc on PGLite via handler branch', async () => {
const rows = await (engine as any).db.query(
`SELECT indexname FROM pg_indexes WHERE indexname = 'idx_pages_updated_at_desc'`
);
expect(rows.rows.length).toBe(1);
});
test('v15 backfilled any max_stalled=1 rows (smoke: schema default is 5)', async () => {
await (engine as any).db.exec(
`INSERT INTO minion_jobs (name, queue, status, max_stalled) VALUES ('test', 'default', 'waiting', 1)`
);
await (engine as any).db.exec(
`UPDATE minion_jobs SET max_stalled = 5
WHERE status IN ('waiting','active','delayed','waiting-children','paused')
AND max_stalled < 5`
);
const rows = await (engine as any).db.query(
`SELECT max_stalled FROM minion_jobs WHERE name = 'test'`
);
expect((rows.rows[0] as any).max_stalled).toBe(5);
await (engine as any).db.exec(
`UPDATE minion_jobs SET max_stalled = 5
WHERE status IN ('waiting','active','delayed','waiting-children','paused')
AND max_stalled < 5`
);
const rows2 = await (engine as any).db.query(
`SELECT max_stalled FROM minion_jobs WHERE name = 'test'`
);
expect((rows2.rows[0] as any).max_stalled).toBe(5);
});
});
describe('migrate: v8 (links_dedup) regression — must be fast on 1K duplicate rows', () => {
let engine: PGLiteEngine;