The drain window was only checked BETWEEN batches: one batch (sequential
LLM calls per page) plus lock acquire/release and the backlog count ran
unbounded, so window=120s runs were observed overrunning to 282.5s.
The drain now derives a real-time deadline signal (window timeout + the
Minion job.signal) and threads it through everything the loop awaits:
- extract-atoms-drain: signal-driven loop; hung count/batch/lock-acquire
are cancelled at the deadline; no post-window final count (remaining
reports null instead of overrunning); external worker abort rethrows
after the lock is released.
- extract-atoms phase: abortSignal forwarded to every gateway chat call,
discovery/count/idempotency queries, and putPage writes, plus a
cooperative between-item check (the bound on PGLite, whose query abort
only abandons the waiter — the default engine keeps a working drain).
Billable usage is recorded before the post-chat abort check; receipt +
rollup bookkeeping runs on a fresh 5s grace signal so committed atoms
never lose their cost trail; deadline-truncated runs don't count as
completed rounds (deadline_aborted detail).
- db-lock: tryAcquireDbLock/withRefreshingLock accept a deadline signal;
release routes through the direct session pool with a fresh grace
timeout; deadline-bound callers skip the unbounded same-host takeover
path (honest busy result; TTL stays the backstop).
- engines: putPage accepts an optional signal (Postgres cancels the
in-flight statement; PGLite pre-checks); executeRawDirect short-circuits
an already-fired signal before pool routing and bounds a stalled
direct-pool acquisition.
Takeover of PR #2752: keeps its signal-threading design, but retains the
putPage write path (so the chunker_version default from #2988 applies —
the PR's bulk INSERT bypassed it), drops the PGLite hard refusal in favor
of cooperative abort, and reverts the unsanctioned transcript-suppression
scope change (_transcripts: []).
Fixes#2750
Co-authored-by: panda850819 <panda850819@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(minions): route lock claim/renewLock through direct session pool
The Minion lock heartbeat (claim + renewLock) ran every UPDATE through
engine.executeRaw(), which is hardcoded to the read pool. On Supabase that
is the transaction-mode pooler (6543), which recycles connections per
transaction. A lock is held for minutes, so the pooler periodically reaps
the socket mid-heartbeat -> CONNECTION_ENDED -> the lock looks expired ->
the worker force-evicts its own job and the claim loop wedges silently.
Add BrainEngine.executeRawDirect(): same contract as executeRaw, but routes
to the direct session-mode pool (5432, GBRAIN_DIRECT_DATABASE_URL) when
dual-pool is active. No-op delegation on PGLite / non-Supabase / kill-switch.
claim/renewLock now use it. Single-statement UPDATEs only, so the double-claim
guard and the renewLock no-inline-retry contract are preserved. Statements
inside an open transaction keep their tx connection (in-transaction guard keys
on peekReadPool() !== _sql); the lock hot-path never runs inside transaction().
The Postgres impl shares its cancellation plumbing with executeRaw via a
private runUnsafe helper. New test/postgres-execute-raw-direct.test.ts covers
the routing decision (dual-pool on/off x in-tx/not + abort short-circuit)
without a live Postgres; queue-lock-retry.test.ts gains a guard that claim
can never fall back to executeRaw.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* v0.42.24.0 chore: bump version and changelog
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: update project documentation for v0.42.24.0
Document executeRawDirect on the BrainEngine contract and the
claim/renewLock direct-session-pool routing in KEY_FILES.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: make D3 executeRaw-no-retry guard refactor-aware
The DRY refactor in this PR extracted executeRaw/executeRawDirect's shared
cancellation plumbing into a private runUnsafe(conn, ...) helper, so the single
conn.unsafe() call moved out of executeRaw's body. The D3 guard read
executeRaw's source and asserted conn.unsafe( appeared exactly once there,
which now fails (it's zero — executeRaw delegates).
The D3 invariant (no per-call retry wrapper) is unchanged; it just spans the
delegate now. Update the guard to check both public methods delegate to
runUnsafe without reconnect/retry, and assert the exactly-once conn.unsafe +
cancel-only catch in runUnsafe. Also extends coverage to executeRawDirect so
the lock hot-path can't reintroduce a retry wrapper either.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
