mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
* fix(cli): exit deliberately after bounded teardown instead of riding the 10s backstop (#2084)
Root cause: bounded teardown (endPoolBounded, #2015) RESOLVES, but lingering
sockets — embedding-provider fetch keep-alive, PgBouncer txn-mode sockets the
bound raced past — keep Bun's event loop alive, so every `gbrain query` paid
a flat 10s tax exiting via the hard-deadline force-exit banner.
Three changes, one contract:
- flushStdoutThenExit (cli-force-exit.ts): when main() resolves and the
command is not a daemon, exit deliberately — after stdout AND stderr drain
(writableLength===0, 'drain'-event + poll loop, 2s unref'd guard for a
blocked pipe). Incident #1959 (force-exit truncating piped stdout) is the
regression class; pinned by a 256KB real-pipe subprocess test.
- drainThenDisconnect (cli.ts): ONE owner-disconnect helper at all 8 sites
(op-dispatch, CLI_ONLY fall-through, search dashboard, doctor remediation
x3, ze-switch, dream, read-only timeout path). Drains the background-work
registry, then disconnect (best-effort), bounded by the 10s unref'd
hard-deadline — which is now armed around the TEARDOWN window only, not
before the op handler (the old placement would have force-killed any op
slower than 10s). Closes the filed TODOS P3 drain-hoist: six sites
previously skipped the drain entirely and had no hang timer at all.
- Inner process.exit sweep: mid-handler exits in engine-owning/output-bearing
paths (status, friction, claw-test, smoke-test, eval cross-modal /
takes-quality replay / conversation-parser / whoknows-thin, status-thin)
become process.exitCode + return so they flow through the drains and the
flush-exit. Pre-engine usage/parse/refusal exits stay as-is.
BrainRegistry.disconnectAll deliberately unchanged: zero production callers
in src/, per-engine disconnects already bounded, and the kernel reclaims
sockets on exit (src/core/timeout.ts doctrine).
DAEMON_COMMANDS gains 'watch' ahead of the #2095 push transport.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(e2e): PgBouncer transaction-mode pooler in CI + teardown e2e (#2084)
Three consecutive waves (#1972 → #2015 → #2084) fixed pooler-teardown bugs
verified only against one production deployment — CI had no transaction-mode
pooler and could never see the class. Now it can:
- docker-compose.ci.yml: `pgbouncer` service (transaction pooling) fronting
postgres-1, mirroring the production split-pool topology (direct :5432 +
pooled :6543). AUTH_TYPE=plain (pg16 SCRAM verifiers need the plaintext
password in the userlist) + IGNORE_STARTUP_PARAMETERS for the
statement_timeout/idle_in_transaction_session_timeout startup params
gbrain's client sets (the Supabase pooler whitelists the same).
- test/e2e/pgbouncer-teardown.test.ts: schema + fixture via the DIRECT url
into a dedicated `gbrain_pgbouncer` database (never races shard TRUNCATEs),
then spawns the real CLI against the POOLED url and asserts: exit 0,
stdout intact (the #1959 truncation class), and NO
"did not return within 10000ms — force-exiting" banner (pre-#2084 it
printed on 100% of query-shaped ops on this topology). Class bound, not
exact timing. Skips gracefully without GBRAIN_PGBOUNCER_URL.
- scripts/ci-local.sh: threads GBRAIN_PGBOUNCER_URL +
GBRAIN_PGBOUNCER_DIRECT_URL into all three e2e phases.
Verified live: both tests green against pgbouncer 1.25.2 in transaction mode.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(schema): context_volunteer_events table (v116) — push-context feedback log (#2095)
One row per page the brain volunteers (op / reflex / watch channels).
"Used" is DERIVED, never written: pages.last_retrieved_at > volunteered_at
(the existing bumpLastRetrievedAt write-back is the open/cite signal), so
there is no second tracking path. session_id/turn are nullable
caller-supplied attribution; rationale is a deterministic template string,
never raw conversation text.
- Migration v116 (idempotent) + mirrors in src/schema.sql +
src/core/pglite-schema.ts + regenerated schema-embedded.ts (regen also
folds in pre-existing comment-only drift from the v114 links edits).
- src/core/context/volunteer-events.ts: insertVolunteerEvents (ONE
multi-row parameterized INSERT — never per-row awaited round-trips) +
purgeStaleVolunteerEvents (90-day GC, returns 0 on pre-v116 brains).
- Dream cycle purge phase prunes stale events alongside op_checkpoints /
brainstorm checkpoints / batch-retry audit files.
- RLS on Postgres comes from the v35 auto_rls_on_create_table event
trigger (the same mechanism that covered v110 page_aliases and v115
op_checkpoint_paths); the volunteer Postgres e2e pins it.
- No ::jsonb anywhere; no bootstrap probe needed (nothing references the
table pre-creation; writers guard with try/catch).
Tests: v116 shape + columns + indexes + live insert/purge round-trip on
PGLite (test/migrate.test.ts, 161 pass); schema-bootstrap-coverage green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(context): multi-turn window extraction + confidence-scored volunteer core (#2095)
- entity-salience.ts: extractCandidatesFromWindow(turns) — runs the existing
per-turn extractor across the last N turns (oldest→newest), merges by the
normalizeAlias form with occurrence/newest-turn/user-mention metadata, and
orders by salience (recency > frequency > user-role) so the MAX_CANDIDATES
cap drops stale assistant chatter, not the entity the user just named.
Closes the filed assistant-introduced-entities recall TODO; true pronoun
coreference (never-named antecedents) stays out of scope.
- retrieval-reflex.ts: ReflexPointer gains source_id + arm + confidence +
matchedNorm. ARM_CONFIDENCE (alias 0.9 / title 0.8 / slug-suffix 0.6)
lives next to the arm definitions so identity and score can't drift.
Arm-2 provenance is classified in JS (codex D8 — the combined OR can't
report which predicate matched). Federated sourceIds[] scope (alias arm
loops per source; arm 2 uses source_id = ANY — no engine-interface
change). Suppression gains 'slug-only' mode (codex D7, REQUIRED for
windowing): the legacy title-whole-word rule would suppress every entity
merely MENTIONED in a prior window turn, breaking the feature by
construction — slugs only enter context when a pointer/page was actually
surfaced. Default stays 'slug-and-title' for the window=1 legacy path.
- volunteer.ts (new): parseWindow (lenient user:/assistant: prefixes, CRLF,
unprefixed → one user turn), volunteerContext (zero-LLM: extract →
resolve → +0.05 multi-turn/newest-turn boost → min_confidence 0.7 gate →
cap 3/5; deterministic rationale strings, never raw conversation text),
and volunteerUsageStats (per-arm/channel precision from the
last_retrieved_at join, labeled approximate — 5-min throttle false
negatives, unrelated-read false positives; codex D9).
Tests: 35 green across volunteer-context (window parsing, pronoun follow-up
via assistant-introduced entity, confidence gating, slug-only suppression,
takes-fence privacy, multi-source scope, caps, stats join math) +
retrieval-reflex back-compat + resolve-ipc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ops): volunteer_context op — CLI (stdin) + MCP, drained event sink (#2095)
New read-scope op on the contract surface (CLI `gbrain volunteer-context`
with stdin → window, MCP tool for free): takes a rolling conversation
window, returns confidence-gated page pointers with rationales + synopses.
`window` is optional-unless-stats (validated in the handler, codex D9);
`stats: true` returns the volunteered-vs-used precision summary, labeled
APPROXIMATE (the 5-min last-retrieved throttle and unrelated reads both
bias the join). Source scope threads through sourceScopeOpts — federated
grants narrow the volunteer to the granted sources.
Event logging is fire-and-forget through a new `volunteer-events`
background-work sink (volunteer-events.ts, mirrors last-retrieved: tracked
dangling promise set + bounded drain + snapshot-drop on timeout so a
long-lived process never accumulates ghosts). ONE batched INSERT per call,
drained on every exit path by the commit-1 drain hoist; failure never
fails the op (pinned by an injected failing-engine test).
cli formatResult renders both shapes (pointer lines with confidence/arm/
rationale; the stats summary with per-arm precision).
Tests: op contract surface, window-required validation, sink round-trip
with session_id/turn attribution, failing-engine fail-open, federated
grant scoping, stats mode (26 green on PGLite) + a real-Postgres e2e
proving the op + sink + stats join AND that context_volunteer_events has
RLS enabled (keeps the auto-RLS event-trigger mechanism honest for v116).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(context): reflex consumes the rolling window + ambient-channel logging (#2095)
The default-on retrieval reflex now extracts entities from the last N turns
(retrieval_reflex_window_turns, default 4; env
GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS; window=1 reproduces the legacy
current-turn-only behavior exactly). assemble() passes the recent
user/assistant turns (hard cap 12); the reflex slices to the configured
window. Assistant-introduced entities and "what did she invest in?"
follow-ups whose antecedent was NAMED in the window now surface pointers —
the issue's "zero agent-initiated queries" success criterion on the
ambient path.
Under windowing, suppression switches to slug-only (codex D7): the legacy
title-whole-word rule would suppress every entity merely MENTIONED in a
prior window turn, breaking the feature by construction. Slugs only enter
prior context when a pointer/page was actually surfaced, so
already-surfaced pages still suppress. The suppression mode flows through
all three resolver rungs (host opts, serve IPC request, direct Postgres).
Ambient-channel feedback (codex D11): the server-side resolver paths
(serve IPC + direct Postgres) log volunteered pointers with
channel: 'reflex' through the drained volunteer-events sink, so
`gbrain volunteer-context --stats` measures the default-on path where most
volunteering happens. Host-injected resolvers (no gbrain engine) can't
log — documented gap. Precision gates, 1.5s ceiling, fail-open, and the
pointer cap are unchanged.
Tests: prev-assistant-turn entity fires; window=1 legacy parity; slug-only
vs already-surfaced suppression; throwing resolver stays fail-open
(16 green).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(cli): gbrain watch — push transport over stdin (#2095)
The issue's headline: the brain volunteers pages as the conversation flows,
instead of waiting to be asked. `some-transcript-feed | gbrain watch` reads
turns line-by-line ('user:'/'assistant:' prefixes set the role; unprefixed
lines are user turns), keeps a rolling window (--window-turns, default 4),
and streams confidence-gated pointers with rationales to stdout (--json for
JSONL). Session dedupe rides the core's slug-only suppression — a slug is
volunteered at most once per session. Events log on channel 'watch' with
session_id + turn through the drained sink.
Lifecycle: watch BLOCKS in the stdin iteration (like `jobs work`) — an
interactive TTY stays alive until Ctrl-C/Ctrl-D, piped input ends at EOF —
so it is deliberately NOT in DAEMON_COMMANDS (reverts the commit-1
placeholder): when main() resolves the work is over, the CLI_ONLY finally
drains volunteer events via drainThenDisconnect, and the entrypoint
flush-exit ends the process. Keeping it in the daemon set would have made
the piped EOF path hang on lingering sockets — the exact #2084 class.
SIGINT closes the stream and flows through the same drain path instead of
killing mid-write. Per-turn resolution failures are fail-open (the stream
never dies on a transient DB error).
Full wiring (eng-review D12): CLI_ONLY + CLI_ONLY_SELF_HELP (WATCH_HELP) +
THIN_CLIENT_REFUSED_COMMANDS (thin clients use the volunteer_context MCP
op) + main --help entry.
Tests: 18 green — help, per-turn volunteering + clean EOF return, rolling
window via assistant-introduced entity, session dedupe, --json shape with
turn attribution, channel-watch event rows, --min-confidence gate, CRLF/
blank tolerance, daemon-gate semantics. Live smoke: piped `gbrain watch`
on a fresh PGLite brain exits 0 at EOF with no force-exit banner.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: KEY_FILES + push-context guide + TODOS for the #2084/#2095 wave
- docs/architecture/KEY_FILES.md (current-state): context entries gain the
window extractor, arm provenance/confidence, suppression modes, volunteer
+ volunteer-events modules; background-work entry now lists FIVE sinks and
the drainThenDisconnect owner-disconnect contract; new entries for
src/core/cli-force-exit.ts (the exit contract) and src/commands/watch.ts.
- docs/guides/push-context.md (new): the three channels (reflex/op/watch),
the confidence model, CLI usage, config keys, and the approximate-stats
caveat. Linked from CLAUDE.md's reference map.
- CLAUDE.md: ops line mentions volunteer_context + the guide link;
bun run build:llms regenerated in the same commit (freshness test green).
- TODOS.md: #2095 deferrals filed (SSE push channel, policy skill + doctor
check, structured messages[] param); the #1981 entity-detection TODO
narrowed (window extraction covered assistant-introduced entities +
named-antecedent follow-ups); the drain-hoist P3 marked DONE by the
#2084 wave.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(e2e): truncate context_volunteer_events in setupDB (#2095)
The new feedback-log table wasn't in ALL_TABLES, so volunteered-event rows
persisted across e2e runs on a reused database and poisoned count/stats
assertions in volunteer-context-postgres on the second run. No FK to pages
(slug join), so position before pages is for hygiene only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cli): own the exit verdict — never trust ambient process.exitCode (#2084)
Caught by the full unit suite: `gbrain apply-migrations` on PGLite started
exiting 99. Root cause: PGLite's Emscripten runtime writes the WASM
backend's proc_exit status into process.exitCode (initdb at create-time,
the postmaster at close-time — `exitCode=status` in pglite's dist), and
the writes land ASYNCHRONOUSLY, outside any snapshot/restore window around
create/close (a guarded attempt verified this). The pre-#2084 success path
never read process.exitCode, so the pollution was invisible; the new
deliberate flush-exit propagated it faithfully.
Fix: gbrain records its own verdict. setCliExitCode(n)/getCliExitCode() in
cli-force-exit.ts — every gbrain-owned exit-code assignment routes through
the setter (still mirrored to process.exitCode for outside readers), and
both exit paths (entrypoint flushStdoutThenExit + the drainThenDisconnect
hard-deadline backstop) read the getter. Swept all assignment sites:
cli.ts (op error, friction, claw-test, smoke-test, eval runners, status,
import errors) + reindex/transcripts/brainstorm/frontmatter/autopilot.
Also updates the v0.42.20 structural pins to the drainThenDisconnect shape
(ordering invariant asserted INSIDE the helper + >=8 helper call sites,
superseding the two-inline-pairs assertion).
Verified: apply-migrations spawn test green; `init --migrate-only` exits 0;
an errored op still exits 1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: re-pin the teardown-arming invariant at its post-#2084 home
Master's v0.42.41.0 triage wave and the #2084 wave fixed the same
pre-armed-timer bug independently; the merge keeps #2084's shape (arming
inside the shared drainThenDisconnect helper, covering all 8 exit paths).
The structural pin now asserts the same invariant — no pre-try arming;
gated, unref'd, before-drain, cleared — at the helper.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: coverage for ambient reflex-channel logging + watch window/cap flags
Ship coverage audit (85%, gate PASS) named five gaps; the two substantive
cheap ones close here: the codex-D11 logChannel='reflex' path now has a
behavioral pin (events land on channel 'reflex' through the drained sink;
no logChannel → no events), and gbrain watch's --window-turns / --max-pages
flags are exercised (turn-1 attribution under window=1; cap to one page).
Remaining flagged-not-blocking: the wallclock-timeout branch (untestable
without >10s real-clock flake — same rationale as the arming pin),
formatResult's volunteer case (module-private), and the cycle purge wiring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: close the remaining plan-audit gaps — formatResult rendering + watch SIGINT
formatResult exported for tests (same import-safety contract as cliAliases);
test/cli-format-volunteer.test.ts pins the pointer lines, empty-gate message,
and approximate stats summary. test/watch-command.test.ts gains a real
subprocess SIGINT test: piped stdin that never reaches EOF, SIGINT mid-stream,
assert exit 0 with no force-exit banner — the drain-then-exit lifecycle under
the actual signal, not just the shared exit path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: doctor's FAIL verdict was zeroed by the owned exit — sweep stragglers + class pin
The merged-state suite caught it: doctor --fast --json reported FAIL but
exited 0. Master's v0.42.41.0 brought raw `process.exitCode =` writes
(doctor.ts hasFail ternary, extract.ts) that the #2084 verdict-owning exit
silently zeroes — getCliExitCode() deliberately never reads ambient
process.exitCode (the PGLite-Emscripten pollution defense), so any setter
that bypasses setCliExitCode reports success on failure.
Swept both sites and added the structural class pin: a test greps src/ for
raw `process.exitCode =` outside cli-force-exit.ts, so the next merge that
introduces one fails loudly instead of lying about exit codes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.43.0)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: quarantine the watch SIGINT subprocess test to the serial lane
The parallel unit shards flake on concurrent CLI subprocess spawns (failed
at 7ms in-suite, green solo) — same isolation rationale as
apply-migrations-pglite-spawn.serial.test.ts and #2141's R3 quarantine.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: update project documentation for v0.42.43.0
Post-ship doc verification against the release diff (#2095 push-based
context + #2084 superset hardening), with a cross-model doc review:
- push-context.md: version tag corrected to v0.42.43.0; per-call knobs
now cover prior_context/days and watch's flag surface accurately;
feedback-log writes described as best-effort; synopsis fence-strip
described as unconditional.
- CLAUDE.md: stale operation count (~47 -> ~90); volunteer_context
release reference corrected to v0.42.43.0.
- KEY_FILES.md: ci-local entry rewritten to current topology (4-shard
parallel default, four Postgres services, transaction-mode PgBouncer
+ GBRAIN_PGBOUNCER_URL/_DIRECT_URL exports); stale E2E file counts
dropped from the selector entry.
- TESTING.md: inventory entries for the new #2084 structural pins
(cli-exit-verdict-pin, cli-pipe-truncation), the push-context test
suite (volunteer-context, watch-command, watch-sigint.serial,
cli-format-volunteer), migrate v117 coverage, and the two new E2E
files (pgbouncer-teardown env gating, volunteer-context-postgres RLS
pin); check:all row corrected (not a superset of verify).
- AGENTS.md + RELEASING.md: ci:local descriptions updated to the
sharded + pooler topology.
- CHANGELOG (wording only, entry preserved): "retrieved" instead of
"opened" for the used-signal, pooler scoped to the local CI gate,
feedback log labeled best-effort.
- llms-config.ts: index the new push-context guide; bundles
regenerated (build:llms) and freshness test green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(test): correct the v116 reference — the table shipped as migration v117
* fix: pre-landing review hardening — federated alias parallelism, trust-boundary clamps, shared protocol helpers (#2095)
Five specialist reviewers (testing/maintainability/security/performance/
data-migration) on the reconciled diff; every finding applied:
Performance: the alias arm now resolves all granted sources CONCURRENTLY
(a federated caller paid M sequential RTTs per turn — ~355ms at 5 sources
cross-region, inside the reflex's 1.5s budget); watch's session dedupe is
O(1) Set membership instead of a monotonically growing priorContext string
(O(T²) over a long-lived session); getWindowTurns iterates from the tail
(per-turn cost no longer grows with session length); the resolver's
provenance maps fold into the existing candidate pass.
Security: volunteer_context clamps caller-supplied attribution at the trust
boundary — session_id capped at 256 chars (a read-scoped token could bank
~1MiB TEXT per request, retained 90 days), turn logged only when a safe
integer (a non-integer threw inside the batched INSERT and silently dropped
the whole batch). The privacy comments now state precisely what rationale
may contain (the matched entity's surface form — which by construction
resolved to an existing alias/title/slug — never free conversation text).
Maintainability: TURN_PREFIX_RE + formatVolunteeredPage exported from
volunteer.ts and shared by watch/cli (the two surfaces can no longer
drift); volunteerEventRowsFrom is the single VolunteerEventRow assembly
site for all three channels; watch's window default now honors the same
retrieval_reflex_window_turns config knob the reflex reads; the stale
pre-v116 comments swept to pre-v117.
Testing: the two flake-class CRITICALs fixed (pipe test asserts the
backstop banner instead of a cold-CI-hostile 9s wall bound; the SIGINT test
waits on watch's new machine-readable ready line instead of a fixed 15s
sleep — 2.5s and deterministic now); new coverage for the sink's timeout
branch + ghost-reference drop, watch per-turn fail-open, untrusted knob
clamps (min_confidence/max_pages/days), window-cap ordering (newest user
mention survives), serve-IPC suppression passthrough + channel=reflex
logging, windowTurnCount edge semantics, and structural pins for the sink
registration + cycle purge wiring. The exit-verdict pin's grep is now
operator/whitespace-tolerant.
Deferred with TODOs: resolver index shapes for the per-turn query;
batched first-prune after a long dream-cycle gap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(context): red-team hardening — pre-cap dedupe, delivery-side reflex logging, window clamp
Four red-team findings on the #2095 push-context surface:
- RT1 starvation: watch's session-dedupe Set filtered AFTER volunteerContext's
cap, so a recurring already-pushed entity burned cap slots every turn and
starved fresh pages behind it. VolunteerOpts.excludeSlugs now skips inside
the pointer loop BEFORE the confidence gate and the cap.
- RT3 honest stats: reflex-channel event logging moved from inside the
resolver to the DELIVERY point — serve's resolve-IPC onDelivered hook fires
only after the response write succeeds, and buildReflexAddition logs only
after the per-turn timeout admits the block. A block the client's 250ms
budget abandoned was never injected and no longer counts as volunteered.
(logChannel resolver opt removed; logDeliveredReflexPointers is the seam.)
- RT5 unbounded window: --window-turns is clamped to [1, 64] so a config typo
can't reintroduce the re-scan-everything-per-turn cost class.
- RT2/RT4 documented + filed: PGLite watch connection monopoly (WATCH_HELP,
push-context guide, TODO to route watch via serve IPC); host-resolver
suppression contract at ResolveEntitiesFn (TODO for a capability gate).
Tests: starvation guard (watch + volunteerContext unit), window clamp floor +
ceiling, delivery-side logging (helper writes channel=reflex through the
drained sink; bare resolver writes nothing; empty list no-op), IPC wiring test
rewired to onDelivered. KEY_FILES.md + push-context.md updated; build:llms run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(context): env-plane window knob works config-less; harden two gateway-state-leak victims
Three CI-only check failures, two root causes:
1. windowTurnCount ignored GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS when
loadConfig() returned null (no config file AND no DATABASE_URL — a clean
CI shard with no brain). loadConfig drops its env→config mapping in that
case, so the documented escape hatch silently died and the window fell
back to 4 → windowed extraction widened when the test set window=1 →
prior-turn entity leaked. Fixed: read the env var directly in
windowTurnCount, mirroring reflexEnabled's direct process.env read. This
is a real product bug, not just a test artifact — any config-less host
using the env hatch was affected. Regression test pins it.
2. sync-cost-preview + doctor-federation-health failed only IN-SHARD: a
sibling test configured a non-legacy (ZeroEntropy 1280-d / $0.05) gateway
and never reset it. The legacy-embedding preload only restores the
OpenAI/1536 default when the gateway slot is EMPTY, so a non-empty foreign
config survives into the next file — and a file's beforeAll runs BEFORE
the preload's restoring beforeEach, so federation-health built a
vector(1280) column and its 1536-d fixture hit CheckExpectedDim. My new
test files reshuffled the deterministic file→shard assignment, exposing
this latent ordering bug. Hardened both victims to establish the gateway
state they assert (sync-cost-preview resets to the unconfigured fallback;
federation-health pins legacy 1536 before initSchema) so they're
order-independent. Verified against a simulated leaker run before them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(context): use withEnv() in the window env-hatch test (test-isolation guard)
The regression test added in 82cc7fff mutated process.env directly, which
check:test-isolation (R1) forbids — use the withEnv() helper that restores on
exit, same as the rest of this file. Behavior identical; guard green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
270 lines
14 KiB
TypeScript
270 lines
14 KiB
TypeScript
/**
|
|
* Structural assertions for fix-wave fixes whose behavior is best verified
|
|
* at source-shape level rather than via a runtime harness:
|
|
*
|
|
* - #1125 query drain cache writes — assert cli.ts awaits the drain after
|
|
* the query op completes.
|
|
* - #1090 admin embed — assert the two-tier resolution (cwd path + embedded
|
|
* manifest fallback) is in serve-http.ts and consumes ADMIN_ASSETS.
|
|
* - #1077 admin register-client PKCE — assert the admin endpoint honors
|
|
* grantTypes / redirectUris / tokenEndpointAuthMethod from the body.
|
|
* - #1100 PGLite phaseASchema — assert the v0.11.0 orchestrator routes
|
|
* in-process when the engine is pglite (not via execSync subprocess).
|
|
* - #1124 query no-expand — assert the parseOpArgs negation logic exists.
|
|
*
|
|
* Source-grep regression tests are the right tool when the rule is "this
|
|
* specific line shape must stay present"; a behavioral test would either
|
|
* duplicate what an E2E covers or require heavy mocking that hides the
|
|
* regression behind a test seam.
|
|
*/
|
|
import { describe, test, expect } from 'bun:test';
|
|
import { readFileSync } from 'fs';
|
|
|
|
describe('v0.42.20.0 — search-cache drained via the background-work registry', () => {
|
|
// Supersedes the v0.36.1.x #1125 query-only drain: search-cache now registers
|
|
// a registry drainer (drained for BOTH search and query, bounded), and cli.ts
|
|
// drains the whole registry rather than calling awaitPendingSearchCacheWrites
|
|
// directly for the 'query' op only.
|
|
test('hybrid.ts registers a bounded search-cache drainer', () => {
|
|
const src = readFileSync('src/core/search/hybrid.ts', 'utf8');
|
|
expect(src).toMatch(/export async function awaitPendingSearchCacheWrites/);
|
|
expect(src).toMatch(/pendingCacheWrites\.add\(promise\)/);
|
|
expect(src).toMatch(/trackCacheWrite\(/);
|
|
// Now bounded (was an unbounded Promise.allSettled) + registered.
|
|
expect(src).toMatch(/registerBackgroundWorkDrainer\(\{[\s\S]*?name:\s*'search-cache'/);
|
|
expect(src).toMatch(/Promise\.race/);
|
|
});
|
|
});
|
|
|
|
describe('v0.36.1.x #1090 — admin embed two-tier resolution', () => {
|
|
test('serve-http.ts uses ADMIN_ASSETS manifest when admin/dist is not next to cwd', () => {
|
|
const src = readFileSync('src/commands/serve-http.ts', 'utf8');
|
|
expect(src).toMatch(/import\(['"]\.\.\/admin-embedded/);
|
|
expect(src).toMatch(/ADMIN_ASSETS/);
|
|
expect(src).toMatch(/ADMIN_INDEX_HTML/);
|
|
// Two-tier: dev path (cwd-relative admin/dist) AND embedded manifest fallback
|
|
expect(src).toMatch(/useDevPath/);
|
|
});
|
|
|
|
test('src/admin-embedded.ts is auto-generated with file: imports', () => {
|
|
const src = readFileSync('src/admin-embedded.ts', 'utf8');
|
|
expect(src).toMatch(/AUTO-GENERATED/);
|
|
expect(src).toMatch(/with \{ type: 'file' \}/);
|
|
expect(src).toMatch(/export const ADMIN_ASSETS/);
|
|
expect(src).toMatch(/export const ADMIN_INDEX_HTML/);
|
|
});
|
|
|
|
test('build script + CI guard exist', () => {
|
|
const buildSrc = readFileSync('scripts/build-admin-embedded.ts', 'utf8');
|
|
expect(buildSrc).toMatch(/walk\(DIST/);
|
|
expect(buildSrc).toMatch(/with \{ type: 'file' \}/);
|
|
const guard = readFileSync('scripts/check-admin-embedded.sh', 'utf8');
|
|
expect(guard).toMatch(/git diff --exit-code -- src\/admin-embedded\.ts/);
|
|
});
|
|
});
|
|
|
|
describe('v0.36.1.x #1077 — admin register-client supports PKCE public clients', () => {
|
|
test('admin endpoint reads grantTypes / redirectUris / tokenEndpointAuthMethod from request body', () => {
|
|
const src = readFileSync('src/commands/serve-http.ts', 'utf8');
|
|
// The destructure must surface name / tokenTtl / grantTypes /
|
|
// redirectUris / tokenEndpointAuthMethod from req.body. v0.39.3.0
|
|
// WARN-9 (PR #1308) moved `scopes` to a separate read line that
|
|
// accepts BOTH `scopes` (admin SPA) AND `scope` (OAuth wire singular)
|
|
// via `?? `, so this regex no longer requires `scopes` in the inline
|
|
// destructure — it's separately covered by the scope-source check
|
|
// below.
|
|
expect(src).toMatch(/const\s+\{\s*name,\s*(?:[^}]*?,\s*)?tokenTtl,\s*grantTypes,\s*redirectUris,\s*tokenEndpointAuthMethod\s*\}\s*=\s*req\.body/);
|
|
// v0.39.3.0 WARN-9: the route must still read a `scope`/`scopes` field
|
|
// (under either name) from req.body. Pin the fallback pattern so the
|
|
// PKCE-fix regression contract stays load-bearing.
|
|
expect(src).toMatch(/req\.body[^;]*scopes\s*\?\?\s*[^;]*scope\b/);
|
|
// v0.41.3 (T4 atomicity fix, codex F4): admin endpoint now validates
|
|
// tokenEndpointAuthMethod via the shared validator and passes it to
|
|
// registerClientManual as a positional arg. Pre-v0.41.3 the route did
|
|
// INSERT (confidential) → UPDATE (NULL out secret_hash) for the 'none'
|
|
// case, which left a confidential row stranded if the UPDATE failed.
|
|
// Atomic now: one INSERT writes the correct shape; no post-insert
|
|
// UPDATE block (the regex deliberately asserts the post-insert UPDATE
|
|
// is GONE).
|
|
expect(src).toMatch(/validateTokenEndpointAuthMethod\(tokenEndpointAuthMethod\)/);
|
|
expect(src).toMatch(/registerClientManual\([^)]*validatedAuthMethod[^)]*\)/);
|
|
// Regression guard: post-insert UPDATE flipping client_secret_hash to
|
|
// NULL based on a runtime check is exactly the non-atomic pattern T4
|
|
// killed. Re-introducing it brings back codex F4.
|
|
expect(src).not.toMatch(/UPDATE oauth_clients SET client_secret_hash = NULL, token_endpoint_auth_method = 'none'/);
|
|
});
|
|
});
|
|
|
|
describe('v0.41.37.0 #1605 — v0.11.0 phaseASchema routes in-process for ALL engines', () => {
|
|
test('phaseASchema calls runMigrateOnlyCore (in-process) + is awaited', () => {
|
|
// Supersedes #1100's PGLite-only in-process branch. v0.41.37.0 #1605 routes
|
|
// EVERY engine through runMigrateOnlyCore (no execSync subprocess at all),
|
|
// which is strictly stronger: PGLite still never subprocesses, AND the
|
|
// Windows+Postgres getaddrinfo-ENOTFOUND spawn bug is closed too.
|
|
// The eng.initSchema() call moved into src/commands/migrations/in-process.ts.
|
|
const src = readFileSync('src/commands/migrations/v0_11_0.ts', 'utf8');
|
|
expect(src).toContain('runMigrateOnlyCore()');
|
|
expect(src).not.toContain("execSync('gbrain init --migrate-only'");
|
|
expect(src).toMatch(/await\s+phaseASchema/);
|
|
});
|
|
|
|
test('apply-migrations skips pre-flight schema-version probe on PGLite', () => {
|
|
const src = readFileSync('src/commands/apply-migrations.ts', 'utf8');
|
|
expect(src).toMatch(/skipPreflight\s*=\s*cfg\.engine\s*===\s*'pglite'/);
|
|
});
|
|
});
|
|
|
|
describe('v0.36.1.x #1124 — query --no-expand actually negates expand', () => {
|
|
test("cli.ts parseOpArgs handles --no-<key> as boolean negation", () => {
|
|
const src = readFileSync('src/cli.ts', 'utf8');
|
|
expect(src).toMatch(/arg\.startsWith\(['"]--no-['"]\)/);
|
|
expect(src).toMatch(/positiveDef\?\.type\s*===\s*'boolean'/);
|
|
expect(src).toMatch(/params\[positiveKey\]\s*=\s*false/);
|
|
});
|
|
});
|
|
|
|
describe('v0.42.20.0 — background-work registry drains every sink before disconnect', () => {
|
|
// Supersedes the v0.41.8.0 #1247/#1269/#1290 per-call last-retrieved drain:
|
|
// last-retrieved is one of four registry sinks. #2084 moved the registry
|
|
// drain out of cli.ts's inline finallys into finishCliTeardown
|
|
// (cli-force-exit.ts), which every cli.ts teardown site routes through —
|
|
// the drain-before-disconnect invariant is pinned there (and behaviorally
|
|
// by test/cli-finish-teardown.test.ts).
|
|
test('cli-force-exit.ts imports + drains the registry inside finishCliTeardown', () => {
|
|
const src = readFileSync('src/core/cli-force-exit.ts', 'utf8');
|
|
expect(src).toMatch(/import\s+\{\s*drainAllBackgroundWorkForCliExit[\s\S]*?\}\s*from\s+['"]\.\/background-work\.ts['"]/);
|
|
expect(src).toMatch(/export async function finishCliTeardown/);
|
|
});
|
|
|
|
test('last-retrieved.ts still exports the bounded drain + registers a drainer', () => {
|
|
const src = readFileSync('src/core/last-retrieved.ts', 'utf8');
|
|
expect(src).toMatch(/export async function awaitPendingLastRetrievedWrites/);
|
|
expect(src).toMatch(/pendingLastRetrievedWrites\s*=\s*new\s+Set/);
|
|
expect(src).toMatch(/Promise\.race/);
|
|
expect(src).toMatch(/registerBackgroundWorkDrainer\(\{[\s\S]*?name:\s*'last-retrieved'/);
|
|
});
|
|
|
|
test('all four sinks register a drainer', () => {
|
|
expect(readFileSync('src/core/facts/queue.ts', 'utf8'))
|
|
.toMatch(/registerBackgroundWorkDrainer\(\{[\s\S]*?name:\s*'facts'[\s\S]*?abort:/);
|
|
expect(readFileSync('src/core/search/hybrid.ts', 'utf8'))
|
|
.toMatch(/name:\s*'search-cache'/);
|
|
expect(readFileSync('src/core/last-retrieved.ts', 'utf8'))
|
|
.toMatch(/name:\s*'last-retrieved'/);
|
|
expect(readFileSync('src/core/eval-capture.ts', 'utf8'))
|
|
.toMatch(/name:\s*'eval-capture'/);
|
|
});
|
|
|
|
test('finishCliTeardown positioning: registry drain appears BEFORE engine disconnect', () => {
|
|
// #2084: the invariant moved from cli.ts's inline finallys into the shared
|
|
// helper. The drain must run against a live engine (facts abort-path
|
|
// logIngest, #1762) before disconnect tears the pools down.
|
|
const src = readFileSync('src/core/cli-force-exit.ts', 'utf8');
|
|
const drainCallRe = /await\s+drain\s*\(\s*\{\s*timeoutMs:\s*drainTimeoutMs\s*\}\s*\)/;
|
|
const disconnectCallRe = /await\s+opts\.engine\.disconnect\s*\(/;
|
|
expect(src).toMatch(drainCallRe);
|
|
expect(src).toMatch(disconnectCallRe);
|
|
const drainIdx = src.indexOf(src.match(drainCallRe)![0]);
|
|
const disconnectIdx = src.indexOf(src.match(disconnectCallRe)![0]);
|
|
expect(drainIdx).toBeLessThan(disconnectIdx);
|
|
});
|
|
|
|
test('background-work.ts: Map registry, ordered drain, awaited abort, test seam', () => {
|
|
const src = readFileSync('src/core/background-work.ts', 'utf8');
|
|
expect(src).toMatch(/new\s+Map<string,\s*BackgroundWorkDrainer>/);
|
|
expect(src).toMatch(/sort\(\s*\(a,\s*b\)\s*=>\s*a\.order\s*-\s*b\.order/);
|
|
expect(src).toMatch(/if\s*\(unfinished\s*>\s*0\s*&&\s*d\.abort\)\s*\{[\s\S]*?await\s+d\.abort\(\)/);
|
|
expect(src).toMatch(/export function __registerDrainerForTest/);
|
|
});
|
|
|
|
test('cli-force-exit.ts daemon guard excludes "serve"', () => {
|
|
const src = readFileSync('src/core/cli-force-exit.ts', 'utf8');
|
|
expect(src).toMatch(/export function shouldForceExitAfterMain/);
|
|
expect(src).toMatch(/DAEMON_COMMANDS[\s\S]*serve/);
|
|
});
|
|
});
|
|
|
|
describe('#2084 — cli.ts owns process-exit teardown via finishCliTeardown', () => {
|
|
test('no bare awaited engine disconnects remain in cli.ts', () => {
|
|
// The awaited forms are the call-site contract (comments never use the
|
|
// awaited literal, so this is comment-proof — eng-review D13.2). A bare
|
|
// disconnect skips the bounded drain + computed-deadline backstop and
|
|
// reopens the lingering-socket hang class.
|
|
const src = readFileSync('src/cli.ts', 'utf8');
|
|
expect(src).not.toContain('await engine.disconnect()');
|
|
expect(src).not.toContain('await eng.disconnect()');
|
|
});
|
|
|
|
test('the pre-handler hard-deadline timer is gone (handler time is not teardown budget)', () => {
|
|
// Pre-#2084 the op-dispatch timer armed BEFORE the op handler, so any op
|
|
// slower than 10s was force-killed mid-run with exit 0 and truncated
|
|
// output. The deadline now arms inside finishCliTeardown, at teardown
|
|
// start only.
|
|
const src = readFileSync('src/cli.ts', 'utf8');
|
|
expect(src).not.toContain('DISCONNECT_HARD_DEADLINE_MS');
|
|
});
|
|
|
|
test('all nine swept sites route through finishCliTeardown; one exit seam', () => {
|
|
const src = readFileSync('src/cli.ts', 'utf8');
|
|
const calls = src.match(/await finishCliTeardown\(/g) ?? [];
|
|
expect(calls.length).toBeGreaterThanOrEqual(9);
|
|
// The single process-exit seam: flushThenExit in the import.meta.main
|
|
// block, fed by currentExitCode().
|
|
expect(src).toMatch(/import\.meta\.main/);
|
|
expect(src).toMatch(/flushThenExit\(currentExitCode\(\)\)/);
|
|
});
|
|
|
|
test('pglite-engine contains the Emscripten process.exitCode hijack', () => {
|
|
// PGLite's WASM runtime writes its own status into process.exitCode (99
|
|
// alive / exit status on close) and ignores `undefined` assignment. The
|
|
// create call runs inside preservingProcessExitCode to keep the global
|
|
// tidy; close is deliberately unwrapped (see below) — the CLI's verdict
|
|
// is immune either way via the owned channel.
|
|
const src = readFileSync('src/core/pglite-engine.ts', 'utf8');
|
|
expect(src).toMatch(/preservingProcessExitCode\(\(\)\s*=>\s*\n?\s*PGlite\.create/);
|
|
// close stays UNWRAPPED by design: its status write is baseline behavior
|
|
// test runners depend on; the CLI's verdict is immune because it lives in
|
|
// the gbrain-owned channel, never read back from process.exitCode.
|
|
const helper = readFileSync('src/core/cli-force-exit.ts', 'utf8');
|
|
expect(helper).toMatch(/let cliVerdict: number \| null = null/);
|
|
expect(helper).toMatch(/return cliVerdict \?\? 0/);
|
|
// The op-dispatch catch must set the verdict through the owned channel.
|
|
const cli = readFileSync('src/cli.ts', 'utf8');
|
|
expect(cli).toMatch(/setCliExitVerdict\(1\);/);
|
|
});
|
|
});
|
|
|
|
describe('v0.41.8.0 #1340 — PGLite WASM init classifier', () => {
|
|
test('pglite-engine.ts exports classifyPgliteInitError + buildPgliteInitErrorMessage', () => {
|
|
const src = readFileSync('src/core/pglite-engine.ts', 'utf8');
|
|
expect(src).toMatch(/export function classifyPgliteInitError/);
|
|
expect(src).toMatch(/export function buildPgliteInitErrorMessage/);
|
|
// Per Codex finding #9: regex tightened to $$bunfs OR ENOENT+pglite.data
|
|
expect(src).toMatch(/\$\$bunfs/);
|
|
expect(src).toMatch(/ENOENT/);
|
|
});
|
|
|
|
test('pglite-engine.ts connect catch block routes through the classifier', () => {
|
|
const src = readFileSync('src/core/pglite-engine.ts', 'utf8');
|
|
expect(src).toMatch(/classifyPgliteInitError\(original\)/);
|
|
expect(src).toMatch(/buildPgliteInitErrorMessage\(verdict, original\)/);
|
|
});
|
|
});
|
|
|
|
describe('v0.42.43.0 #2095 — volunteer-events sink + cycle purge wiring (structural pins)', () => {
|
|
test('volunteer-events registers a background-work drainer (order 4)', () => {
|
|
// Deleting this registration would silently drop volunteer events on
|
|
// every CLI exit with no behavioral test failing — same pin class as the
|
|
// other four sinks above.
|
|
const src = readFileSync('src/core/context/volunteer-events.ts', 'utf8');
|
|
expect(src).toMatch(/registerBackgroundWorkDrainer\(\{[\s\S]*?name:\s*'volunteer-events'/);
|
|
expect(src).toMatch(/order:\s*4/);
|
|
});
|
|
|
|
test("the dream cycle's purge phase invokes purgeStaleVolunteerEvents and reports the count", () => {
|
|
const src = readFileSync('src/core/cycle.ts', 'utf8');
|
|
expect(src).toMatch(/purgeStaleVolunteerEvents\(engine\)/);
|
|
expect(src).toMatch(/purged_volunteer_events_count/);
|
|
});
|
|
});
|