mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
* fix(cli): exit deliberately after bounded teardown instead of riding the 10s backstop (#2084)
Root cause: bounded teardown (endPoolBounded, #2015) RESOLVES, but lingering
sockets — embedding-provider fetch keep-alive, PgBouncer txn-mode sockets the
bound raced past — keep Bun's event loop alive, so every `gbrain query` paid
a flat 10s tax exiting via the hard-deadline force-exit banner.
Three changes, one contract:
- flushStdoutThenExit (cli-force-exit.ts): when main() resolves and the
command is not a daemon, exit deliberately — after stdout AND stderr drain
(writableLength===0, 'drain'-event + poll loop, 2s unref'd guard for a
blocked pipe). Incident #1959 (force-exit truncating piped stdout) is the
regression class; pinned by a 256KB real-pipe subprocess test.
- drainThenDisconnect (cli.ts): ONE owner-disconnect helper at all 8 sites
(op-dispatch, CLI_ONLY fall-through, search dashboard, doctor remediation
x3, ze-switch, dream, read-only timeout path). Drains the background-work
registry, then disconnect (best-effort), bounded by the 10s unref'd
hard-deadline — which is now armed around the TEARDOWN window only, not
before the op handler (the old placement would have force-killed any op
slower than 10s). Closes the filed TODOS P3 drain-hoist: six sites
previously skipped the drain entirely and had no hang timer at all.
- Inner process.exit sweep: mid-handler exits in engine-owning/output-bearing
paths (status, friction, claw-test, smoke-test, eval cross-modal /
takes-quality replay / conversation-parser / whoknows-thin, status-thin)
become process.exitCode + return so they flow through the drains and the
flush-exit. Pre-engine usage/parse/refusal exits stay as-is.
BrainRegistry.disconnectAll deliberately unchanged: zero production callers
in src/, per-engine disconnects already bounded, and the kernel reclaims
sockets on exit (src/core/timeout.ts doctrine).
DAEMON_COMMANDS gains 'watch' ahead of the #2095 push transport.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(e2e): PgBouncer transaction-mode pooler in CI + teardown e2e (#2084)
Three consecutive waves (#1972 → #2015 → #2084) fixed pooler-teardown bugs
verified only against one production deployment — CI had no transaction-mode
pooler and could never see the class. Now it can:
- docker-compose.ci.yml: `pgbouncer` service (transaction pooling) fronting
postgres-1, mirroring the production split-pool topology (direct :5432 +
pooled :6543). AUTH_TYPE=plain (pg16 SCRAM verifiers need the plaintext
password in the userlist) + IGNORE_STARTUP_PARAMETERS for the
statement_timeout/idle_in_transaction_session_timeout startup params
gbrain's client sets (the Supabase pooler whitelists the same).
- test/e2e/pgbouncer-teardown.test.ts: schema + fixture via the DIRECT url
into a dedicated `gbrain_pgbouncer` database (never races shard TRUNCATEs),
then spawns the real CLI against the POOLED url and asserts: exit 0,
stdout intact (the #1959 truncation class), and NO
"did not return within 10000ms — force-exiting" banner (pre-#2084 it
printed on 100% of query-shaped ops on this topology). Class bound, not
exact timing. Skips gracefully without GBRAIN_PGBOUNCER_URL.
- scripts/ci-local.sh: threads GBRAIN_PGBOUNCER_URL +
GBRAIN_PGBOUNCER_DIRECT_URL into all three e2e phases.
Verified live: both tests green against pgbouncer 1.25.2 in transaction mode.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(schema): context_volunteer_events table (v116) — push-context feedback log (#2095)
One row per page the brain volunteers (op / reflex / watch channels).
"Used" is DERIVED, never written: pages.last_retrieved_at > volunteered_at
(the existing bumpLastRetrievedAt write-back is the open/cite signal), so
there is no second tracking path. session_id/turn are nullable
caller-supplied attribution; rationale is a deterministic template string,
never raw conversation text.
- Migration v116 (idempotent) + mirrors in src/schema.sql +
src/core/pglite-schema.ts + regenerated schema-embedded.ts (regen also
folds in pre-existing comment-only drift from the v114 links edits).
- src/core/context/volunteer-events.ts: insertVolunteerEvents (ONE
multi-row parameterized INSERT — never per-row awaited round-trips) +
purgeStaleVolunteerEvents (90-day GC, returns 0 on pre-v116 brains).
- Dream cycle purge phase prunes stale events alongside op_checkpoints /
brainstorm checkpoints / batch-retry audit files.
- RLS on Postgres comes from the v35 auto_rls_on_create_table event
trigger (the same mechanism that covered v110 page_aliases and v115
op_checkpoint_paths); the volunteer Postgres e2e pins it.
- No ::jsonb anywhere; no bootstrap probe needed (nothing references the
table pre-creation; writers guard with try/catch).
Tests: v116 shape + columns + indexes + live insert/purge round-trip on
PGLite (test/migrate.test.ts, 161 pass); schema-bootstrap-coverage green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(context): multi-turn window extraction + confidence-scored volunteer core (#2095)
- entity-salience.ts: extractCandidatesFromWindow(turns) — runs the existing
per-turn extractor across the last N turns (oldest→newest), merges by the
normalizeAlias form with occurrence/newest-turn/user-mention metadata, and
orders by salience (recency > frequency > user-role) so the MAX_CANDIDATES
cap drops stale assistant chatter, not the entity the user just named.
Closes the filed assistant-introduced-entities recall TODO; true pronoun
coreference (never-named antecedents) stays out of scope.
- retrieval-reflex.ts: ReflexPointer gains source_id + arm + confidence +
matchedNorm. ARM_CONFIDENCE (alias 0.9 / title 0.8 / slug-suffix 0.6)
lives next to the arm definitions so identity and score can't drift.
Arm-2 provenance is classified in JS (codex D8 — the combined OR can't
report which predicate matched). Federated sourceIds[] scope (alias arm
loops per source; arm 2 uses source_id = ANY — no engine-interface
change). Suppression gains 'slug-only' mode (codex D7, REQUIRED for
windowing): the legacy title-whole-word rule would suppress every entity
merely MENTIONED in a prior window turn, breaking the feature by
construction — slugs only enter context when a pointer/page was actually
surfaced. Default stays 'slug-and-title' for the window=1 legacy path.
- volunteer.ts (new): parseWindow (lenient user:/assistant: prefixes, CRLF,
unprefixed → one user turn), volunteerContext (zero-LLM: extract →
resolve → +0.05 multi-turn/newest-turn boost → min_confidence 0.7 gate →
cap 3/5; deterministic rationale strings, never raw conversation text),
and volunteerUsageStats (per-arm/channel precision from the
last_retrieved_at join, labeled approximate — 5-min throttle false
negatives, unrelated-read false positives; codex D9).
Tests: 35 green across volunteer-context (window parsing, pronoun follow-up
via assistant-introduced entity, confidence gating, slug-only suppression,
takes-fence privacy, multi-source scope, caps, stats join math) +
retrieval-reflex back-compat + resolve-ipc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ops): volunteer_context op — CLI (stdin) + MCP, drained event sink (#2095)
New read-scope op on the contract surface (CLI `gbrain volunteer-context`
with stdin → window, MCP tool for free): takes a rolling conversation
window, returns confidence-gated page pointers with rationales + synopses.
`window` is optional-unless-stats (validated in the handler, codex D9);
`stats: true` returns the volunteered-vs-used precision summary, labeled
APPROXIMATE (the 5-min last-retrieved throttle and unrelated reads both
bias the join). Source scope threads through sourceScopeOpts — federated
grants narrow the volunteer to the granted sources.
Event logging is fire-and-forget through a new `volunteer-events`
background-work sink (volunteer-events.ts, mirrors last-retrieved: tracked
dangling promise set + bounded drain + snapshot-drop on timeout so a
long-lived process never accumulates ghosts). ONE batched INSERT per call,
drained on every exit path by the commit-1 drain hoist; failure never
fails the op (pinned by an injected failing-engine test).
cli formatResult renders both shapes (pointer lines with confidence/arm/
rationale; the stats summary with per-arm precision).
Tests: op contract surface, window-required validation, sink round-trip
with session_id/turn attribution, failing-engine fail-open, federated
grant scoping, stats mode (26 green on PGLite) + a real-Postgres e2e
proving the op + sink + stats join AND that context_volunteer_events has
RLS enabled (keeps the auto-RLS event-trigger mechanism honest for v116).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(context): reflex consumes the rolling window + ambient-channel logging (#2095)
The default-on retrieval reflex now extracts entities from the last N turns
(retrieval_reflex_window_turns, default 4; env
GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS; window=1 reproduces the legacy
current-turn-only behavior exactly). assemble() passes the recent
user/assistant turns (hard cap 12); the reflex slices to the configured
window. Assistant-introduced entities and "what did she invest in?"
follow-ups whose antecedent was NAMED in the window now surface pointers —
the issue's "zero agent-initiated queries" success criterion on the
ambient path.
Under windowing, suppression switches to slug-only (codex D7): the legacy
title-whole-word rule would suppress every entity merely MENTIONED in a
prior window turn, breaking the feature by construction. Slugs only enter
prior context when a pointer/page was actually surfaced, so
already-surfaced pages still suppress. The suppression mode flows through
all three resolver rungs (host opts, serve IPC request, direct Postgres).
Ambient-channel feedback (codex D11): the server-side resolver paths
(serve IPC + direct Postgres) log volunteered pointers with
channel: 'reflex' through the drained volunteer-events sink, so
`gbrain volunteer-context --stats` measures the default-on path where most
volunteering happens. Host-injected resolvers (no gbrain engine) can't
log — documented gap. Precision gates, 1.5s ceiling, fail-open, and the
pointer cap are unchanged.
Tests: prev-assistant-turn entity fires; window=1 legacy parity; slug-only
vs already-surfaced suppression; throwing resolver stays fail-open
(16 green).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(cli): gbrain watch — push transport over stdin (#2095)
The issue's headline: the brain volunteers pages as the conversation flows,
instead of waiting to be asked. `some-transcript-feed | gbrain watch` reads
turns line-by-line ('user:'/'assistant:' prefixes set the role; unprefixed
lines are user turns), keeps a rolling window (--window-turns, default 4),
and streams confidence-gated pointers with rationales to stdout (--json for
JSONL). Session dedupe rides the core's slug-only suppression — a slug is
volunteered at most once per session. Events log on channel 'watch' with
session_id + turn through the drained sink.
Lifecycle: watch BLOCKS in the stdin iteration (like `jobs work`) — an
interactive TTY stays alive until Ctrl-C/Ctrl-D, piped input ends at EOF —
so it is deliberately NOT in DAEMON_COMMANDS (reverts the commit-1
placeholder): when main() resolves the work is over, the CLI_ONLY finally
drains volunteer events via drainThenDisconnect, and the entrypoint
flush-exit ends the process. Keeping it in the daemon set would have made
the piped EOF path hang on lingering sockets — the exact #2084 class.
SIGINT closes the stream and flows through the same drain path instead of
killing mid-write. Per-turn resolution failures are fail-open (the stream
never dies on a transient DB error).
Full wiring (eng-review D12): CLI_ONLY + CLI_ONLY_SELF_HELP (WATCH_HELP) +
THIN_CLIENT_REFUSED_COMMANDS (thin clients use the volunteer_context MCP
op) + main --help entry.
Tests: 18 green — help, per-turn volunteering + clean EOF return, rolling
window via assistant-introduced entity, session dedupe, --json shape with
turn attribution, channel-watch event rows, --min-confidence gate, CRLF/
blank tolerance, daemon-gate semantics. Live smoke: piped `gbrain watch`
on a fresh PGLite brain exits 0 at EOF with no force-exit banner.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: KEY_FILES + push-context guide + TODOS for the #2084/#2095 wave
- docs/architecture/KEY_FILES.md (current-state): context entries gain the
window extractor, arm provenance/confidence, suppression modes, volunteer
+ volunteer-events modules; background-work entry now lists FIVE sinks and
the drainThenDisconnect owner-disconnect contract; new entries for
src/core/cli-force-exit.ts (the exit contract) and src/commands/watch.ts.
- docs/guides/push-context.md (new): the three channels (reflex/op/watch),
the confidence model, CLI usage, config keys, and the approximate-stats
caveat. Linked from CLAUDE.md's reference map.
- CLAUDE.md: ops line mentions volunteer_context + the guide link;
bun run build:llms regenerated in the same commit (freshness test green).
- TODOS.md: #2095 deferrals filed (SSE push channel, policy skill + doctor
check, structured messages[] param); the #1981 entity-detection TODO
narrowed (window extraction covered assistant-introduced entities +
named-antecedent follow-ups); the drain-hoist P3 marked DONE by the
#2084 wave.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(e2e): truncate context_volunteer_events in setupDB (#2095)
The new feedback-log table wasn't in ALL_TABLES, so volunteered-event rows
persisted across e2e runs on a reused database and poisoned count/stats
assertions in volunteer-context-postgres on the second run. No FK to pages
(slug join), so position before pages is for hygiene only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cli): own the exit verdict — never trust ambient process.exitCode (#2084)
Caught by the full unit suite: `gbrain apply-migrations` on PGLite started
exiting 99. Root cause: PGLite's Emscripten runtime writes the WASM
backend's proc_exit status into process.exitCode (initdb at create-time,
the postmaster at close-time — `exitCode=status` in pglite's dist), and
the writes land ASYNCHRONOUSLY, outside any snapshot/restore window around
create/close (a guarded attempt verified this). The pre-#2084 success path
never read process.exitCode, so the pollution was invisible; the new
deliberate flush-exit propagated it faithfully.
Fix: gbrain records its own verdict. setCliExitCode(n)/getCliExitCode() in
cli-force-exit.ts — every gbrain-owned exit-code assignment routes through
the setter (still mirrored to process.exitCode for outside readers), and
both exit paths (entrypoint flushStdoutThenExit + the drainThenDisconnect
hard-deadline backstop) read the getter. Swept all assignment sites:
cli.ts (op error, friction, claw-test, smoke-test, eval runners, status,
import errors) + reindex/transcripts/brainstorm/frontmatter/autopilot.
Also updates the v0.42.20 structural pins to the drainThenDisconnect shape
(ordering invariant asserted INSIDE the helper + >=8 helper call sites,
superseding the two-inline-pairs assertion).
Verified: apply-migrations spawn test green; `init --migrate-only` exits 0;
an errored op still exits 1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: re-pin the teardown-arming invariant at its post-#2084 home
Master's v0.42.41.0 triage wave and the #2084 wave fixed the same
pre-armed-timer bug independently; the merge keeps #2084's shape (arming
inside the shared drainThenDisconnect helper, covering all 8 exit paths).
The structural pin now asserts the same invariant — no pre-try arming;
gated, unref'd, before-drain, cleared — at the helper.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: coverage for ambient reflex-channel logging + watch window/cap flags
Ship coverage audit (85%, gate PASS) named five gaps; the two substantive
cheap ones close here: the codex-D11 logChannel='reflex' path now has a
behavioral pin (events land on channel 'reflex' through the drained sink;
no logChannel → no events), and gbrain watch's --window-turns / --max-pages
flags are exercised (turn-1 attribution under window=1; cap to one page).
Remaining flagged-not-blocking: the wallclock-timeout branch (untestable
without >10s real-clock flake — same rationale as the arming pin),
formatResult's volunteer case (module-private), and the cycle purge wiring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: close the remaining plan-audit gaps — formatResult rendering + watch SIGINT
formatResult exported for tests (same import-safety contract as cliAliases);
test/cli-format-volunteer.test.ts pins the pointer lines, empty-gate message,
and approximate stats summary. test/watch-command.test.ts gains a real
subprocess SIGINT test: piped stdin that never reaches EOF, SIGINT mid-stream,
assert exit 0 with no force-exit banner — the drain-then-exit lifecycle under
the actual signal, not just the shared exit path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: doctor's FAIL verdict was zeroed by the owned exit — sweep stragglers + class pin
The merged-state suite caught it: doctor --fast --json reported FAIL but
exited 0. Master's v0.42.41.0 brought raw `process.exitCode =` writes
(doctor.ts hasFail ternary, extract.ts) that the #2084 verdict-owning exit
silently zeroes — getCliExitCode() deliberately never reads ambient
process.exitCode (the PGLite-Emscripten pollution defense), so any setter
that bypasses setCliExitCode reports success on failure.
Swept both sites and added the structural class pin: a test greps src/ for
raw `process.exitCode =` outside cli-force-exit.ts, so the next merge that
introduces one fails loudly instead of lying about exit codes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.43.0)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: quarantine the watch SIGINT subprocess test to the serial lane
The parallel unit shards flake on concurrent CLI subprocess spawns (failed
at 7ms in-suite, green solo) — same isolation rationale as
apply-migrations-pglite-spawn.serial.test.ts and #2141's R3 quarantine.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: update project documentation for v0.42.43.0
Post-ship doc verification against the release diff (#2095 push-based
context + #2084 superset hardening), with a cross-model doc review:
- push-context.md: version tag corrected to v0.42.43.0; per-call knobs
now cover prior_context/days and watch's flag surface accurately;
feedback-log writes described as best-effort; synopsis fence-strip
described as unconditional.
- CLAUDE.md: stale operation count (~47 -> ~90); volunteer_context
release reference corrected to v0.42.43.0.
- KEY_FILES.md: ci-local entry rewritten to current topology (4-shard
parallel default, four Postgres services, transaction-mode PgBouncer
+ GBRAIN_PGBOUNCER_URL/_DIRECT_URL exports); stale E2E file counts
dropped from the selector entry.
- TESTING.md: inventory entries for the new #2084 structural pins
(cli-exit-verdict-pin, cli-pipe-truncation), the push-context test
suite (volunteer-context, watch-command, watch-sigint.serial,
cli-format-volunteer), migrate v117 coverage, and the two new E2E
files (pgbouncer-teardown env gating, volunteer-context-postgres RLS
pin); check:all row corrected (not a superset of verify).
- AGENTS.md + RELEASING.md: ci:local descriptions updated to the
sharded + pooler topology.
- CHANGELOG (wording only, entry preserved): "retrieved" instead of
"opened" for the used-signal, pooler scoped to the local CI gate,
feedback log labeled best-effort.
- llms-config.ts: index the new push-context guide; bundles
regenerated (build:llms) and freshness test green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(test): correct the v116 reference — the table shipped as migration v117
* fix: pre-landing review hardening — federated alias parallelism, trust-boundary clamps, shared protocol helpers (#2095)
Five specialist reviewers (testing/maintainability/security/performance/
data-migration) on the reconciled diff; every finding applied:
Performance: the alias arm now resolves all granted sources CONCURRENTLY
(a federated caller paid M sequential RTTs per turn — ~355ms at 5 sources
cross-region, inside the reflex's 1.5s budget); watch's session dedupe is
O(1) Set membership instead of a monotonically growing priorContext string
(O(T²) over a long-lived session); getWindowTurns iterates from the tail
(per-turn cost no longer grows with session length); the resolver's
provenance maps fold into the existing candidate pass.
Security: volunteer_context clamps caller-supplied attribution at the trust
boundary — session_id capped at 256 chars (a read-scoped token could bank
~1MiB TEXT per request, retained 90 days), turn logged only when a safe
integer (a non-integer threw inside the batched INSERT and silently dropped
the whole batch). The privacy comments now state precisely what rationale
may contain (the matched entity's surface form — which by construction
resolved to an existing alias/title/slug — never free conversation text).
Maintainability: TURN_PREFIX_RE + formatVolunteeredPage exported from
volunteer.ts and shared by watch/cli (the two surfaces can no longer
drift); volunteerEventRowsFrom is the single VolunteerEventRow assembly
site for all three channels; watch's window default now honors the same
retrieval_reflex_window_turns config knob the reflex reads; the stale
pre-v116 comments swept to pre-v117.
Testing: the two flake-class CRITICALs fixed (pipe test asserts the
backstop banner instead of a cold-CI-hostile 9s wall bound; the SIGINT test
waits on watch's new machine-readable ready line instead of a fixed 15s
sleep — 2.5s and deterministic now); new coverage for the sink's timeout
branch + ghost-reference drop, watch per-turn fail-open, untrusted knob
clamps (min_confidence/max_pages/days), window-cap ordering (newest user
mention survives), serve-IPC suppression passthrough + channel=reflex
logging, windowTurnCount edge semantics, and structural pins for the sink
registration + cycle purge wiring. The exit-verdict pin's grep is now
operator/whitespace-tolerant.
Deferred with TODOs: resolver index shapes for the per-turn query;
batched first-prune after a long dream-cycle gap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(context): red-team hardening — pre-cap dedupe, delivery-side reflex logging, window clamp
Four red-team findings on the #2095 push-context surface:
- RT1 starvation: watch's session-dedupe Set filtered AFTER volunteerContext's
cap, so a recurring already-pushed entity burned cap slots every turn and
starved fresh pages behind it. VolunteerOpts.excludeSlugs now skips inside
the pointer loop BEFORE the confidence gate and the cap.
- RT3 honest stats: reflex-channel event logging moved from inside the
resolver to the DELIVERY point — serve's resolve-IPC onDelivered hook fires
only after the response write succeeds, and buildReflexAddition logs only
after the per-turn timeout admits the block. A block the client's 250ms
budget abandoned was never injected and no longer counts as volunteered.
(logChannel resolver opt removed; logDeliveredReflexPointers is the seam.)
- RT5 unbounded window: --window-turns is clamped to [1, 64] so a config typo
can't reintroduce the re-scan-everything-per-turn cost class.
- RT2/RT4 documented + filed: PGLite watch connection monopoly (WATCH_HELP,
push-context guide, TODO to route watch via serve IPC); host-resolver
suppression contract at ResolveEntitiesFn (TODO for a capability gate).
Tests: starvation guard (watch + volunteerContext unit), window clamp floor +
ceiling, delivery-side logging (helper writes channel=reflex through the
drained sink; bare resolver writes nothing; empty list no-op), IPC wiring test
rewired to onDelivered. KEY_FILES.md + push-context.md updated; build:llms run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(context): env-plane window knob works config-less; harden two gateway-state-leak victims
Three CI-only check failures, two root causes:
1. windowTurnCount ignored GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS when
loadConfig() returned null (no config file AND no DATABASE_URL — a clean
CI shard with no brain). loadConfig drops its env→config mapping in that
case, so the documented escape hatch silently died and the window fell
back to 4 → windowed extraction widened when the test set window=1 →
prior-turn entity leaked. Fixed: read the env var directly in
windowTurnCount, mirroring reflexEnabled's direct process.env read. This
is a real product bug, not just a test artifact — any config-less host
using the env hatch was affected. Regression test pins it.
2. sync-cost-preview + doctor-federation-health failed only IN-SHARD: a
sibling test configured a non-legacy (ZeroEntropy 1280-d / $0.05) gateway
and never reset it. The legacy-embedding preload only restores the
OpenAI/1536 default when the gateway slot is EMPTY, so a non-empty foreign
config survives into the next file — and a file's beforeAll runs BEFORE
the preload's restoring beforeEach, so federation-health built a
vector(1280) column and its 1536-d fixture hit CheckExpectedDim. My new
test files reshuffled the deterministic file→shard assignment, exposing
this latent ordering bug. Hardened both victims to establish the gateway
state they assert (sync-cost-preview resets to the unconfigured fallback;
federation-health pins legacy 1536 before initSchema) so they're
order-independent. Verified against a simulated leaker run before them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(context): use withEnv() in the window env-hatch test (test-isolation guard)
The regression test added in 82cc7fff mutated process.env directly, which
check:test-isolation (R1) forbids — use the withEnv() helper that restores on
exit, same as the rest of this file. Behavior identical; guard green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
co-authored by
Claude Fable 5
Claude Fable 5
parent
4ee530f3c5
commit
a81f7e05e8
@@ -0,0 +1,34 @@
|
||||
/**
|
||||
* #2084 class pin — every exit-code write in src/ routes through
|
||||
* setCliExitVerdict.
|
||||
*
|
||||
* A RAW `process.exitCode = N` is silently ZEROED by the deliberate
|
||||
* flush-exit: currentExitCode() reads only gbrain's owned verdict (the
|
||||
* PGLite-Emscripten-pollution defense), so a command that bypasses the
|
||||
* setter reports success on failure. Caught live twice in one day: doctor's
|
||||
* FAIL path exited 0 after a merge introduced a raw write. Runtime variants
|
||||
* are pinned in test/cli-finish-teardown.test.ts; this is the structural
|
||||
* guard that catches the NEXT raw write at review time.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { execSync } from 'child_process';
|
||||
|
||||
describe('exit-verdict ownership — no raw process.exitCode assignments', () => {
|
||||
test('every exit-code write in src/ routes through setCliExitVerdict', () => {
|
||||
// Two legitimate writers are exempt:
|
||||
// - cli-force-exit.ts: setCliExitVerdict's own mirror-write.
|
||||
// - pglite-engine.ts preservingProcessExitCode: #2141's containment
|
||||
// RESTORE around PGlite.create() — it keeps the GLOBAL tidy for
|
||||
// external readers and is explicitly not a verdict write (the owned
|
||||
// channel never reads process.exitCode).
|
||||
// Whitespace/operator-tolerant: catches `process.exitCode=1`,
|
||||
// `process.exitCode ??= 1`, `process.exitCode ||= 1`, and the bracket
|
||||
// form — every shape is equally zeroed by the owned-verdict read.
|
||||
const hits = execSync(
|
||||
String.raw`grep -rnE "process(\.|\[')exitCode('\])?[[:space:]]*([?|&]{2})?=[^=]" src --include='*.ts' | grep -v "core/cli-force-exit.ts" | grep -v "core/pglite-engine.ts" || true`,
|
||||
{ encoding: 'utf-8', cwd: new URL('..', import.meta.url).pathname },
|
||||
).trim();
|
||||
expect(hits).toBe('');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,67 @@
|
||||
/**
|
||||
* v0.43 (#2095) — formatResult's volunteer_context human rendering
|
||||
* (ship coverage G3): pointer lines with confidence/arm/rationale,
|
||||
* the empty-result message, and the approximate stats summary.
|
||||
*/
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { formatResult } from '../src/cli.ts';
|
||||
|
||||
describe('formatResult — volunteer_context', () => {
|
||||
test('renders pointer lines with confidence, arm, rationale, and synopsis', () => {
|
||||
const out = formatResult('volunteer_context', {
|
||||
pages: [
|
||||
{
|
||||
slug: 'people/alice-example',
|
||||
source_id: 'default',
|
||||
display: 'Alice Example',
|
||||
confidence: 0.85,
|
||||
arm: 'title',
|
||||
rationale: 'exact title match "Alice Example"; mentioned in the newest turn',
|
||||
synopsis: 'Alice is an early founder.',
|
||||
},
|
||||
],
|
||||
count: 1,
|
||||
window_turns: 3,
|
||||
});
|
||||
expect(out).toContain('Alice Example → people/alice-example (0.85, title)');
|
||||
expect(out).toContain('exact title match');
|
||||
expect(out).toContain('Alice is an early founder.');
|
||||
});
|
||||
|
||||
test('empty result explains the confidence gate', () => {
|
||||
const out = formatResult('volunteer_context', { pages: [], count: 0, window_turns: 1 });
|
||||
expect(out).toContain('Nothing volunteered');
|
||||
expect(out).toContain('confidence gate');
|
||||
});
|
||||
|
||||
test('stats mode renders totals, per-arm precision, and the approximate note', () => {
|
||||
const out = formatResult('volunteer_context', {
|
||||
days: 30,
|
||||
approximate: true,
|
||||
note: 'approximate: "used" = pages.last_retrieved_at > volunteered_at.',
|
||||
total_volunteered: 4,
|
||||
total_used: 3,
|
||||
by_arm: [
|
||||
{ match_arm: 'alias', channel: 'reflex', volunteered: 2, used: 2, precision: 1 },
|
||||
{ match_arm: 'title', channel: 'op', volunteered: 2, used: 1, precision: 0.5 },
|
||||
],
|
||||
});
|
||||
expect(out).toContain('last 30 day(s)');
|
||||
expect(out).toContain('approximate');
|
||||
expect(out).toContain('total: 4 volunteered, 3 used');
|
||||
expect(out).toContain('alias/reflex: 2/2 used (precision 1)');
|
||||
expect(out).toContain('title/op: 1/2 used (precision 0.5)');
|
||||
});
|
||||
|
||||
test('stats mode with zero events prints the empty-window line', () => {
|
||||
const out = formatResult('volunteer_context', {
|
||||
days: 7,
|
||||
approximate: true,
|
||||
note: 'approximate',
|
||||
total_volunteered: 0,
|
||||
total_used: 0,
|
||||
by_arm: [],
|
||||
});
|
||||
expect(out).toContain('no volunteer events in the window');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,47 @@
|
||||
/**
|
||||
* v0.43 (#2084) — real-CLI pipe completeness pin (the incident #1959 class).
|
||||
*
|
||||
* The synthetic flush-mechanism coverage lives in
|
||||
* test/flush-then-exit-harness.test.ts (4MB late-reader byte-complete pin).
|
||||
* This file keeps the IMPLEMENTATION-AGNOSTIC check: the actual CLI, run the
|
||||
* way agents run it (piped stdout), produces complete, parseable, byte-stable
|
||||
* output and exits deliberately — well under the teardown backstop.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { spawnSync } from 'child_process';
|
||||
import { join, resolve } from 'path';
|
||||
|
||||
const REPO = resolve(import.meta.dir, '..');
|
||||
const CLI = join(REPO, 'src', 'cli.ts');
|
||||
|
||||
describe('cli pipe completeness — deliberate exit never truncates piped stdout (#2084)', () => {
|
||||
test('real CLI: --tools-json over a pipe is complete, parseable, byte-stable, and prompt', () => {
|
||||
const run = () => {
|
||||
const t0 = Date.now();
|
||||
const res = spawnSync('bun', [CLI, '--tools-json'], {
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
encoding: 'utf-8',
|
||||
timeout: 60_000,
|
||||
env: { ...process.env, GBRAIN_SKIP_STARTUP_HOOKS: '1' },
|
||||
maxBuffer: 64 * 1024 * 1024,
|
||||
});
|
||||
return { stdout: res.stdout ?? '', stderr: res.stderr ?? '', status: res.status, ms: Date.now() - t0 };
|
||||
};
|
||||
const first = run();
|
||||
expect(first.status).toBe(0);
|
||||
expect(Buffer.byteLength(first.stdout, 'utf-8')).toBeGreaterThan(16 * 1024);
|
||||
// Truncated JSON does not parse — the strongest single-run completeness check.
|
||||
const parsed = JSON.parse(first.stdout);
|
||||
expect(Array.isArray(parsed)).toBe(true);
|
||||
// Deliberate exit, not the teardown backstop. A wall-clock bound is flaky
|
||||
// on cold CI (bun parse alone runs 10-20s there) — the backstop's banner
|
||||
// is the truthful signal, same assertion the pgbouncer e2e uses.
|
||||
expect(first.stderr).not.toContain('force-exiting');
|
||||
expect(first.stderr).not.toContain('did not return within');
|
||||
|
||||
const second = run();
|
||||
expect(second.status).toBe(0);
|
||||
expect(second.stdout).toBe(first.stdout);
|
||||
}, 180_000);
|
||||
});
|
||||
@@ -82,4 +82,20 @@ describe('shouldForceExitAfterMain — daemon survival gate', () => {
|
||||
expect(shouldForceExitAfterMain(['serves'])).toBe(true);
|
||||
expect(shouldForceExitAfterMain(['serve-cluster'])).toBe(true);
|
||||
});
|
||||
|
||||
test('awaited long-runners exit deliberately when their handler resolves', () => {
|
||||
// `jobs work`, `jobs watch --follow`, `autopilot`, and `gbrain watch`
|
||||
// (#2095) all BLOCK inside their awaited handler until done — when
|
||||
// main() resolves for them, the work is over and the deliberate exit is
|
||||
// correct (v0.43 #2084 contract). Only commands that RETURN from main()
|
||||
// while the event loop carries the daemon (`serve`) belong in
|
||||
// DAEMON_COMMANDS — `watch` blocks in its stdin iteration, so piped EOF
|
||||
// must flow through the flush-exit instead of hanging on lingering
|
||||
// sockets.
|
||||
expect(shouldForceExitAfterMain(['jobs', 'work'])).toBe(true);
|
||||
expect(shouldForceExitAfterMain(['jobs', 'watch', '--follow'])).toBe(true);
|
||||
expect(shouldForceExitAfterMain(['autopilot'])).toBe(true);
|
||||
expect(shouldForceExitAfterMain(['watch'])).toBe(true);
|
||||
expect(shouldForceExitAfterMain(['watch', '--json'])).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -26,7 +26,10 @@ describe('resolve IPC', () => {
|
||||
test('round-trip: client gets the pointer block the server returns', async () => {
|
||||
const dir = tmpDir();
|
||||
const sock = resolveSocketPath(dir);
|
||||
const block: PointerBlock = { pointers: [{ display: 'Alice', slug: 'people/alice', synopsis: 'x' }], text: 'BLOCK' };
|
||||
const block: PointerBlock = {
|
||||
pointers: [{ display: 'Alice', slug: 'people/alice', source_id: 'default', synopsis: 'x', arm: 'alias', confidence: 0.9 }],
|
||||
text: 'BLOCK',
|
||||
};
|
||||
const server = await startResolveIpcServer(sock, async (req) => {
|
||||
expect(req.candidates[0].query).toBe('Alice');
|
||||
return block;
|
||||
|
||||
@@ -8,11 +8,24 @@
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { configureGateway } from '../src/core/ai/gateway.ts';
|
||||
import { checkFederationHealth } from '../src/commands/doctor.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
// Pin the legacy OpenAI/1536 embedding shape BEFORE initSchema builds the
|
||||
// content_chunks vector column. beforeAll runs before the legacy-embedding
|
||||
// preload's restoring beforeEach fires, so the gateway here is whatever the
|
||||
// PRIOR file in this shard left it — if that file configured a 1280-d model
|
||||
// and didn't reset, initSchema would build a vector(1280) column and the
|
||||
// 1536-d coverage fixture below would fail CheckExpectedDim. Establish the
|
||||
// dimension this file's fixtures assume rather than inheriting it.
|
||||
configureGateway({
|
||||
embedding_model: 'openai:text-embedding-3-large',
|
||||
embedding_dimensions: 1536,
|
||||
env: { ...process.env },
|
||||
});
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
|
||||
@@ -49,6 +49,9 @@ const ALL_TABLES = [
|
||||
'page_versions',
|
||||
'ingest_log',
|
||||
'files',
|
||||
// v0.43 (#2095): volunteered-context feedback log — no FK to pages (slug
|
||||
// join), but stale rows poison stats/count assertions across runs.
|
||||
'context_volunteer_events',
|
||||
'pages', // last because of foreign keys
|
||||
'config',
|
||||
'minion_attachments',
|
||||
|
||||
@@ -0,0 +1,148 @@
|
||||
/**
|
||||
* v0.43 (#2084 / eng-review TD1) — PgBouncer transaction-mode teardown E2E.
|
||||
*
|
||||
* Three consecutive waves (#1972 → #2015 → #2084) fixed pooler-teardown bugs
|
||||
* that were verified only against one production deployment, because CI had
|
||||
* no transaction-mode pooler. This file pins the bug CLASS, not exact
|
||||
* timings: a CLI op against a txn-mode pooled URL must
|
||||
*
|
||||
* 1. exit zero with intact stdout, and
|
||||
* 2. NOT ride the 10s hard-deadline backstop (the
|
||||
* "[cli] engine.disconnect() did not return within 10000ms" banner is
|
||||
* the smoking gun — pre-#2084 it printed on 100% of query-shaped ops).
|
||||
*
|
||||
* Topology: docker-compose.ci.yml runs `pgbouncer` (transaction mode) in
|
||||
* front of postgres-1. The test uses a DEDICATED database
|
||||
* (`gbrain_pgbouncer`) created via the direct URL, so it never races the
|
||||
* TRUNCATE-based fixtures any shard runs against `gbrain_test`.
|
||||
*
|
||||
* Gated by GBRAIN_PGBOUNCER_URL + GBRAIN_PGBOUNCER_DIRECT_URL — skips
|
||||
* gracefully outside the docker CI gate. Run manually:
|
||||
*
|
||||
* GBRAIN_PGBOUNCER_URL=postgresql://postgres:postgres@localhost:6543/gbrain_pgbouncer \
|
||||
* GBRAIN_PGBOUNCER_DIRECT_URL=postgresql://postgres:postgres@localhost:5434/gbrain_test \
|
||||
* bun test test/e2e/pgbouncer-teardown.test.ts
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'fs';
|
||||
import { join, resolve } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import postgres from 'postgres';
|
||||
import { PostgresEngine } from '../../src/core/postgres-engine.ts';
|
||||
|
||||
const POOLED_URL = process.env.GBRAIN_PGBOUNCER_URL;
|
||||
const DIRECT_ADMIN_URL = process.env.GBRAIN_PGBOUNCER_DIRECT_URL;
|
||||
const SKIP = !POOLED_URL || !DIRECT_ADMIN_URL;
|
||||
const describePooled = SKIP ? describe.skip : describe;
|
||||
|
||||
const REPO = resolve(import.meta.dir, '..', '..');
|
||||
const TEST_DB = 'gbrain_pgbouncer';
|
||||
const SLUG = 'test/pgbouncer-teardown-fixture';
|
||||
const MARKER = 'pgbouncer-teardown-marker-content-7c4f';
|
||||
|
||||
/** Direct URL pointing at the dedicated test database (same server). */
|
||||
function directTestDbUrl(): string {
|
||||
const u = new URL(DIRECT_ADMIN_URL!);
|
||||
u.pathname = `/${TEST_DB}`;
|
||||
return u.toString();
|
||||
}
|
||||
|
||||
async function runCli(
|
||||
args: string[],
|
||||
env: Record<string, string>,
|
||||
timeoutMs: number,
|
||||
): Promise<{ exitCode: number; stdout: string; stderr: string; wallMs: number }> {
|
||||
const t0 = Date.now();
|
||||
const proc = Bun.spawn(['bun', 'run', join(REPO, 'src', 'cli.ts'), ...args], {
|
||||
cwd: REPO,
|
||||
env: { ...process.env, ...env, GBRAIN_SKIP_STARTUP_HOOKS: '1' },
|
||||
stdout: 'pipe',
|
||||
stderr: 'pipe',
|
||||
});
|
||||
const killer = setTimeout(() => {
|
||||
try { proc.kill('SIGKILL'); } catch { /* already dead */ }
|
||||
}, timeoutMs);
|
||||
try {
|
||||
const [stdout, stderr, exitCode] = await Promise.all([
|
||||
new Response(proc.stdout).text(),
|
||||
new Response(proc.stderr).text(),
|
||||
proc.exited,
|
||||
]);
|
||||
return { exitCode, stdout, stderr, wallMs: Date.now() - t0 };
|
||||
} finally {
|
||||
clearTimeout(killer);
|
||||
}
|
||||
}
|
||||
|
||||
describePooled('pgbouncer txn-mode teardown (#2084 / TD1)', () => {
|
||||
let home: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
// Dedicated database on the same server, created via the DIRECT url
|
||||
// (CREATE DATABASE cannot run through a transaction-mode pooler).
|
||||
const admin = postgres(DIRECT_ADMIN_URL!, { max: 1 });
|
||||
try {
|
||||
await admin.unsafe(`DROP DATABASE IF EXISTS ${TEST_DB} WITH (FORCE)`);
|
||||
await admin.unsafe(`CREATE DATABASE ${TEST_DB}`);
|
||||
} finally {
|
||||
await admin.end({ timeout: 5 });
|
||||
}
|
||||
|
||||
// Schema + fixture via the direct connection (DDL stays off the pooler,
|
||||
// matching the production split-pool discipline).
|
||||
const eng = new PostgresEngine();
|
||||
await eng.connect({ engine: 'postgres', database_url: directTestDbUrl() });
|
||||
await eng.initSchema();
|
||||
await eng.putPage(SLUG, {
|
||||
type: 'note',
|
||||
title: 'PgBouncer teardown fixture',
|
||||
compiled_truth: MARKER,
|
||||
timeline: '',
|
||||
});
|
||||
await eng.disconnect();
|
||||
|
||||
// Brain config pointing the CLI at the POOLED url.
|
||||
home = mkdtempSync(join(tmpdir(), 'gbrain-pgbouncer-'));
|
||||
mkdirSync(join(home, '.gbrain'), { recursive: true });
|
||||
const pooled = new URL(POOLED_URL!);
|
||||
pooled.pathname = `/${TEST_DB}`;
|
||||
writeFileSync(
|
||||
join(home, '.gbrain', 'config.json'),
|
||||
JSON.stringify({ engine: 'postgres', database_url: pooled.toString() }) + '\n',
|
||||
);
|
||||
}, 240_000);
|
||||
|
||||
afterAll(() => {
|
||||
try { rmSync(home, { recursive: true, force: true }); } catch { /* best effort */ }
|
||||
});
|
||||
|
||||
test('op against the pooled URL exits clean — output intact, no force-exit banner', async () => {
|
||||
const env = { HOME: home, GBRAIN_HOME: home };
|
||||
const res = await runCli(['get', SLUG], env, 90_000);
|
||||
|
||||
if (res.exitCode !== 0 || /force-exiting/.test(res.stderr)) {
|
||||
console.error('--- stdout ---\n' + res.stdout);
|
||||
console.error('--- stderr ---\n' + res.stderr);
|
||||
}
|
||||
expect(res.exitCode).toBe(0);
|
||||
// Output is complete — the #1959 truncation class.
|
||||
expect(res.stdout).toContain(MARKER);
|
||||
// The smoking gun: pre-#2084 the hard-deadline banner printed every time
|
||||
// a query-shaped op ran against a txn-mode pooler.
|
||||
expect(res.stderr).not.toMatch(/force-exiting/);
|
||||
expect(res.stderr).not.toMatch(/did not return within/);
|
||||
// Generous CLASS bound (cold bun parse on CI is 10-20s): the op itself is
|
||||
// milliseconds; anything that ALSO waited out a 10s teardown backstop
|
||||
// lands well past this.
|
||||
expect(res.wallMs).toBeLessThan(60_000);
|
||||
}, 120_000);
|
||||
|
||||
test('second run (warm schema probe) also exits clean through the pooler', async () => {
|
||||
const env = { HOME: home, GBRAIN_HOME: home };
|
||||
const res = await runCli(['get', SLUG], env, 90_000);
|
||||
expect(res.exitCode).toBe(0);
|
||||
expect(res.stdout).toContain(MARKER);
|
||||
expect(res.stderr).not.toMatch(/force-exiting/);
|
||||
}, 120_000);
|
||||
});
|
||||
@@ -0,0 +1,97 @@
|
||||
/**
|
||||
* v0.43 (#2095) — volunteer_context on REAL Postgres (engine parity beyond
|
||||
* PGLite) + the RLS pin for the v117 table.
|
||||
*
|
||||
* The unit suite covers the volunteer core hermetically on PGLite; this file
|
||||
* proves the same op handler against pgvector Postgres: resolution arms,
|
||||
* the fire-and-forget event sink landing rows, the stats join, and that
|
||||
* context_volunteer_events (migration v117) has ROW LEVEL SECURITY enabled (the v35
|
||||
* auto_rls_on_create_table event trigger covers migration-created tables —
|
||||
* this is the assertion that keeps that mechanism honest for new tables).
|
||||
*
|
||||
* Gated by DATABASE_URL — skips gracefully without real Postgres.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { hasDatabase, setupDB, teardownDB, getEngine } from './helpers.ts';
|
||||
import { operationsByName } from '../../src/core/operations.ts';
|
||||
import {
|
||||
awaitPendingVolunteerEventWrites,
|
||||
_resetPendingVolunteerEventWritesForTests,
|
||||
} from '../../src/core/context/volunteer-events.ts';
|
||||
|
||||
const SKIP = !hasDatabase();
|
||||
const describePg = SKIP ? describe.skip : describe;
|
||||
|
||||
function mkCtx(engine: unknown, overrides: Record<string, unknown> = {}) {
|
||||
return {
|
||||
engine,
|
||||
config: {} as never,
|
||||
logger: { info: () => {}, warn: () => {}, error: () => {} } as never,
|
||||
dryRun: false,
|
||||
remote: false,
|
||||
sourceId: 'default',
|
||||
...overrides,
|
||||
} as never;
|
||||
}
|
||||
|
||||
describePg('volunteer_context on real Postgres (#2095)', () => {
|
||||
beforeAll(async () => {
|
||||
await setupDB();
|
||||
const engine = getEngine();
|
||||
await engine.putPage('people/alice-example', {
|
||||
type: 'person',
|
||||
title: 'Alice Example',
|
||||
compiled_truth: 'Alice is an early founder.',
|
||||
timeline: '',
|
||||
});
|
||||
}, 240_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await teardownDB();
|
||||
});
|
||||
|
||||
test('op volunteers, logs through the sink, and stats join works', async () => {
|
||||
_resetPendingVolunteerEventWritesForTests();
|
||||
const engine = getEngine();
|
||||
const op = operationsByName.volunteer_context;
|
||||
|
||||
const result = (await op.handler(mkCtx(engine), {
|
||||
window: 'user: who knows the seed market?\nassistant: Alice Example does.\nuser: ask her',
|
||||
session_id: 'e2e-pg',
|
||||
})) as any;
|
||||
expect(result.count).toBe(1);
|
||||
expect(result.pages[0].slug).toBe('people/alice-example');
|
||||
expect(result.pages[0].arm).toBe('title');
|
||||
|
||||
const { unfinished } = await awaitPendingVolunteerEventWrites(10_000);
|
||||
expect(unfinished).toBe(0);
|
||||
|
||||
const rows = await engine.executeRaw<{ slug: string; session_id: string }>(
|
||||
`SELECT slug, session_id FROM context_volunteer_events WHERE session_id = 'e2e-pg'`,
|
||||
[],
|
||||
);
|
||||
expect(rows.length).toBe(1);
|
||||
|
||||
// Mark the page as opened after volunteering → stats counts it used.
|
||||
await engine.executeRaw(
|
||||
`UPDATE pages SET last_retrieved_at = now() + interval '1 minute' WHERE slug = 'people/alice-example'`,
|
||||
[],
|
||||
);
|
||||
const stats = (await op.handler(mkCtx(engine), { stats: true })) as any;
|
||||
expect(stats.approximate).toBe(true);
|
||||
expect(stats.total_volunteered).toBeGreaterThanOrEqual(1);
|
||||
expect(stats.total_used).toBeGreaterThanOrEqual(1);
|
||||
}, 120_000);
|
||||
|
||||
test('RLS is enabled on context_volunteer_events (auto-RLS covers v117)', async () => {
|
||||
const engine = getEngine();
|
||||
const rows = await engine.executeRaw<{ relrowsecurity: boolean }>(
|
||||
`SELECT relrowsecurity FROM pg_class
|
||||
WHERE oid = 'public.context_volunteer_events'::regclass`,
|
||||
[],
|
||||
);
|
||||
expect(rows.length).toBe(1);
|
||||
expect(rows[0].relrowsecurity).toBe(true);
|
||||
}, 60_000);
|
||||
});
|
||||
@@ -250,3 +250,20 @@ describe('v0.41.8.0 #1340 — PGLite WASM init classifier', () => {
|
||||
expect(src).toMatch(/buildPgliteInitErrorMessage\(verdict, original\)/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('v0.42.43.0 #2095 — volunteer-events sink + cycle purge wiring (structural pins)', () => {
|
||||
test('volunteer-events registers a background-work drainer (order 4)', () => {
|
||||
// Deleting this registration would silently drop volunteer events on
|
||||
// every CLI exit with no behavioral test failing — same pin class as the
|
||||
// other four sinks above.
|
||||
const src = readFileSync('src/core/context/volunteer-events.ts', 'utf8');
|
||||
expect(src).toMatch(/registerBackgroundWorkDrainer\(\{[\s\S]*?name:\s*'volunteer-events'/);
|
||||
expect(src).toMatch(/order:\s*4/);
|
||||
});
|
||||
|
||||
test("the dream cycle's purge phase invokes purgeStaleVolunteerEvents and reports the count", () => {
|
||||
const src = readFileSync('src/core/cycle.ts', 'utf8');
|
||||
expect(src).toMatch(/purgeStaleVolunteerEvents\(engine\)/);
|
||||
expect(src).toMatch(/purged_volunteer_events_count/);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -2179,3 +2179,67 @@ describe('v112 — pages_links_extracted_at', () => {
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
// v0.43 (#2095): context_volunteer_events — push-based-context feedback log.
|
||||
describe('v117 — context_volunteer_events_table', () => {
|
||||
let engine: PGLiteEngine;
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
}, 60_000);
|
||||
afterAll(async () => { if (engine) await engine.disconnect(); }, 60_000);
|
||||
|
||||
test('v117 entry exists, named + idempotent', () => {
|
||||
const m = MIGRATIONS.find(x => x.version === 117);
|
||||
expect(m).toBeDefined();
|
||||
expect(m!.name).toBe('context_volunteer_events_table');
|
||||
expect(m!.idempotent).toBe(true);
|
||||
});
|
||||
|
||||
test('LATEST_VERSION is at or above 117', () => {
|
||||
expect(LATEST_VERSION).toBeGreaterThanOrEqual(117);
|
||||
});
|
||||
|
||||
test('table exists after initSchema with the documented columns', async () => {
|
||||
const rows = await engine.executeRaw<{ column_name: string; is_nullable: string }>(
|
||||
`SELECT column_name, is_nullable FROM information_schema.columns
|
||||
WHERE table_name = 'context_volunteer_events' ORDER BY ordinal_position`, [],
|
||||
);
|
||||
const byName = new Map(rows.map(r => [r.column_name, r.is_nullable]));
|
||||
for (const required of ['source_id', 'slug', 'confidence', 'match_arm', 'rationale', 'channel', 'volunteered_at']) {
|
||||
expect(byName.get(required)).toBe('NO');
|
||||
}
|
||||
// Caller-supplied attribution columns are nullable by contract (codex D9).
|
||||
expect(byName.get('session_id')).toBe('YES');
|
||||
expect(byName.get('turn')).toBe('YES');
|
||||
});
|
||||
|
||||
test('both source-scoped indexes exist after initSchema', async () => {
|
||||
const rows = await engine.executeRaw<{ indexname: string }>(
|
||||
`SELECT indexname FROM pg_indexes WHERE tablename = 'context_volunteer_events'`, [],
|
||||
);
|
||||
const names = rows.map(r => r.indexname);
|
||||
expect(names).toContain('context_volunteer_events_src_time_idx');
|
||||
expect(names).toContain('context_volunteer_events_src_slug_idx');
|
||||
});
|
||||
|
||||
test('insert + 90-day purge round-trip (purgeStaleVolunteerEvents)', async () => {
|
||||
const { insertVolunteerEvents, purgeStaleVolunteerEvents } = await import('../src/core/context/volunteer-events.ts');
|
||||
await insertVolunteerEvents(engine, [
|
||||
{ source_id: 'default', slug: 'people/alice-example', confidence: 0.9, match_arm: 'alias', rationale: 'alias match', channel: 'op' },
|
||||
{ source_id: 'default', slug: 'projects/acme-example', confidence: 0.8, match_arm: 'title', rationale: 'exact title', channel: 'reflex', session_id: 's1', turn: 3 },
|
||||
]);
|
||||
// Age one row past the TTL, leave the other fresh.
|
||||
await engine.executeRaw(
|
||||
`UPDATE context_volunteer_events SET volunteered_at = now() - interval '120 days'
|
||||
WHERE slug = 'projects/acme-example'`, [],
|
||||
);
|
||||
const purged = await purgeStaleVolunteerEvents(engine, 90);
|
||||
expect(purged).toBe(1);
|
||||
const left = await engine.executeRaw<{ slug: string }>(
|
||||
`SELECT slug FROM context_volunteer_events`, [],
|
||||
);
|
||||
expect(left.map(r => r.slug)).toEqual(['people/alice-example']);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -182,3 +182,258 @@ describe('context-engine assemble() — Retrieval Reflex integration', () => {
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('v0.43 (#2095) — rolling window extraction through assemble()', () => {
|
||||
const REFLEX_ON = { GBRAIN_RETRIEVAL_REFLEX: 'true' };
|
||||
|
||||
test('entity named ONLY in a previous assistant turn yields a pointer now', async () => {
|
||||
await withEnv(REFLEX_ON, async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Alice is a founder.');
|
||||
const ce = createGBrainContextEngine({
|
||||
workspaceDir: '/tmp/rr-test-ws-w1',
|
||||
resolveEntities: (candidates, opts) =>
|
||||
resolveEntitiesToPointers(engine, 'default', candidates, opts),
|
||||
});
|
||||
// Current turn is a pronoun follow-up; the antecedent was NAMED two
|
||||
// turns back by the ASSISTANT. Pre-window this never fired.
|
||||
const res = await ce.assemble({
|
||||
sessionId: 'w1',
|
||||
messages: [
|
||||
{ role: 'user', content: 'who should I talk to about the seed round?' },
|
||||
{ role: 'assistant', content: 'Alice Example led a similar round last year.' },
|
||||
{ role: 'user', content: 'what did she invest in?' },
|
||||
],
|
||||
});
|
||||
expect(res.systemPromptAddition).toContain('people/alice-example');
|
||||
});
|
||||
});
|
||||
|
||||
test('window=1 reproduces the legacy current-turn-only behavior', async () => {
|
||||
await withEnv({ ...REFLEX_ON, GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS: '1' }, async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Alice is a founder.');
|
||||
const ce = createGBrainContextEngine({
|
||||
workspaceDir: '/tmp/rr-test-ws-w2',
|
||||
resolveEntities: (candidates, opts) =>
|
||||
resolveEntitiesToPointers(engine, 'default', candidates, opts),
|
||||
});
|
||||
const res = await ce.assemble({
|
||||
sessionId: 'w2',
|
||||
messages: [
|
||||
{ role: 'assistant', content: 'Alice Example led a similar round last year.' },
|
||||
{ role: 'user', content: 'what did she invest in?' },
|
||||
],
|
||||
});
|
||||
// Current turn has no extractable entity; window=1 must NOT widen.
|
||||
expect(res.systemPromptAddition).not.toContain('people/alice-example');
|
||||
});
|
||||
});
|
||||
|
||||
test('windowed suppression is slug-only: a prior-turn MENTION does not suppress (codex D7)', async () => {
|
||||
await withEnv(REFLEX_ON, async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'A founder.');
|
||||
const ce = createGBrainContextEngine({
|
||||
workspaceDir: '/tmp/rr-test-ws-w3',
|
||||
resolveEntities: (candidates, opts) =>
|
||||
resolveEntitiesToPointers(engine, 'default', candidates, opts),
|
||||
});
|
||||
// "Alice Example" appears in a PRIOR turn (a bare mention — prior
|
||||
// context contains the TITLE). Under the legacy title rule the pointer
|
||||
// would be suppressed; slug-only windowing must still fire.
|
||||
const res = await ce.assemble({
|
||||
sessionId: 'w3',
|
||||
messages: [
|
||||
{ role: 'user', content: 'I met Alice Example yesterday' },
|
||||
{ role: 'assistant', content: 'How did the meeting with Alice Example go?' },
|
||||
{ role: 'user', content: 'she wants to invest — thoughts?' },
|
||||
],
|
||||
});
|
||||
expect(res.systemPromptAddition).toContain('people/alice-example');
|
||||
});
|
||||
});
|
||||
|
||||
test('windowed suppression still drops an already-surfaced page (slug in prior context)', async () => {
|
||||
await withEnv(REFLEX_ON, async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'A founder.');
|
||||
const ce = createGBrainContextEngine({
|
||||
workspaceDir: '/tmp/rr-test-ws-w4',
|
||||
resolveEntities: (candidates, opts) =>
|
||||
resolveEntitiesToPointers(engine, 'default', candidates, opts),
|
||||
});
|
||||
const res = await ce.assemble({
|
||||
sessionId: 'w4',
|
||||
messages: [
|
||||
{ role: 'assistant', content: 'Pointer: **Alice Example** → `people/alice-example` (use get_page)' },
|
||||
{ role: 'user', content: 'tell me more about Alice Example' },
|
||||
],
|
||||
});
|
||||
expect(res.systemPromptAddition).not.toContain('Brain pages mentioned this turn');
|
||||
});
|
||||
});
|
||||
|
||||
test('fail-open: a throwing resolver under windowing never breaks the turn', async () => {
|
||||
await withEnv(REFLEX_ON, async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'A founder.');
|
||||
const ce = createGBrainContextEngine({
|
||||
workspaceDir: '/tmp/rr-test-ws-w5',
|
||||
resolveEntities: async () => { throw new Error('resolver exploded'); },
|
||||
});
|
||||
const res = await ce.assemble({
|
||||
sessionId: 'w5',
|
||||
messages: [
|
||||
{ role: 'assistant', content: 'Alice Example is relevant here.' },
|
||||
{ role: 'user', content: 'ok tell me about her' },
|
||||
],
|
||||
});
|
||||
expect(res.systemPromptAddition).toContain('Live Context');
|
||||
expect(res.systemPromptAddition).not.toContain('Brain pages mentioned this turn');
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('ambient-channel event logging (codex D11 — accept-side logDeliveredReflexPointers)', () => {
|
||||
test('logDeliveredReflexPointers logs channel=reflex events through the drained sink', async () => {
|
||||
const { logDeliveredReflexPointers } = await import('../src/core/context/retrieval-reflex.ts');
|
||||
const { awaitPendingVolunteerEventWrites, _resetPendingVolunteerEventWritesForTests } =
|
||||
await import('../src/core/context/volunteer-events.ts');
|
||||
_resetPendingVolunteerEventWritesForTests();
|
||||
await engine.executeRaw('DELETE FROM context_volunteer_events').catch(() => {});
|
||||
await seed('people/alice-example', 'Alice Example', 'A founder.');
|
||||
|
||||
const block = await resolveEntitiesToPointers(
|
||||
engine,
|
||||
'default',
|
||||
extractCandidates('what do you think about Alice Example?'),
|
||||
{},
|
||||
);
|
||||
expect(block).not.toBeNull();
|
||||
logDeliveredReflexPointers(engine, block!.pointers);
|
||||
const { unfinished } = await awaitPendingVolunteerEventWrites(5_000);
|
||||
expect(unfinished).toBe(0);
|
||||
const rows = await engine.executeRaw<{ channel: string; slug: string; match_arm: string }>(
|
||||
'SELECT channel, slug, match_arm FROM context_volunteer_events',
|
||||
[],
|
||||
);
|
||||
expect(rows.length).toBe(1);
|
||||
expect(rows[0].channel).toBe('reflex');
|
||||
expect(rows[0].slug).toBe('people/alice-example');
|
||||
expect(rows[0].match_arm).toBe('title');
|
||||
});
|
||||
|
||||
test('the bare resolver logs nothing — delivery is the only logging seam', async () => {
|
||||
await engine.executeRaw('DELETE FROM context_volunteer_events').catch(() => {});
|
||||
await seed('people/alice-example', 'Alice Example', 'A founder.');
|
||||
const block = await resolveEntitiesToPointers(
|
||||
engine,
|
||||
'default',
|
||||
extractCandidates('about Alice Example'),
|
||||
{},
|
||||
);
|
||||
expect(block).not.toBeNull();
|
||||
const { awaitPendingVolunteerEventWrites } = await import('../src/core/context/volunteer-events.ts');
|
||||
await awaitPendingVolunteerEventWrites(5_000);
|
||||
const rows = await engine.executeRaw<{ channel: string }>('SELECT channel FROM context_volunteer_events', []);
|
||||
expect(rows.length).toBe(0);
|
||||
});
|
||||
|
||||
test('logDeliveredReflexPointers with an empty pointer list is a no-op', async () => {
|
||||
const { logDeliveredReflexPointers } = await import('../src/core/context/retrieval-reflex.ts');
|
||||
const { awaitPendingVolunteerEventWrites } = await import('../src/core/context/volunteer-events.ts');
|
||||
await engine.executeRaw('DELETE FROM context_volunteer_events').catch(() => {});
|
||||
logDeliveredReflexPointers(engine, []);
|
||||
await awaitPendingVolunteerEventWrites(5_000);
|
||||
const rows = await engine.executeRaw<{ channel: string }>('SELECT channel FROM context_volunteer_events', []);
|
||||
expect(rows.length).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('serve IPC wiring — suppression passthrough + reflex-channel logging (review hardening)', () => {
|
||||
test('the IPC round-trip honors slug-only suppression and logs channel=reflex', async () => {
|
||||
const { startResolveIpcServer, resolveViaIpc, resolveSocketPath, IPC_UNAVAILABLE } =
|
||||
await import('../src/core/context/resolve-ipc.ts');
|
||||
const { awaitPendingVolunteerEventWrites, _resetPendingVolunteerEventWritesForTests } =
|
||||
await import('../src/core/context/volunteer-events.ts');
|
||||
const { mkdtempSync, rmSync } = await import('fs');
|
||||
const { join } = await import('path');
|
||||
const { tmpdir } = await import('os');
|
||||
|
||||
_resetPendingVolunteerEventWritesForTests();
|
||||
await engine.executeRaw('DELETE FROM context_volunteer_events').catch(() => {});
|
||||
await seed('people/alice-example', 'Alice Example', 'A founder.');
|
||||
|
||||
const dir = mkdtempSync(join(tmpdir(), 'rr-ipc-'));
|
||||
const sock = resolveSocketPath(dir);
|
||||
// The SAME wiring shape src/mcp/server.ts uses for serve: forwards
|
||||
// suppression from the request; logging happens at DELIVERY via the
|
||||
// onDelivered hook (post-write), never inside the resolver.
|
||||
const { logDeliveredReflexPointers } = await import('../src/core/context/retrieval-reflex.ts');
|
||||
const server = await startResolveIpcServer(
|
||||
sock,
|
||||
(req) =>
|
||||
resolveEntitiesToPointers(engine, req.sourceId || 'default', req.candidates ?? [], {
|
||||
priorContextText: req.priorContextText,
|
||||
maxPointers: req.maxPointers,
|
||||
suppression: req.suppression,
|
||||
}),
|
||||
(block) => logDeliveredReflexPointers(engine, block.pointers),
|
||||
);
|
||||
expect(server).not.toBeNull();
|
||||
try {
|
||||
// slug-only suppression: a TITLE mention in prior context must NOT
|
||||
// suppress (the windowing contract), and the resolve must log.
|
||||
const block = await resolveViaIpc(sock, {
|
||||
candidates: extractCandidates('tell me about Alice Example'),
|
||||
priorContextText: 'earlier turn merely mentioned Alice Example',
|
||||
suppression: 'slug-only',
|
||||
});
|
||||
expect(block).not.toBe(IPC_UNAVAILABLE);
|
||||
expect(block).not.toBeNull();
|
||||
expect((block as { pointers: Array<{ slug: string }> }).pointers[0].slug).toBe('people/alice-example');
|
||||
|
||||
const { unfinished } = await awaitPendingVolunteerEventWrites(5_000);
|
||||
expect(unfinished).toBe(0);
|
||||
const rows = await engine.executeRaw<{ channel: string }>(
|
||||
'SELECT channel FROM context_volunteer_events', [],
|
||||
);
|
||||
expect(rows.length).toBe(1);
|
||||
expect(rows[0].channel).toBe('reflex');
|
||||
} finally {
|
||||
server!.close();
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('windowTurnCount — knob edge semantics', () => {
|
||||
test('0, negative, NaN, and absent all fall back to the default of 4 (1 = legacy off)', async () => {
|
||||
const { windowTurnCount, DEFAULT_WINDOW_TURNS } = await import('../src/core/context/reflex.ts');
|
||||
expect(DEFAULT_WINDOW_TURNS).toBe(4);
|
||||
expect(windowTurnCount(null)).toBe(4);
|
||||
expect(windowTurnCount({ retrieval_reflex_window_turns: 0 } as never)).toBe(4);
|
||||
expect(windowTurnCount({ retrieval_reflex_window_turns: -3 } as never)).toBe(4);
|
||||
expect(windowTurnCount({ retrieval_reflex_window_turns: Number.NaN } as never)).toBe(4);
|
||||
// The documented "off" switch is 1 (legacy single-turn), not 0.
|
||||
expect(windowTurnCount({ retrieval_reflex_window_turns: 1 } as never)).toBe(1);
|
||||
expect(windowTurnCount({ retrieval_reflex_window_turns: 6.9 } as never)).toBe(6);
|
||||
});
|
||||
|
||||
test('the env escape hatch is honored even when config is null (no config file / DB)', async () => {
|
||||
const { windowTurnCount } = await import('../src/core/context/reflex.ts');
|
||||
// loadConfig() returns null in a config-less environment (clean CI shard,
|
||||
// no brain) and drops its env→config mapping — windowTurnCount must still
|
||||
// read the env var directly, or the documented escape hatch is dead and
|
||||
// the window silently defaults to 4. withEnv() (not raw process.env
|
||||
// mutation) keeps the linter + isolation guard happy.
|
||||
await withEnv({ GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS: '1' }, async () => {
|
||||
expect(windowTurnCount(null)).toBe(1);
|
||||
});
|
||||
await withEnv({ GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS: '7' }, async () => {
|
||||
expect(windowTurnCount(null)).toBe(7);
|
||||
// Env wins over a config value too (env is the higher-precedence plane).
|
||||
expect(windowTurnCount({ retrieval_reflex_window_turns: 3 } as never)).toBe(7);
|
||||
});
|
||||
await withEnv({ GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS: 'not-a-number' }, async () => {
|
||||
// Garbage env falls through to config / default, not a crash.
|
||||
expect(windowTurnCount(null)).toBe(4);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
* envelope paths don't depend on DB state.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { describe, test, expect, beforeEach } from 'bun:test';
|
||||
import {
|
||||
EMBEDDING_COST_PER_1K_TOKENS,
|
||||
estimateEmbeddingCostUsd,
|
||||
@@ -21,9 +21,20 @@ import {
|
||||
shouldBlockSync,
|
||||
} from '../src/core/embedding.ts';
|
||||
import { lookupEmbeddingPrice } from '../src/core/embedding-pricing.ts';
|
||||
import { resetGateway } from '../src/core/ai/gateway.ts';
|
||||
import { estimateTokens } from '../src/core/chunkers/code.ts';
|
||||
|
||||
describe('Layer 8 D1 — embedding cost model', () => {
|
||||
// The estimateEmbeddingCostUsd tests assert the gateway-UNCONFIGURED OpenAI
|
||||
// fallback ($0.13/Mtok). That fallback only fires when no gateway is
|
||||
// configured — so guarantee that precondition rather than depending on
|
||||
// ambient state. Without this, a sibling test file in the same shard that
|
||||
// configured (and didn't reset) a cheaper model leaks its rate in here and
|
||||
// these assertions flip (the legacy-embedding preload only restores when the
|
||||
// gateway slot is EMPTY, so a non-empty foreign config survives).
|
||||
beforeEach(() => resetGateway());
|
||||
|
||||
|
||||
test('EMBEDDING_COST_PER_1K_TOKENS back-compat constant is the OpenAI 3-large rate', () => {
|
||||
// Retained only for back-compat imports. Live cost math now resolves the
|
||||
// CONFIGURED model's rate via embedding-pricing.ts (see model-aware test
|
||||
|
||||
@@ -0,0 +1,483 @@
|
||||
/**
|
||||
* v0.43 (#2095) — push-based context core: window parsing, multi-turn
|
||||
* extraction, confidence-gated volunteering, slug-only suppression, privacy,
|
||||
* and the usage-stats join. Hermetic in-memory PGLite (no file lock),
|
||||
* modeled on test/retrieval-reflex.test.ts.
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { normalizeAlias } from '../src/core/search/alias-normalize.ts';
|
||||
import {
|
||||
extractCandidatesFromWindow,
|
||||
MAX_CANDIDATES,
|
||||
type WindowTurn,
|
||||
} from '../src/core/context/entity-salience.ts';
|
||||
import { resolveEntitiesToPointers } from '../src/core/context/retrieval-reflex.ts';
|
||||
import {
|
||||
parseWindow,
|
||||
volunteerContext,
|
||||
volunteerUsageStats,
|
||||
VOLUNTEER_DEFAULT_MIN_CONFIDENCE,
|
||||
} from '../src/core/context/volunteer.ts';
|
||||
import { insertVolunteerEvents } from '../src/core/context/volunteer-events.ts';
|
||||
import { TAKES_FENCE_BEGIN, TAKES_FENCE_END } from '../src/core/takes-fence.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
async function seed(slug: string, title: string, body: string, source = 'default') {
|
||||
if (source !== 'default') {
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO sources (id, name, local_path) VALUES ($1, $1, $2)
|
||||
ON CONFLICT (id) DO NOTHING`,
|
||||
[source, `/tmp/${source}`],
|
||||
);
|
||||
}
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO pages (slug, source_id, type, title, compiled_truth, timeline)
|
||||
VALUES ($1, $2, 'person', $3, $4, '')`,
|
||||
[slug, source, title, body],
|
||||
);
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
}, 60_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await engine.executeRaw('DELETE FROM page_aliases').catch(() => {});
|
||||
await engine.executeRaw('DELETE FROM context_volunteer_events').catch(() => {});
|
||||
await engine.executeRaw('DELETE FROM pages');
|
||||
});
|
||||
|
||||
describe('parseWindow', () => {
|
||||
test('role prefixes split turns oldest → newest', () => {
|
||||
const turns = parseWindow('user: ask Alice about the deal\nassistant: noted\nuser: what did she say?');
|
||||
expect(turns).toEqual([
|
||||
{ role: 'user', text: 'ask Alice about the deal' },
|
||||
{ role: 'assistant', text: 'noted' },
|
||||
{ role: 'user', text: 'what did she say?' },
|
||||
]);
|
||||
});
|
||||
|
||||
test('CRLF input parses identically', () => {
|
||||
const turns = parseWindow('user: hello\r\nassistant: hi\r\n');
|
||||
expect(turns).toEqual([
|
||||
{ role: 'user', text: 'hello' },
|
||||
{ role: 'assistant', text: 'hi' },
|
||||
]);
|
||||
});
|
||||
|
||||
test('unprefixed text is ONE user turn (echo | volunteer-context just works)', () => {
|
||||
const turns = parseWindow('met with Alice Example today\nshe asked about acme');
|
||||
expect(turns).toEqual([{ role: 'user', text: 'met with Alice Example today\nshe asked about acme' }]);
|
||||
});
|
||||
|
||||
test('continuation lines append to the open turn', () => {
|
||||
const turns = parseWindow('user: first line\nsecond line\nassistant: ok');
|
||||
expect(turns[0]).toEqual({ role: 'user', text: 'first line\nsecond line' });
|
||||
expect(turns[1]).toEqual({ role: 'assistant', text: 'ok' });
|
||||
});
|
||||
|
||||
test('empty / whitespace input → []', () => {
|
||||
expect(parseWindow('')).toEqual([]);
|
||||
expect(parseWindow(' \n \n')).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('extractCandidatesFromWindow', () => {
|
||||
test('merges across turns with occurrence + newest-turn metadata', () => {
|
||||
const turns: WindowTurn[] = [
|
||||
{ role: 'user', text: 'ask Alice Example about the deal' },
|
||||
{ role: 'assistant', text: 'Alice Example said she will follow up' },
|
||||
{ role: 'user', text: 'and ping Bob Sample too' },
|
||||
];
|
||||
const cands = extractCandidatesFromWindow(turns);
|
||||
const alice = cands.find((c) => normalizeAlias(c.query) === normalizeAlias('Alice Example'));
|
||||
const bob = cands.find((c) => normalizeAlias(c.query) === normalizeAlias('Bob Sample'));
|
||||
expect(alice).toBeDefined();
|
||||
expect(alice!.occurrences).toBe(2);
|
||||
expect(alice!.inNewestTurn).toBe(false);
|
||||
expect(alice!.userMention).toBe(true);
|
||||
expect(bob).toBeDefined();
|
||||
expect(bob!.inNewestTurn).toBe(true);
|
||||
});
|
||||
|
||||
test('assistant-only mention is flagged (userMention=false)', () => {
|
||||
const cands = extractCandidatesFromWindow([
|
||||
{ role: 'assistant', text: 'You should talk to Charlie Demo about this' },
|
||||
{ role: 'user', text: 'good idea' },
|
||||
]);
|
||||
const charlie = cands.find((c) => normalizeAlias(c.query) === normalizeAlias('Charlie Demo'));
|
||||
expect(charlie).toBeDefined();
|
||||
expect(charlie!.userMention).toBe(false);
|
||||
});
|
||||
|
||||
test('cap holds across a noisy window', () => {
|
||||
const noisy = Array.from({ length: 30 }, (_, i) => `Entity Number${i} did something.`).join(' ');
|
||||
const cands = extractCandidatesFromWindow([{ role: 'user', text: noisy }]);
|
||||
expect(cands.length).toBeLessThanOrEqual(MAX_CANDIDATES);
|
||||
});
|
||||
});
|
||||
|
||||
describe('volunteerContext', () => {
|
||||
test('assistant-introduced entity two turns back resolves on a pronoun follow-up', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Alice is an early founder.');
|
||||
const turns = parseWindow(
|
||||
'user: who should I talk to about the seed round?\n' +
|
||||
'assistant: Alice Example led a similar round last year.\n' +
|
||||
'user: what did she invest in?',
|
||||
);
|
||||
const pages = await volunteerContext(engine, turns, { sourceIds: ['default'] });
|
||||
expect(pages.length).toBe(1);
|
||||
expect(pages[0].slug).toBe('people/alice-example');
|
||||
expect(pages[0].arm).toBe('title');
|
||||
expect(pages[0].rationale).toContain('assistant-introduced');
|
||||
});
|
||||
|
||||
test('excludeSlugs skips BEFORE the cap: an excluded entity never starves a fresh page', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Founder.');
|
||||
await seed('people/bob-sample', 'Bob Sample', 'Engineer.');
|
||||
const turns = parseWindow('user: intro Alice Example to Bob Sample');
|
||||
const pages = await volunteerContext(engine, turns, {
|
||||
sourceIds: ['default'],
|
||||
maxPages: 1,
|
||||
excludeSlugs: new Set(['people/alice-example']),
|
||||
});
|
||||
// A post-call filter would return [] here (Alice burns the single cap
|
||||
// slot, then gets filtered). The pre-cap exclusion hands Bob the slot.
|
||||
expect(pages.length).toBe(1);
|
||||
expect(pages[0].slug).toBe('people/bob-sample');
|
||||
});
|
||||
|
||||
test('confidence gate drops slug-suffix matches at the default threshold', async () => {
|
||||
// Page resolvable ONLY via slug-suffix: title differs from the mention.
|
||||
await seed('projects/widget-co', 'The Widget Company Project', 'A project page.');
|
||||
const turns = parseWindow('user: any updates on Widget-Co this week?');
|
||||
const gated = await volunteerContext(engine, turns, { sourceIds: ['default'] });
|
||||
expect(gated).toEqual([]);
|
||||
// Lowering min_confidence lets it through with honest provenance.
|
||||
const loose = await volunteerContext(engine, turns, { sourceIds: ['default'], minConfidence: 0.5 });
|
||||
expect(loose.length).toBe(1);
|
||||
expect(loose[0].arm).toBe('slug-suffix');
|
||||
expect(loose[0].confidence).toBeLessThan(VOLUNTEER_DEFAULT_MIN_CONFIDENCE);
|
||||
});
|
||||
|
||||
test('alias arm volunteers with boost when mentioned in the newest turn', async () => {
|
||||
await seed('people/swami-x', 'Swami X', 'A close friend.');
|
||||
await engine.setPageAliases('people/swami-x', 'default', [normalizeAlias('Swami')]);
|
||||
const pages = await volunteerContext(
|
||||
engine,
|
||||
parseWindow('user: Spoke with Swami today'),
|
||||
{ sourceIds: ['default'] },
|
||||
);
|
||||
expect(pages.length).toBe(1);
|
||||
expect(pages[0].arm).toBe('alias');
|
||||
expect(pages[0].confidence).toBeCloseTo(0.95, 5); // 0.9 + newest-turn boost
|
||||
expect(pages[0].rationale).toContain('alias match');
|
||||
});
|
||||
|
||||
test('suppression is slug-only under windowing: a prior-turn MENTION does not suppress', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'A founder.');
|
||||
// Prior context contains the TITLE (a bare mention from an earlier turn)
|
||||
// but NOT the slug — the page was never actually surfaced.
|
||||
const pages = await volunteerContext(
|
||||
engine,
|
||||
parseWindow('user: ping Alice Example again'),
|
||||
{ sourceIds: ['default'], priorContext: 'earlier the user said: tell Alice Example the news' },
|
||||
);
|
||||
expect(pages.length).toBe(1);
|
||||
// A slug in prior context (the page WAS surfaced) does suppress.
|
||||
const suppressed = await volunteerContext(
|
||||
engine,
|
||||
parseWindow('user: ping Alice Example again'),
|
||||
{ sourceIds: ['default'], priorContext: 'pointer: people/alice-example was injected last turn' },
|
||||
);
|
||||
expect(suppressed).toEqual([]);
|
||||
});
|
||||
|
||||
test('privacy: takes-fence content never leaks into the synopsis', async () => {
|
||||
const body = `${TAKES_FENCE_BEGIN}\nSECRET_HUNCH_DO_NOT_LEAK\n${TAKES_FENCE_END}\nAlice is a founder.`;
|
||||
await seed('people/alice-example', 'Alice Example', body);
|
||||
const pages = await volunteerContext(engine, parseWindow('user: about Alice Example'), { sourceIds: ['default'] });
|
||||
expect(pages.length).toBe(1);
|
||||
expect(JSON.stringify(pages)).not.toContain('SECRET_HUNCH_DO_NOT_LEAK');
|
||||
});
|
||||
|
||||
test('multi-source scope: resolves from both sources, never outside the scope', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Founder.', 'default');
|
||||
await seed('people/bob-sample', 'Bob Sample', 'Engineer.', 'team-brain');
|
||||
await seed('people/eve-other', 'Eve Other', 'Out of scope.', 'private-brain');
|
||||
const turns = parseWindow('user: intro Alice Example to Bob Sample and Eve Other');
|
||||
const pages = await volunteerContext(engine, turns, { sourceIds: ['default', 'team-brain'] });
|
||||
const keys = pages.map((p) => `${p.source_id}:${p.slug}`).sort();
|
||||
expect(keys).toEqual(['default:people/alice-example', 'team-brain:people/bob-sample']);
|
||||
});
|
||||
|
||||
test('zero-candidate fast path: no entities → [] without touching resolution', async () => {
|
||||
const pages = await volunteerContext(engine, parseWindow('user: ok thanks, sounds good'), {
|
||||
sourceIds: ['default'],
|
||||
});
|
||||
expect(pages).toEqual([]);
|
||||
});
|
||||
|
||||
test('max_pages caps at 5 even when asked for more', async () => {
|
||||
for (let i = 0; i < 8; i++) {
|
||||
await seed(`people/person-${i}`, `Person Alpha${i}`, 'A person.');
|
||||
}
|
||||
const text = Array.from({ length: 8 }, (_, i) => `Person Alpha${i}`).join(' and ');
|
||||
const pages = await volunteerContext(engine, parseWindow(`user: intro ${text}`), {
|
||||
sourceIds: ['default'],
|
||||
maxPages: 50,
|
||||
});
|
||||
expect(pages.length).toBeLessThanOrEqual(5);
|
||||
});
|
||||
});
|
||||
|
||||
describe('volunteerUsageStats', () => {
|
||||
test('join math: used = last_retrieved_at > volunteered_at, labeled approximate', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Founder.');
|
||||
await seed('people/bob-sample', 'Bob Sample', 'Engineer.');
|
||||
await insertVolunteerEvents(engine, [
|
||||
{ source_id: 'default', slug: 'people/alice-example', confidence: 0.9, match_arm: 'alias', rationale: 'r', channel: 'op' },
|
||||
{ source_id: 'default', slug: 'people/bob-sample', confidence: 0.8, match_arm: 'title', rationale: 'r', channel: 'op' },
|
||||
]);
|
||||
// Alice was opened AFTER being volunteered; Bob never was.
|
||||
await engine.executeRaw(
|
||||
`UPDATE pages SET last_retrieved_at = now() + interval '1 minute' WHERE slug = 'people/alice-example'`, [],
|
||||
);
|
||||
const stats = await volunteerUsageStats(engine, ['default'], 30);
|
||||
expect(stats.approximate).toBe(true);
|
||||
expect(stats.note).toContain('approximate');
|
||||
expect(stats.total_volunteered).toBe(2);
|
||||
expect(stats.total_used).toBe(1);
|
||||
const alias = stats.by_arm.find((a) => a.match_arm === 'alias')!;
|
||||
expect(alias.used).toBe(1);
|
||||
expect(alias.precision).toBe(1);
|
||||
const title = stats.by_arm.find((a) => a.match_arm === 'title')!;
|
||||
expect(title.used).toBe(0);
|
||||
});
|
||||
|
||||
test('zero events → zeroed stats, not an error', async () => {
|
||||
const stats = await volunteerUsageStats(engine, ['default'], 7);
|
||||
expect(stats.total_volunteered).toBe(0);
|
||||
expect(stats.by_arm).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveEntitiesToPointers — new provenance surface (back-compat)', () => {
|
||||
test('pointers carry arm + confidence + source_id', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Founder.');
|
||||
const block = await resolveEntitiesToPointers(
|
||||
engine,
|
||||
'default',
|
||||
[{ display: 'Alice Example', query: 'Alice Example' }],
|
||||
{},
|
||||
);
|
||||
expect(block).not.toBeNull();
|
||||
expect(block!.pointers[0].arm).toBe('title');
|
||||
expect(block!.pointers[0].confidence).toBe(0.8);
|
||||
expect(block!.pointers[0].source_id).toBe('default');
|
||||
});
|
||||
|
||||
test('legacy suppression (slug-and-title) still drops a title mention in prior context', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Founder.');
|
||||
const block = await resolveEntitiesToPointers(
|
||||
engine,
|
||||
'default',
|
||||
[{ display: 'Alice Example', query: 'Alice Example' }],
|
||||
{ priorContextText: 'we discussed Alice Example earlier' },
|
||||
);
|
||||
expect(block).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('volunteer_context op (contract surface)', () => {
|
||||
const { operationsByName } = require('../src/core/operations.ts') as typeof import('../src/core/operations.ts');
|
||||
const {
|
||||
awaitPendingVolunteerEventWrites,
|
||||
_resetPendingVolunteerEventWritesForTests,
|
||||
} = require('../src/core/context/volunteer-events.ts') as typeof import('../src/core/context/volunteer-events.ts');
|
||||
|
||||
function mkCtx(overrides: Record<string, unknown> = {}) {
|
||||
return {
|
||||
engine,
|
||||
config: {} as never,
|
||||
logger: { info: () => {}, warn: () => {}, error: () => {} } as never,
|
||||
dryRun: false,
|
||||
remote: false,
|
||||
sourceId: 'default',
|
||||
...overrides,
|
||||
} as never;
|
||||
}
|
||||
|
||||
test('registered, read-scope, not localOnly, stdin cliHint on window', () => {
|
||||
const op = operationsByName.volunteer_context;
|
||||
expect(op).toBeDefined();
|
||||
expect(op.scope).toBe('read');
|
||||
expect(op.localOnly).toBeFalsy();
|
||||
expect(op.cliHints?.name).toBe('volunteer-context');
|
||||
expect(op.cliHints?.stdin).toBe('window');
|
||||
});
|
||||
|
||||
test('window required unless stats: true', async () => {
|
||||
const op = operationsByName.volunteer_context;
|
||||
await expect(op.handler(mkCtx(), {})).rejects.toThrow(/window is required/);
|
||||
const stats = (await op.handler(mkCtx(), { stats: true })) as any;
|
||||
expect(stats.approximate).toBe(true);
|
||||
});
|
||||
|
||||
test('volunteers pages and logs events through the drained sink', async () => {
|
||||
_resetPendingVolunteerEventWritesForTests();
|
||||
await seed('people/alice-example', 'Alice Example', 'Founder.');
|
||||
const op = operationsByName.volunteer_context;
|
||||
const result = (await op.handler(mkCtx(), {
|
||||
window: 'user: ping Alice Example about the deal',
|
||||
session_id: 's-42',
|
||||
turn: 7,
|
||||
})) as any;
|
||||
expect(result.count).toBe(1);
|
||||
expect(result.pages[0].slug).toBe('people/alice-example');
|
||||
expect(result.window_turns).toBe(1);
|
||||
// Event row lands via the fire-and-forget sink once drained.
|
||||
const { unfinished } = await awaitPendingVolunteerEventWrites(5_000);
|
||||
expect(unfinished).toBe(0);
|
||||
const rows = await engine.executeRaw<{ slug: string; channel: string; session_id: string; turn: number }>(
|
||||
`SELECT slug, channel, session_id, turn FROM context_volunteer_events`, [],
|
||||
);
|
||||
expect(rows.length).toBe(1);
|
||||
expect(rows[0].slug).toBe('people/alice-example');
|
||||
expect(rows[0].channel).toBe('op');
|
||||
expect(rows[0].session_id).toBe('s-42');
|
||||
expect(Number(rows[0].turn)).toBe(7);
|
||||
});
|
||||
|
||||
test('event-log failure never fails the op (failing engine injected for the INSERT)', async () => {
|
||||
_resetPendingVolunteerEventWritesForTests();
|
||||
await seed('people/alice-example', 'Alice Example', 'Founder.');
|
||||
const op = operationsByName.volunteer_context;
|
||||
// Wrap the engine: reads pass through, INSERTs into the events table throw.
|
||||
const failingEngine = new Proxy(engine, {
|
||||
get(target, prop, receiver) {
|
||||
if (prop === 'executeRaw') {
|
||||
return (sql: string, params: unknown[]) => {
|
||||
if (/INSERT INTO context_volunteer_events/.test(sql)) {
|
||||
return Promise.reject(new Error('telemetry db down'));
|
||||
}
|
||||
return target.executeRaw(sql, params);
|
||||
};
|
||||
}
|
||||
return Reflect.get(target, prop, receiver);
|
||||
},
|
||||
});
|
||||
const result = (await op.handler(mkCtx({ engine: failingEngine }), {
|
||||
window: 'user: ping Alice Example',
|
||||
})) as any;
|
||||
expect(result.count).toBe(1); // the volunteer result is unaffected
|
||||
const { unfinished } = await awaitPendingVolunteerEventWrites(5_000);
|
||||
expect(unfinished).toBe(0); // failed write settled (swallowed), not stuck
|
||||
});
|
||||
|
||||
test('federated grant scopes the volunteer (allowedSources)', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Founder.', 'default');
|
||||
await seed('people/bob-sample', 'Bob Sample', 'Engineer.', 'grant-brain');
|
||||
await seed('people/eve-other', 'Eve Other', 'Out of grant.', 'secret-brain');
|
||||
const op = operationsByName.volunteer_context;
|
||||
const result = (await op.handler(
|
||||
mkCtx({ remote: true, auth: { allowedSources: ['grant-brain'] } }),
|
||||
{ window: 'user: intro Alice Example to Bob Sample and Eve Other' },
|
||||
)) as any;
|
||||
expect(result.pages.map((p: any) => p.slug)).toEqual(['people/bob-sample']);
|
||||
});
|
||||
|
||||
test('stats mode is source-scoped and returns the approximate note', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Founder.');
|
||||
await insertVolunteerEvents(engine, [
|
||||
{ source_id: 'default', slug: 'people/alice-example', confidence: 0.9, match_arm: 'alias', rationale: 'r', channel: 'watch' },
|
||||
]);
|
||||
const op = operationsByName.volunteer_context;
|
||||
const stats = (await op.handler(mkCtx(), { stats: true, days: 7 })) as any;
|
||||
expect(stats.days).toBe(7);
|
||||
expect(stats.total_volunteered).toBe(1);
|
||||
expect(stats.by_arm[0].channel).toBe('watch');
|
||||
expect(stats.note).toContain('approximate');
|
||||
});
|
||||
});
|
||||
|
||||
describe('knob clamps — untrusted MCP caller inputs (review hardening)', () => {
|
||||
test('minConfidence outside [0,1] (or NaN) falls back to the 0.7 default gate', async () => {
|
||||
await seed('projects/widget-co', 'The Widget Company Project', 'A project.');
|
||||
const turns = parseWindow('user: updates on Widget-Co?');
|
||||
for (const bad of [5, -1, Number.NaN]) {
|
||||
const pages = await volunteerContext(engine, turns, { sourceIds: ['default'], minConfidence: bad });
|
||||
expect(pages).toEqual([]); // slug-suffix (0.6+boost) stays gated at the default 0.7
|
||||
}
|
||||
});
|
||||
|
||||
test('maxPages 0 / negative / NaN fall back to the 3-page default', async () => {
|
||||
for (let i = 0; i < 5; i++) await seed(`people/person-${i}`, `Person Alpha${i}`, 'A person.');
|
||||
const text = Array.from({ length: 5 }, (_, i) => `Person Alpha${i}`).join(' and ');
|
||||
for (const bad of [0, -3, Number.NaN]) {
|
||||
const pages = await volunteerContext(engine, parseWindow(`user: intro ${text}`), {
|
||||
sourceIds: ['default'],
|
||||
maxPages: bad,
|
||||
});
|
||||
expect(pages.length).toBeLessThanOrEqual(3);
|
||||
expect(pages.length).toBeGreaterThan(0);
|
||||
}
|
||||
});
|
||||
|
||||
test('stats days <= 0 / NaN falls back to 30', async () => {
|
||||
const stats = await volunteerUsageStats(engine, ['default'], -5);
|
||||
expect(stats.days).toBe(30);
|
||||
const stats2 = await volunteerUsageStats(engine, ['default'], Number.NaN);
|
||||
expect(stats2.days).toBe(30);
|
||||
});
|
||||
});
|
||||
|
||||
describe('window-cap ordering — the newest user mention survives the cap', () => {
|
||||
test('stale assistant-only chatter is dropped before a newest-turn user entity', async () => {
|
||||
const { extractCandidatesFromWindow: extract, MAX_CANDIDATES: CAP } = await import('../src/core/context/entity-salience.ts');
|
||||
// 14 stale assistant-introduced entities in turn 1, then the user names
|
||||
// ONE entity in the newest turn. The cap (12) must keep the user's.
|
||||
const stale = Array.from({ length: 14 }, (_, i) => `Stale Chatter${i}`).join(', ');
|
||||
const cands = extract([
|
||||
{ role: 'assistant', text: `consider ${stale}.` },
|
||||
{ role: 'user', text: 'actually ask Alice Example first' },
|
||||
]);
|
||||
expect(cands.length).toBeLessThanOrEqual(CAP);
|
||||
const alice = cands.find((c) => normalizeAlias(c.query) === normalizeAlias('Alice Example'));
|
||||
expect(alice).toBeDefined();
|
||||
// Recency + user-role weighting puts the newest user mention FIRST.
|
||||
expect(normalizeAlias(cands[0].query)).toBe(normalizeAlias('Alice Example'));
|
||||
});
|
||||
});
|
||||
|
||||
describe('volunteer-events sink — timeout branch (long-lived process safety)', () => {
|
||||
test('a hung write reports unfinished and drops the snapshot (no ghost references)', async () => {
|
||||
const {
|
||||
logVolunteerEventsFireAndForget,
|
||||
awaitPendingVolunteerEventWrites,
|
||||
_resetPendingVolunteerEventWritesForTests,
|
||||
_peekPendingVolunteerEventWritesForTests,
|
||||
} = await import('../src/core/context/volunteer-events.ts');
|
||||
_resetPendingVolunteerEventWritesForTests();
|
||||
const hangingEngine = {
|
||||
executeRaw: () => new Promise(() => { /* never settles */ }),
|
||||
} as never;
|
||||
logVolunteerEventsFireAndForget(hangingEngine, [
|
||||
{ source_id: 'default', slug: 'people/x', confidence: 0.9, match_arm: 'alias', rationale: 'r', channel: 'watch' },
|
||||
]);
|
||||
const { unfinished } = await awaitPendingVolunteerEventWrites(20);
|
||||
expect(unfinished).toBe(1);
|
||||
// Snapshot dropped so a long-lived `gbrain watch` never accumulates
|
||||
// references to forever-pending work (the last-retrieved C1 class).
|
||||
expect(_peekPendingVolunteerEventWritesForTests()).toBe(0);
|
||||
_resetPendingVolunteerEventWritesForTests();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,231 @@
|
||||
/**
|
||||
* v0.43 (#2095) — `gbrain watch` push transport: streaming loop, rolling
|
||||
* window, session dedupe, --json shape, event logging on channel 'watch',
|
||||
* and clean EOF return. Hermetic PGLite + injected line/write deps (no
|
||||
* subprocess, no real stdin).
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { runWatch, WATCH_HELP } from '../src/commands/watch.ts';
|
||||
import { awaitPendingVolunteerEventWrites, _resetPendingVolunteerEventWritesForTests } from '../src/core/context/volunteer-events.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
async function seed(slug: string, title: string, body: string) {
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO pages (slug, source_id, type, title, compiled_truth, timeline)
|
||||
VALUES ($1, 'default', 'person', $2, $3, '')`,
|
||||
[slug, title, body],
|
||||
);
|
||||
}
|
||||
|
||||
async function* feed(lines: string[]): AsyncGenerator<string> {
|
||||
for (const l of lines) yield l;
|
||||
}
|
||||
|
||||
async function watchRun(lines: string[], extraArgs: string[] = []): Promise<string[]> {
|
||||
const out: string[] = [];
|
||||
await runWatch(engine, ['--source', 'default', ...extraArgs], {
|
||||
lines: feed(lines),
|
||||
write: (s) => out.push(s),
|
||||
isTTY: false,
|
||||
});
|
||||
return out;
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
}, 60_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
_resetPendingVolunteerEventWritesForTests();
|
||||
await engine.executeRaw('DELETE FROM context_volunteer_events').catch(() => {});
|
||||
await engine.executeRaw('DELETE FROM pages');
|
||||
});
|
||||
|
||||
describe('gbrain watch (#2095)', () => {
|
||||
test('--help prints WATCH_HELP and touches nothing', async () => {
|
||||
const out = await watchRun([], ['--help']);
|
||||
expect(out.join('')).toBe(WATCH_HELP);
|
||||
});
|
||||
|
||||
test('volunteers per turn and returns cleanly at EOF', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Alice is a founder.');
|
||||
const out = await watchRun(['ping Alice Example about the deal']);
|
||||
const text = out.join('');
|
||||
expect(text).toContain('people/alice-example');
|
||||
expect(text).toContain('exact title match');
|
||||
// runWatch RESOLVED — the EOF → clean-return contract (the entrypoint
|
||||
// flush-exit + finally drain handle the rest in the real CLI).
|
||||
});
|
||||
|
||||
test('rolling window: assistant-introduced entity fires on the pronoun follow-up turn', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Alice is a founder.');
|
||||
const out = await watchRun([
|
||||
'user: who should I ask about the round?',
|
||||
'assistant: Alice Example led one last year.',
|
||||
'user: what did she invest in?',
|
||||
]);
|
||||
expect(out.join('')).toContain('people/alice-example');
|
||||
});
|
||||
|
||||
test('session dedupe: a slug is volunteered at most once per session', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Alice is a founder.');
|
||||
const out = await watchRun([
|
||||
'user: ping Alice Example',
|
||||
'user: ok',
|
||||
'user: Alice Example again please',
|
||||
]);
|
||||
const hits = out.join('').split('people/alice-example').length - 1;
|
||||
expect(hits).toBe(1);
|
||||
});
|
||||
|
||||
test('--json emits one JSONL row per volunteered page with turn attribution', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Alice is a founder.');
|
||||
const out = await watchRun(['user: hello there', 'user: ping Alice Example'], ['--json']);
|
||||
expect(out.length).toBe(1);
|
||||
const row = JSON.parse(out[0]);
|
||||
expect(row.slug).toBe('people/alice-example');
|
||||
expect(row.turn).toBe(2);
|
||||
expect(row.arm).toBe('title');
|
||||
expect(typeof row.confidence).toBe('number');
|
||||
});
|
||||
|
||||
test('events land on channel watch with session_id + turn (drained sink)', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Alice is a founder.');
|
||||
await watchRun(['user: ping Alice Example']);
|
||||
const { unfinished } = await awaitPendingVolunteerEventWrites(5_000);
|
||||
expect(unfinished).toBe(0);
|
||||
const rows = await engine.executeRaw<{ channel: string; session_id: string; turn: number }>(
|
||||
`SELECT channel, session_id, turn FROM context_volunteer_events`, [],
|
||||
);
|
||||
expect(rows.length).toBe(1);
|
||||
expect(rows[0].channel).toBe('watch');
|
||||
expect(rows[0].session_id).toMatch(/^watch-/);
|
||||
expect(Number(rows[0].turn)).toBe(1);
|
||||
});
|
||||
|
||||
test('min-confidence flag gates exactly like the op', async () => {
|
||||
await seed('projects/widget-co', 'The Widget Company Project', 'A project.');
|
||||
const gated = await watchRun(['user: updates on Widget-Co?']);
|
||||
expect(gated.join('')).toBe('');
|
||||
const loose = await watchRun(['user: updates on Widget-Co?'], ['--min-confidence', '0.5']);
|
||||
expect(loose.join('')).toContain('projects/widget-co');
|
||||
});
|
||||
|
||||
test('blank lines and CRLF are tolerated; no turn, no volunteer', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Alice is a founder.');
|
||||
const out = await watchRun(['', ' ', 'user: ping Alice Example\r']);
|
||||
expect(out.join('')).toContain('people/alice-example');
|
||||
});
|
||||
});
|
||||
|
||||
describe('gbrain watch — window + cap flags (ship coverage G4)', () => {
|
||||
test('--window-turns 1: fires on the mention turn itself; the pronoun follow-up adds nothing', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Alice is a founder.');
|
||||
const out = await watchRun(
|
||||
[
|
||||
'assistant: Alice Example led one last year.',
|
||||
'user: what did she invest in?',
|
||||
],
|
||||
['--window-turns', '1', '--json'],
|
||||
);
|
||||
// Watch volunteers per turn: the entity fires at turn 1 (its mention
|
||||
// turn). With window=1 the follow-up turn extracts nothing, so exactly
|
||||
// one row exists and it carries turn 1 attribution.
|
||||
expect(out.length).toBe(1);
|
||||
expect(JSON.parse(out[0]).turn).toBe(1);
|
||||
});
|
||||
|
||||
test('--max-pages 1 caps a multi-entity turn to one volunteered page', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Founder.');
|
||||
await seed('people/bob-sample', 'Bob Sample', 'Engineer.');
|
||||
const out = await watchRun(
|
||||
['user: intro Alice Example to Bob Sample'],
|
||||
['--max-pages', '1', '--json'],
|
||||
);
|
||||
expect(out.length).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('gbrain watch — red-team hardening', () => {
|
||||
test('starvation guard: an already-pushed slug never burns the --max-pages cap slot', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Founder.');
|
||||
await seed('people/bob-sample', 'Bob Sample', 'Engineer.');
|
||||
const out = await watchRun(
|
||||
[
|
||||
'user: ping Alice Example about the round',
|
||||
'user: also intro Alice Example to Bob Sample',
|
||||
],
|
||||
['--max-pages', '1', '--json'],
|
||||
);
|
||||
// Turn 1: Alice takes the single cap slot. Turn 2: Alice is excluded
|
||||
// BEFORE the cap (not filtered after), so Bob gets the slot instead of
|
||||
// the turn volunteering nothing forever.
|
||||
expect(out.length).toBe(2);
|
||||
expect(JSON.parse(out[0]).slug).toBe('people/alice-example');
|
||||
expect(JSON.parse(out[1]).slug).toBe('people/bob-sample');
|
||||
});
|
||||
|
||||
test('--window-turns clamps: 0 and negatives behave as window 1', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Founder.');
|
||||
const out = await watchRun(
|
||||
[
|
||||
'assistant: Alice Example led one last year.',
|
||||
'user: what did she invest in?',
|
||||
],
|
||||
['--window-turns', '0', '--json'],
|
||||
);
|
||||
// Clamped to 1: fires on the mention turn only — identical to the
|
||||
// --window-turns 1 contract above, not a crash or an unbounded window.
|
||||
expect(out.length).toBe(1);
|
||||
expect(JSON.parse(out[0]).turn).toBe(1);
|
||||
});
|
||||
|
||||
test('--window-turns absurdly large still runs (hard cap, no unbounded re-scan)', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Founder.');
|
||||
const filler = Array.from({ length: 70 }, (_, i) => `user: filler line ${i}`);
|
||||
const out = await watchRun(
|
||||
[...filler, 'user: ping Alice Example'],
|
||||
['--window-turns', '999999', '--json'],
|
||||
);
|
||||
expect(out.length).toBe(1);
|
||||
expect(JSON.parse(out[0]).slug).toBe('people/alice-example');
|
||||
});
|
||||
});
|
||||
|
||||
describe('gbrain watch — per-turn fail-open (review hardening)', () => {
|
||||
test('a transient DB error on one turn never kills the stream', async () => {
|
||||
await seed('people/alice-example', 'Alice Example', 'Alice is a founder.');
|
||||
// Fail the FIRST resolver query against pages (turn 1's resolution);
|
||||
// everything else — incl. resolveSourceId's pre-loop check — passes.
|
||||
let pagesQueries = 0;
|
||||
const flaky = new Proxy(engine, {
|
||||
get(target, prop, receiver) {
|
||||
if (prop === 'executeRaw') {
|
||||
return (sql: string, params: unknown[]) => {
|
||||
if (/FROM pages/.test(sql) && ++pagesQueries === 1) {
|
||||
return Promise.reject(new Error('transient db hiccup'));
|
||||
}
|
||||
return target.executeRaw(sql, params);
|
||||
};
|
||||
}
|
||||
return Reflect.get(target, prop, receiver);
|
||||
},
|
||||
});
|
||||
const out: string[] = [];
|
||||
await runWatch(flaky as never, ['--source', 'default'], {
|
||||
lines: feed(['user: ping Alice Example', 'user: ping Alice Example please']),
|
||||
write: (s) => out.push(s),
|
||||
isTTY: false,
|
||||
});
|
||||
// Turn 1 failed open; turn 2 volunteered. The stream survived.
|
||||
expect(out.join('')).toContain('people/alice-example');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,75 @@
|
||||
/**
|
||||
* v0.43 (#2095) — `gbrain watch` SIGINT lifecycle. SERIAL: spawns a real CLI
|
||||
* subprocess with a tmpdir brain (the parallel unit shards flake on
|
||||
* concurrent subprocess spawns — same isolation rationale as
|
||||
* apply-migrations-pglite-spawn.serial.test.ts).
|
||||
*/
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
|
||||
describe('gbrain watch — SIGINT lifecycle (real subprocess)', () => {
|
||||
test('SIGINT mid-stream closes the stream and exits cleanly (drain path, exit 0)', async () => {
|
||||
const { mkdtempSync, mkdirSync, writeFileSync, rmSync } = await import('fs');
|
||||
const { join, resolve } = await import('path');
|
||||
const { tmpdir } = await import('os');
|
||||
const REPO = resolve(import.meta.dir, '..');
|
||||
const home = mkdtempSync(join(tmpdir(), 'gbrain-watch-sigint-'));
|
||||
try {
|
||||
mkdirSync(join(home, '.gbrain'), { recursive: true });
|
||||
writeFileSync(
|
||||
join(home, '.gbrain', 'config.json'),
|
||||
JSON.stringify({
|
||||
engine: 'pglite',
|
||||
database_path: join(home, '.gbrain', 'brain.pglite'),
|
||||
embedding_dimensions: 1536,
|
||||
}) + '\n',
|
||||
);
|
||||
// Piped stdin that NEVER reaches EOF — only SIGINT can end the stream.
|
||||
const proc = Bun.spawn(['bun', 'run', join(REPO, 'src', 'cli.ts'), 'watch'], {
|
||||
cwd: REPO,
|
||||
env: { ...process.env, HOME: home, GBRAIN_HOME: home, GBRAIN_SKIP_STARTUP_HOOKS: '1' },
|
||||
stdin: 'pipe',
|
||||
stdout: 'pipe',
|
||||
stderr: 'pipe',
|
||||
});
|
||||
proc.stdin.write('user: nothing relevant here\n');
|
||||
await proc.stdin.flush();
|
||||
// Readiness probe: watch prints "[watch] session <id> ready" on stderr
|
||||
// once engine + source resolution are done and the stdin loop is live.
|
||||
// A fixed sleep raced cold PGLite init (other tests budget 60s for it)
|
||||
// — SIGINT before the handler registers means default-disposition kill.
|
||||
const stderrChunks: string[] = [];
|
||||
const reader = (proc.stderr as ReadableStream<Uint8Array>).getReader();
|
||||
const decoder = new TextDecoder();
|
||||
const deadline = Date.now() + 90_000;
|
||||
let ready = false;
|
||||
while (Date.now() < deadline) {
|
||||
const { value, done } = await reader.read();
|
||||
if (done) break;
|
||||
stderrChunks.push(decoder.decode(value, { stream: true }));
|
||||
if (stderrChunks.join('').includes('ready')) { ready = true; break; }
|
||||
}
|
||||
expect(ready).toBe(true);
|
||||
// Brief settle so the first turn's volunteerContext round-trip is in
|
||||
// flight or done, then interrupt mid-stream.
|
||||
await new Promise((r) => setTimeout(r, 500));
|
||||
proc.kill('SIGINT');
|
||||
const killer = setTimeout(() => {
|
||||
try { proc.kill('SIGKILL'); } catch { /* already dead */ }
|
||||
}, 30_000);
|
||||
const exitCode = await proc.exited;
|
||||
clearTimeout(killer);
|
||||
// Drain the rest of stderr for the banner assertion.
|
||||
while (true) {
|
||||
const { value, done } = await reader.read();
|
||||
if (done) break;
|
||||
stderrChunks.push(decoder.decode(value, { stream: true }));
|
||||
}
|
||||
const stderr = stderrChunks.join('');
|
||||
// Clean drain-then-exit: no force-exit banner, no SIGKILL (137), exit 0.
|
||||
expect(stderr).not.toContain('force-exiting');
|
||||
expect(exitCode).toBe(0);
|
||||
} finally {
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
}
|
||||
}, 120_000);
|
||||
});
|
||||
Reference in New Issue
Block a user