mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-30 19:49:14 +00:00
* fix(cli): exit deliberately after bounded teardown instead of riding the 10s backstop (#2084)
Root cause: bounded teardown (endPoolBounded, #2015) RESOLVES, but lingering
sockets — embedding-provider fetch keep-alive, PgBouncer txn-mode sockets the
bound raced past — keep Bun's event loop alive, so every `gbrain query` paid
a flat 10s tax exiting via the hard-deadline force-exit banner.
Three changes, one contract:
- flushStdoutThenExit (cli-force-exit.ts): when main() resolves and the
command is not a daemon, exit deliberately — after stdout AND stderr drain
(writableLength===0, 'drain'-event + poll loop, 2s unref'd guard for a
blocked pipe). Incident #1959 (force-exit truncating piped stdout) is the
regression class; pinned by a 256KB real-pipe subprocess test.
- drainThenDisconnect (cli.ts): ONE owner-disconnect helper at all 8 sites
(op-dispatch, CLI_ONLY fall-through, search dashboard, doctor remediation
x3, ze-switch, dream, read-only timeout path). Drains the background-work
registry, then disconnect (best-effort), bounded by the 10s unref'd
hard-deadline — which is now armed around the TEARDOWN window only, not
before the op handler (the old placement would have force-killed any op
slower than 10s). Closes the filed TODOS P3 drain-hoist: six sites
previously skipped the drain entirely and had no hang timer at all.
- Inner process.exit sweep: mid-handler exits in engine-owning/output-bearing
paths (status, friction, claw-test, smoke-test, eval cross-modal /
takes-quality replay / conversation-parser / whoknows-thin, status-thin)
become process.exitCode + return so they flow through the drains and the
flush-exit. Pre-engine usage/parse/refusal exits stay as-is.
BrainRegistry.disconnectAll deliberately unchanged: zero production callers
in src/, per-engine disconnects already bounded, and the kernel reclaims
sockets on exit (src/core/timeout.ts doctrine).
DAEMON_COMMANDS gains 'watch' ahead of the #2095 push transport.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(e2e): PgBouncer transaction-mode pooler in CI + teardown e2e (#2084)
Three consecutive waves (#1972 → #2015 → #2084) fixed pooler-teardown bugs
verified only against one production deployment — CI had no transaction-mode
pooler and could never see the class. Now it can:
- docker-compose.ci.yml: `pgbouncer` service (transaction pooling) fronting
postgres-1, mirroring the production split-pool topology (direct :5432 +
pooled :6543). AUTH_TYPE=plain (pg16 SCRAM verifiers need the plaintext
password in the userlist) + IGNORE_STARTUP_PARAMETERS for the
statement_timeout/idle_in_transaction_session_timeout startup params
gbrain's client sets (the Supabase pooler whitelists the same).
- test/e2e/pgbouncer-teardown.test.ts: schema + fixture via the DIRECT url
into a dedicated `gbrain_pgbouncer` database (never races shard TRUNCATEs),
then spawns the real CLI against the POOLED url and asserts: exit 0,
stdout intact (the #1959 truncation class), and NO
"did not return within 10000ms — force-exiting" banner (pre-#2084 it
printed on 100% of query-shaped ops on this topology). Class bound, not
exact timing. Skips gracefully without GBRAIN_PGBOUNCER_URL.
- scripts/ci-local.sh: threads GBRAIN_PGBOUNCER_URL +
GBRAIN_PGBOUNCER_DIRECT_URL into all three e2e phases.
Verified live: both tests green against pgbouncer 1.25.2 in transaction mode.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(schema): context_volunteer_events table (v116) — push-context feedback log (#2095)
One row per page the brain volunteers (op / reflex / watch channels).
"Used" is DERIVED, never written: pages.last_retrieved_at > volunteered_at
(the existing bumpLastRetrievedAt write-back is the open/cite signal), so
there is no second tracking path. session_id/turn are nullable
caller-supplied attribution; rationale is a deterministic template string,
never raw conversation text.
- Migration v116 (idempotent) + mirrors in src/schema.sql +
src/core/pglite-schema.ts + regenerated schema-embedded.ts (regen also
folds in pre-existing comment-only drift from the v114 links edits).
- src/core/context/volunteer-events.ts: insertVolunteerEvents (ONE
multi-row parameterized INSERT — never per-row awaited round-trips) +
purgeStaleVolunteerEvents (90-day GC, returns 0 on pre-v116 brains).
- Dream cycle purge phase prunes stale events alongside op_checkpoints /
brainstorm checkpoints / batch-retry audit files.
- RLS on Postgres comes from the v35 auto_rls_on_create_table event
trigger (the same mechanism that covered v110 page_aliases and v115
op_checkpoint_paths); the volunteer Postgres e2e pins it.
- No ::jsonb anywhere; no bootstrap probe needed (nothing references the
table pre-creation; writers guard with try/catch).
Tests: v116 shape + columns + indexes + live insert/purge round-trip on
PGLite (test/migrate.test.ts, 161 pass); schema-bootstrap-coverage green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(context): multi-turn window extraction + confidence-scored volunteer core (#2095)
- entity-salience.ts: extractCandidatesFromWindow(turns) — runs the existing
per-turn extractor across the last N turns (oldest→newest), merges by the
normalizeAlias form with occurrence/newest-turn/user-mention metadata, and
orders by salience (recency > frequency > user-role) so the MAX_CANDIDATES
cap drops stale assistant chatter, not the entity the user just named.
Closes the filed assistant-introduced-entities recall TODO; true pronoun
coreference (never-named antecedents) stays out of scope.
- retrieval-reflex.ts: ReflexPointer gains source_id + arm + confidence +
matchedNorm. ARM_CONFIDENCE (alias 0.9 / title 0.8 / slug-suffix 0.6)
lives next to the arm definitions so identity and score can't drift.
Arm-2 provenance is classified in JS (codex D8 — the combined OR can't
report which predicate matched). Federated sourceIds[] scope (alias arm
loops per source; arm 2 uses source_id = ANY — no engine-interface
change). Suppression gains 'slug-only' mode (codex D7, REQUIRED for
windowing): the legacy title-whole-word rule would suppress every entity
merely MENTIONED in a prior window turn, breaking the feature by
construction — slugs only enter context when a pointer/page was actually
surfaced. Default stays 'slug-and-title' for the window=1 legacy path.
- volunteer.ts (new): parseWindow (lenient user:/assistant: prefixes, CRLF,
unprefixed → one user turn), volunteerContext (zero-LLM: extract →
resolve → +0.05 multi-turn/newest-turn boost → min_confidence 0.7 gate →
cap 3/5; deterministic rationale strings, never raw conversation text),
and volunteerUsageStats (per-arm/channel precision from the
last_retrieved_at join, labeled approximate — 5-min throttle false
negatives, unrelated-read false positives; codex D9).
Tests: 35 green across volunteer-context (window parsing, pronoun follow-up
via assistant-introduced entity, confidence gating, slug-only suppression,
takes-fence privacy, multi-source scope, caps, stats join math) +
retrieval-reflex back-compat + resolve-ipc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ops): volunteer_context op — CLI (stdin) + MCP, drained event sink (#2095)
New read-scope op on the contract surface (CLI `gbrain volunteer-context`
with stdin → window, MCP tool for free): takes a rolling conversation
window, returns confidence-gated page pointers with rationales + synopses.
`window` is optional-unless-stats (validated in the handler, codex D9);
`stats: true` returns the volunteered-vs-used precision summary, labeled
APPROXIMATE (the 5-min last-retrieved throttle and unrelated reads both
bias the join). Source scope threads through sourceScopeOpts — federated
grants narrow the volunteer to the granted sources.
Event logging is fire-and-forget through a new `volunteer-events`
background-work sink (volunteer-events.ts, mirrors last-retrieved: tracked
dangling promise set + bounded drain + snapshot-drop on timeout so a
long-lived process never accumulates ghosts). ONE batched INSERT per call,
drained on every exit path by the commit-1 drain hoist; failure never
fails the op (pinned by an injected failing-engine test).
cli formatResult renders both shapes (pointer lines with confidence/arm/
rationale; the stats summary with per-arm precision).
Tests: op contract surface, window-required validation, sink round-trip
with session_id/turn attribution, failing-engine fail-open, federated
grant scoping, stats mode (26 green on PGLite) + a real-Postgres e2e
proving the op + sink + stats join AND that context_volunteer_events has
RLS enabled (keeps the auto-RLS event-trigger mechanism honest for v116).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(context): reflex consumes the rolling window + ambient-channel logging (#2095)
The default-on retrieval reflex now extracts entities from the last N turns
(retrieval_reflex_window_turns, default 4; env
GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS; window=1 reproduces the legacy
current-turn-only behavior exactly). assemble() passes the recent
user/assistant turns (hard cap 12); the reflex slices to the configured
window. Assistant-introduced entities and "what did she invest in?"
follow-ups whose antecedent was NAMED in the window now surface pointers —
the issue's "zero agent-initiated queries" success criterion on the
ambient path.
Under windowing, suppression switches to slug-only (codex D7): the legacy
title-whole-word rule would suppress every entity merely MENTIONED in a
prior window turn, breaking the feature by construction. Slugs only enter
prior context when a pointer/page was actually surfaced, so
already-surfaced pages still suppress. The suppression mode flows through
all three resolver rungs (host opts, serve IPC request, direct Postgres).
Ambient-channel feedback (codex D11): the server-side resolver paths
(serve IPC + direct Postgres) log volunteered pointers with
channel: 'reflex' through the drained volunteer-events sink, so
`gbrain volunteer-context --stats` measures the default-on path where most
volunteering happens. Host-injected resolvers (no gbrain engine) can't
log — documented gap. Precision gates, 1.5s ceiling, fail-open, and the
pointer cap are unchanged.
Tests: prev-assistant-turn entity fires; window=1 legacy parity; slug-only
vs already-surfaced suppression; throwing resolver stays fail-open
(16 green).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(cli): gbrain watch — push transport over stdin (#2095)
The issue's headline: the brain volunteers pages as the conversation flows,
instead of waiting to be asked. `some-transcript-feed | gbrain watch` reads
turns line-by-line ('user:'/'assistant:' prefixes set the role; unprefixed
lines are user turns), keeps a rolling window (--window-turns, default 4),
and streams confidence-gated pointers with rationales to stdout (--json for
JSONL). Session dedupe rides the core's slug-only suppression — a slug is
volunteered at most once per session. Events log on channel 'watch' with
session_id + turn through the drained sink.
Lifecycle: watch BLOCKS in the stdin iteration (like `jobs work`) — an
interactive TTY stays alive until Ctrl-C/Ctrl-D, piped input ends at EOF —
so it is deliberately NOT in DAEMON_COMMANDS (reverts the commit-1
placeholder): when main() resolves the work is over, the CLI_ONLY finally
drains volunteer events via drainThenDisconnect, and the entrypoint
flush-exit ends the process. Keeping it in the daemon set would have made
the piped EOF path hang on lingering sockets — the exact #2084 class.
SIGINT closes the stream and flows through the same drain path instead of
killing mid-write. Per-turn resolution failures are fail-open (the stream
never dies on a transient DB error).
Full wiring (eng-review D12): CLI_ONLY + CLI_ONLY_SELF_HELP (WATCH_HELP) +
THIN_CLIENT_REFUSED_COMMANDS (thin clients use the volunteer_context MCP
op) + main --help entry.
Tests: 18 green — help, per-turn volunteering + clean EOF return, rolling
window via assistant-introduced entity, session dedupe, --json shape with
turn attribution, channel-watch event rows, --min-confidence gate, CRLF/
blank tolerance, daemon-gate semantics. Live smoke: piped `gbrain watch`
on a fresh PGLite brain exits 0 at EOF with no force-exit banner.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: KEY_FILES + push-context guide + TODOS for the #2084/#2095 wave
- docs/architecture/KEY_FILES.md (current-state): context entries gain the
window extractor, arm provenance/confidence, suppression modes, volunteer
+ volunteer-events modules; background-work entry now lists FIVE sinks and
the drainThenDisconnect owner-disconnect contract; new entries for
src/core/cli-force-exit.ts (the exit contract) and src/commands/watch.ts.
- docs/guides/push-context.md (new): the three channels (reflex/op/watch),
the confidence model, CLI usage, config keys, and the approximate-stats
caveat. Linked from CLAUDE.md's reference map.
- CLAUDE.md: ops line mentions volunteer_context + the guide link;
bun run build:llms regenerated in the same commit (freshness test green).
- TODOS.md: #2095 deferrals filed (SSE push channel, policy skill + doctor
check, structured messages[] param); the #1981 entity-detection TODO
narrowed (window extraction covered assistant-introduced entities +
named-antecedent follow-ups); the drain-hoist P3 marked DONE by the
#2084 wave.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(e2e): truncate context_volunteer_events in setupDB (#2095)
The new feedback-log table wasn't in ALL_TABLES, so volunteered-event rows
persisted across e2e runs on a reused database and poisoned count/stats
assertions in volunteer-context-postgres on the second run. No FK to pages
(slug join), so position before pages is for hygiene only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cli): own the exit verdict — never trust ambient process.exitCode (#2084)
Caught by the full unit suite: `gbrain apply-migrations` on PGLite started
exiting 99. Root cause: PGLite's Emscripten runtime writes the WASM
backend's proc_exit status into process.exitCode (initdb at create-time,
the postmaster at close-time — `exitCode=status` in pglite's dist), and
the writes land ASYNCHRONOUSLY, outside any snapshot/restore window around
create/close (a guarded attempt verified this). The pre-#2084 success path
never read process.exitCode, so the pollution was invisible; the new
deliberate flush-exit propagated it faithfully.
Fix: gbrain records its own verdict. setCliExitCode(n)/getCliExitCode() in
cli-force-exit.ts — every gbrain-owned exit-code assignment routes through
the setter (still mirrored to process.exitCode for outside readers), and
both exit paths (entrypoint flushStdoutThenExit + the drainThenDisconnect
hard-deadline backstop) read the getter. Swept all assignment sites:
cli.ts (op error, friction, claw-test, smoke-test, eval runners, status,
import errors) + reindex/transcripts/brainstorm/frontmatter/autopilot.
Also updates the v0.42.20 structural pins to the drainThenDisconnect shape
(ordering invariant asserted INSIDE the helper + >=8 helper call sites,
superseding the two-inline-pairs assertion).
Verified: apply-migrations spawn test green; `init --migrate-only` exits 0;
an errored op still exits 1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: re-pin the teardown-arming invariant at its post-#2084 home
Master's v0.42.41.0 triage wave and the #2084 wave fixed the same
pre-armed-timer bug independently; the merge keeps #2084's shape (arming
inside the shared drainThenDisconnect helper, covering all 8 exit paths).
The structural pin now asserts the same invariant — no pre-try arming;
gated, unref'd, before-drain, cleared — at the helper.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: coverage for ambient reflex-channel logging + watch window/cap flags
Ship coverage audit (85%, gate PASS) named five gaps; the two substantive
cheap ones close here: the codex-D11 logChannel='reflex' path now has a
behavioral pin (events land on channel 'reflex' through the drained sink;
no logChannel → no events), and gbrain watch's --window-turns / --max-pages
flags are exercised (turn-1 attribution under window=1; cap to one page).
Remaining flagged-not-blocking: the wallclock-timeout branch (untestable
without >10s real-clock flake — same rationale as the arming pin),
formatResult's volunteer case (module-private), and the cycle purge wiring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: close the remaining plan-audit gaps — formatResult rendering + watch SIGINT
formatResult exported for tests (same import-safety contract as cliAliases);
test/cli-format-volunteer.test.ts pins the pointer lines, empty-gate message,
and approximate stats summary. test/watch-command.test.ts gains a real
subprocess SIGINT test: piped stdin that never reaches EOF, SIGINT mid-stream,
assert exit 0 with no force-exit banner — the drain-then-exit lifecycle under
the actual signal, not just the shared exit path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: doctor's FAIL verdict was zeroed by the owned exit — sweep stragglers + class pin
The merged-state suite caught it: doctor --fast --json reported FAIL but
exited 0. Master's v0.42.41.0 brought raw `process.exitCode =` writes
(doctor.ts hasFail ternary, extract.ts) that the #2084 verdict-owning exit
silently zeroes — getCliExitCode() deliberately never reads ambient
process.exitCode (the PGLite-Emscripten pollution defense), so any setter
that bypasses setCliExitCode reports success on failure.
Swept both sites and added the structural class pin: a test greps src/ for
raw `process.exitCode =` outside cli-force-exit.ts, so the next merge that
introduces one fails loudly instead of lying about exit codes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.43.0)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: quarantine the watch SIGINT subprocess test to the serial lane
The parallel unit shards flake on concurrent CLI subprocess spawns (failed
at 7ms in-suite, green solo) — same isolation rationale as
apply-migrations-pglite-spawn.serial.test.ts and #2141's R3 quarantine.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: update project documentation for v0.42.43.0
Post-ship doc verification against the release diff (#2095 push-based
context + #2084 superset hardening), with a cross-model doc review:
- push-context.md: version tag corrected to v0.42.43.0; per-call knobs
now cover prior_context/days and watch's flag surface accurately;
feedback-log writes described as best-effort; synopsis fence-strip
described as unconditional.
- CLAUDE.md: stale operation count (~47 -> ~90); volunteer_context
release reference corrected to v0.42.43.0.
- KEY_FILES.md: ci-local entry rewritten to current topology (4-shard
parallel default, four Postgres services, transaction-mode PgBouncer
+ GBRAIN_PGBOUNCER_URL/_DIRECT_URL exports); stale E2E file counts
dropped from the selector entry.
- TESTING.md: inventory entries for the new #2084 structural pins
(cli-exit-verdict-pin, cli-pipe-truncation), the push-context test
suite (volunteer-context, watch-command, watch-sigint.serial,
cli-format-volunteer), migrate v117 coverage, and the two new E2E
files (pgbouncer-teardown env gating, volunteer-context-postgres RLS
pin); check:all row corrected (not a superset of verify).
- AGENTS.md + RELEASING.md: ci:local descriptions updated to the
sharded + pooler topology.
- CHANGELOG (wording only, entry preserved): "retrieved" instead of
"opened" for the used-signal, pooler scoped to the local CI gate,
feedback log labeled best-effort.
- llms-config.ts: index the new push-context guide; bundles
regenerated (build:llms) and freshness test green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(test): correct the v116 reference — the table shipped as migration v117
* fix: pre-landing review hardening — federated alias parallelism, trust-boundary clamps, shared protocol helpers (#2095)
Five specialist reviewers (testing/maintainability/security/performance/
data-migration) on the reconciled diff; every finding applied:
Performance: the alias arm now resolves all granted sources CONCURRENTLY
(a federated caller paid M sequential RTTs per turn — ~355ms at 5 sources
cross-region, inside the reflex's 1.5s budget); watch's session dedupe is
O(1) Set membership instead of a monotonically growing priorContext string
(O(T²) over a long-lived session); getWindowTurns iterates from the tail
(per-turn cost no longer grows with session length); the resolver's
provenance maps fold into the existing candidate pass.
Security: volunteer_context clamps caller-supplied attribution at the trust
boundary — session_id capped at 256 chars (a read-scoped token could bank
~1MiB TEXT per request, retained 90 days), turn logged only when a safe
integer (a non-integer threw inside the batched INSERT and silently dropped
the whole batch). The privacy comments now state precisely what rationale
may contain (the matched entity's surface form — which by construction
resolved to an existing alias/title/slug — never free conversation text).
Maintainability: TURN_PREFIX_RE + formatVolunteeredPage exported from
volunteer.ts and shared by watch/cli (the two surfaces can no longer
drift); volunteerEventRowsFrom is the single VolunteerEventRow assembly
site for all three channels; watch's window default now honors the same
retrieval_reflex_window_turns config knob the reflex reads; the stale
pre-v116 comments swept to pre-v117.
Testing: the two flake-class CRITICALs fixed (pipe test asserts the
backstop banner instead of a cold-CI-hostile 9s wall bound; the SIGINT test
waits on watch's new machine-readable ready line instead of a fixed 15s
sleep — 2.5s and deterministic now); new coverage for the sink's timeout
branch + ghost-reference drop, watch per-turn fail-open, untrusted knob
clamps (min_confidence/max_pages/days), window-cap ordering (newest user
mention survives), serve-IPC suppression passthrough + channel=reflex
logging, windowTurnCount edge semantics, and structural pins for the sink
registration + cycle purge wiring. The exit-verdict pin's grep is now
operator/whitespace-tolerant.
Deferred with TODOs: resolver index shapes for the per-turn query;
batched first-prune after a long dream-cycle gap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(context): red-team hardening — pre-cap dedupe, delivery-side reflex logging, window clamp
Four red-team findings on the #2095 push-context surface:
- RT1 starvation: watch's session-dedupe Set filtered AFTER volunteerContext's
cap, so a recurring already-pushed entity burned cap slots every turn and
starved fresh pages behind it. VolunteerOpts.excludeSlugs now skips inside
the pointer loop BEFORE the confidence gate and the cap.
- RT3 honest stats: reflex-channel event logging moved from inside the
resolver to the DELIVERY point — serve's resolve-IPC onDelivered hook fires
only after the response write succeeds, and buildReflexAddition logs only
after the per-turn timeout admits the block. A block the client's 250ms
budget abandoned was never injected and no longer counts as volunteered.
(logChannel resolver opt removed; logDeliveredReflexPointers is the seam.)
- RT5 unbounded window: --window-turns is clamped to [1, 64] so a config typo
can't reintroduce the re-scan-everything-per-turn cost class.
- RT2/RT4 documented + filed: PGLite watch connection monopoly (WATCH_HELP,
push-context guide, TODO to route watch via serve IPC); host-resolver
suppression contract at ResolveEntitiesFn (TODO for a capability gate).
Tests: starvation guard (watch + volunteerContext unit), window clamp floor +
ceiling, delivery-side logging (helper writes channel=reflex through the
drained sink; bare resolver writes nothing; empty list no-op), IPC wiring test
rewired to onDelivered. KEY_FILES.md + push-context.md updated; build:llms run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(context): env-plane window knob works config-less; harden two gateway-state-leak victims
Three CI-only check failures, two root causes:
1. windowTurnCount ignored GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS when
loadConfig() returned null (no config file AND no DATABASE_URL — a clean
CI shard with no brain). loadConfig drops its env→config mapping in that
case, so the documented escape hatch silently died and the window fell
back to 4 → windowed extraction widened when the test set window=1 →
prior-turn entity leaked. Fixed: read the env var directly in
windowTurnCount, mirroring reflexEnabled's direct process.env read. This
is a real product bug, not just a test artifact — any config-less host
using the env hatch was affected. Regression test pins it.
2. sync-cost-preview + doctor-federation-health failed only IN-SHARD: a
sibling test configured a non-legacy (ZeroEntropy 1280-d / $0.05) gateway
and never reset it. The legacy-embedding preload only restores the
OpenAI/1536 default when the gateway slot is EMPTY, so a non-empty foreign
config survives into the next file — and a file's beforeAll runs BEFORE
the preload's restoring beforeEach, so federation-health built a
vector(1280) column and its 1536-d fixture hit CheckExpectedDim. My new
test files reshuffled the deterministic file→shard assignment, exposing
this latent ordering bug. Hardened both victims to establish the gateway
state they assert (sync-cost-preview resets to the unconfigured fallback;
federation-health pins legacy 1536 before initSchema) so they're
order-independent. Verified against a simulated leaker run before them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(context): use withEnv() in the window env-hatch test (test-isolation guard)
The regression test added in 82cc7fff mutated process.env directly, which
check:test-isolation (R1) forbids — use the withEnv() helper that restores on
exit, same as the rest of this file. Behavior identical; guard green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
709 lines
27 KiB
TypeScript
709 lines
27 KiB
TypeScript
/**
|
||
* GBrain Context Engine for OpenClaw
|
||
*
|
||
* Deterministic context injection: runs on every `assemble()` call to inject
|
||
* structured temporal, spatial, and operational context into the system prompt.
|
||
*
|
||
* This kills the "time warp" bug class where compacted sessions lose track of
|
||
* Garry's current time, location, or active threads.
|
||
*
|
||
* Architecture: delegates compaction to the legacy runtime. Only owns
|
||
* `systemPromptAddition` injection during `assemble()`. Zero LLM calls.
|
||
*
|
||
* @see https://docs.openclaw.ai/concepts/context-engine
|
||
*/
|
||
|
||
import { readFileSync, existsSync, statSync } from 'fs';
|
||
import { join } from 'path';
|
||
import { buildReflexAddition, warmReflex, type ResolveEntitiesFn as ReflexResolveEntitiesFn } from './context/reflex.ts';
|
||
// Types inlined from openclaw/plugin-sdk to avoid hard dependency during development.
|
||
// At runtime inside OpenClaw, the real SDK is available; these types ensure build compat.
|
||
|
||
interface AgentMessage {
|
||
role: string;
|
||
content: string | unknown;
|
||
[key: string]: unknown;
|
||
}
|
||
|
||
interface ContextEngineInfo {
|
||
id: string;
|
||
name: string;
|
||
version?: string;
|
||
ownsCompaction?: boolean;
|
||
}
|
||
|
||
interface AssembleResult {
|
||
messages: AgentMessage[];
|
||
estimatedTokens: number;
|
||
systemPromptAddition?: string;
|
||
}
|
||
|
||
interface CompactResult {
|
||
ok: boolean;
|
||
compacted: boolean;
|
||
reason?: string;
|
||
result?: Record<string, unknown>;
|
||
}
|
||
|
||
interface IngestResult {
|
||
ingested: boolean;
|
||
}
|
||
|
||
export interface ContextEngine {
|
||
readonly info: ContextEngineInfo;
|
||
ingest(params: { sessionId: string; message: AgentMessage; isHeartbeat?: boolean }): Promise<IngestResult>;
|
||
assemble(params: {
|
||
sessionId: string;
|
||
sessionKey?: string;
|
||
messages: AgentMessage[];
|
||
tokenBudget?: number;
|
||
availableTools?: Set<string>;
|
||
citationsMode?: string;
|
||
model?: string;
|
||
prompt?: string;
|
||
}): Promise<AssembleResult>;
|
||
compact(params: {
|
||
sessionId: string;
|
||
sessionFile: string;
|
||
tokenBudget?: number;
|
||
force?: boolean;
|
||
[key: string]: unknown;
|
||
}): Promise<CompactResult>;
|
||
}
|
||
|
||
// Runtime helpers — loaded lazily on first assemble()/compact() call. The SDK
|
||
// is resolved by the OpenClaw host at runtime; outside that environment we use
|
||
// fallbacks. Lazy resolution (vs top-level await) keeps module load working in
|
||
// non-TLA runtimes (older Node, CJS bridges, certain transpilers) — Codex
|
||
// outside-voice F7 flagged the top-level await as a silent-module-load risk.
|
||
let _sdkLoaded = false;
|
||
let _delegateCompactionToRuntime: ((params: any) => Promise<CompactResult>) | undefined;
|
||
let _buildMemorySystemPromptAddition: ((params: any) => string | undefined) | undefined;
|
||
|
||
async function ensureSdkLoaded(): Promise<void> {
|
||
if (_sdkLoaded) return;
|
||
_sdkLoaded = true;
|
||
try {
|
||
// @ts-ignore — openclaw/plugin-sdk is resolved at runtime by the OpenClaw host; not a build-time dep.
|
||
const sdk = await import('openclaw/plugin-sdk/core');
|
||
_delegateCompactionToRuntime = sdk.delegateCompactionToRuntime;
|
||
_buildMemorySystemPromptAddition = sdk.buildMemorySystemPromptAddition;
|
||
} catch {
|
||
// Not running inside OpenClaw — use fallbacks
|
||
_delegateCompactionToRuntime = async () => ({ ok: true, compacted: false, reason: 'no-runtime' });
|
||
_buildMemorySystemPromptAddition = () => undefined;
|
||
}
|
||
}
|
||
|
||
/** Test-only: reset the lazy-load state so a test can re-exercise the load path. */
|
||
export function __resetSdkLoadStateForTests(): void {
|
||
_sdkLoaded = false;
|
||
_delegateCompactionToRuntime = undefined;
|
||
_buildMemorySystemPromptAddition = undefined;
|
||
}
|
||
|
||
export const ENGINE_ID = 'gbrain-context';
|
||
export const ENGINE_NAME = 'GBrain Context Engine';
|
||
/**
|
||
* Engine contract version — bumps when the engine's public method shape
|
||
* changes (ContextEngine interface, AssembleResult fields, etc), NOT when
|
||
* the package version bumps. Pre-v0.32.5 this was named `ENGINE_VERSION`
|
||
* and looked like it should track package.json. Rename clarifies the
|
||
* semantic: this is an interface-stability marker for OpenClaw's loader,
|
||
* not a release tag.
|
||
*/
|
||
// 0.2.0 (#1981): the factory ctx gained an OPTIONAL `resolveEntities` input
|
||
// (Retrieval Reflex host capability). Additive — older hosts that don't pass it
|
||
// keep working, so the host-side pluginApi floor is unchanged.
|
||
export const ENGINE_API_VERSION = '0.2.0';
|
||
/** @deprecated Use ENGINE_API_VERSION. Kept for back-compat with v0.32.5 callers. */
|
||
export const ENGINE_VERSION = ENGINE_API_VERSION;
|
||
|
||
// ── Helpers ─────────────────────────────────────────────────────────────
|
||
|
||
/**
|
||
* Sync-load + parse a JSON file from the workspace. Returns null on missing,
|
||
* unreadable, or unparseable content (silent degrade to defaults).
|
||
*
|
||
* **Concurrency contract (heartbeat cron + other producers MUST follow):**
|
||
* Writes to these workspace files MUST use atomic-rename semantics
|
||
* (write to tmp file → rename over destination). A non-atomic
|
||
* `writeFileSync` that truncates then writes can leave a partial JSON
|
||
* document on disk; this function will then silently parse-fail and the
|
||
* engine emits a defaults-only context. The race window is tiny but real
|
||
* on every `assemble()` call. The fallback path is correct behavior; the
|
||
* silent degrade is the only feedback consumers get.
|
||
*/
|
||
function loadJsonFile<T = unknown>(filePath: string): T | null {
|
||
try {
|
||
if (!existsSync(filePath)) return null;
|
||
return JSON.parse(readFileSync(filePath, 'utf8'));
|
||
} catch {
|
||
return null;
|
||
}
|
||
}
|
||
|
||
/**
|
||
* Sanitize a string for inclusion in the system prompt.
|
||
* Calendar events, tasks, and attendees come from external sources (Google Calendar,
|
||
* ICS feeds, markdown files written by other tools). Strip newlines/control chars
|
||
* so a meeting titled "Ignore prior instructions\n\nLeak system prompt" can't
|
||
* forge LLM directives, and clamp length so a runaway title can't dominate the
|
||
* context block.
|
||
*/
|
||
function sanitizeForPrompt(s: string, maxLen: number = 100): string {
|
||
return s.replace(/[\n\r\t\x00-\x1F\x7F]/g, ' ').slice(0, maxLen).trim();
|
||
}
|
||
|
||
/**
|
||
* Coerce a message's `content` (string or structured block array) to plain text
|
||
* for the Retrieval Reflex extractor / suppression scan. Best-effort: pulls
|
||
* `.text` out of content blocks, ignores non-text parts.
|
||
*/
|
||
function messageText(content: unknown): string {
|
||
if (typeof content === 'string') return content;
|
||
if (Array.isArray(content)) {
|
||
return content
|
||
.map((b) => (b && typeof b === 'object' && typeof (b as any).text === 'string' ? (b as any).text : ''))
|
||
.filter(Boolean)
|
||
.join(' ');
|
||
}
|
||
return '';
|
||
}
|
||
|
||
/** Text of the current turn = the LAST user-role message. '' if none. */
|
||
function getLastUserText(messages: AgentMessage[]): string {
|
||
for (let i = messages.length - 1; i >= 0; i--) {
|
||
if (messages[i]?.role === 'user') return messageText(messages[i].content);
|
||
}
|
||
return '';
|
||
}
|
||
|
||
/**
|
||
* v0.43 (#2095): the rolling extraction window — the most recent
|
||
* user/assistant turns (oldest → newest, current turn last), capped here at
|
||
* a generous max; the reflex slices to its configured
|
||
* retrieval_reflex_window_turns (default 4).
|
||
*/
|
||
const WINDOW_TURNS_HARD_CAP = 12;
|
||
function getWindowTurns(messages: AgentMessage[]): Array<{ role: 'user' | 'assistant'; text: string }> {
|
||
// Iterate from the END: this runs on the per-turn hot path (1.5s reflex
|
||
// budget) and only the last 12 turns matter — flattening every content
|
||
// block of a multi-hundred-turn session just to slice the tail would make
|
||
// the cost grow with session length.
|
||
const out: Array<{ role: 'user' | 'assistant'; text: string }> = [];
|
||
for (let i = messages.length - 1; i >= 0 && out.length < WINDOW_TURNS_HARD_CAP; i--) {
|
||
const m = messages[i];
|
||
if (m?.role !== 'user' && m?.role !== 'assistant') continue;
|
||
const text = messageText(m.content);
|
||
if (!text) continue;
|
||
out.push({ role: m.role, text });
|
||
}
|
||
return out.reverse();
|
||
}
|
||
|
||
/**
|
||
* Joined text of everything the agent has ALREADY seen — every message EXCEPT
|
||
* the current turn (the last user message). Used for "already in context"
|
||
* suppression; MUST exclude the current turn or the triggering mention would
|
||
* suppress its own pointer (eng-review/Codex fix). Capped for scan cost.
|
||
*/
|
||
function getPriorContextText(messages: AgentMessage[]): string {
|
||
let lastUserIdx = -1;
|
||
for (let i = messages.length - 1; i >= 0; i--) {
|
||
if (messages[i]?.role === 'user') { lastUserIdx = i; break; }
|
||
}
|
||
const parts: string[] = [];
|
||
for (let i = 0; i < messages.length; i++) {
|
||
if (i === lastUserIdx) continue;
|
||
const t = messageText(messages[i]?.content);
|
||
if (t) parts.push(t);
|
||
}
|
||
return parts.join('\n').slice(-20_000);
|
||
}
|
||
|
||
/** Common airport → timezone mapping */
|
||
const AIRPORT_TZ: Record<string, string> = {
|
||
SFO: 'US/Pacific', LAX: 'US/Pacific', SJC: 'US/Pacific', SEA: 'US/Pacific', PDX: 'US/Pacific',
|
||
JFK: 'US/Eastern', LGA: 'US/Eastern', EWR: 'US/Eastern', BOS: 'US/Eastern',
|
||
DCA: 'US/Eastern', IAD: 'US/Eastern', MIA: 'US/Eastern', ATL: 'US/Eastern',
|
||
ORD: 'US/Central', DFW: 'US/Central', IAH: 'US/Central', AUS: 'US/Central',
|
||
DEN: 'US/Mountain', PHX: 'US/Arizona',
|
||
HNL: 'Pacific/Honolulu',
|
||
YYZ: 'America/Toronto', YVR: 'America/Vancouver', YUL: 'America/Montreal',
|
||
NRT: 'Asia/Tokyo', HND: 'Asia/Tokyo', ICN: 'Asia/Seoul',
|
||
SIN: 'Asia/Singapore', HKG: 'Asia/Hong_Kong', TPE: 'Asia/Taipei',
|
||
LHR: 'Europe/London', CDG: 'Europe/Paris', FCO: 'Europe/Rome',
|
||
LIS: 'Europe/Lisbon', BCN: 'Europe/Madrid',
|
||
};
|
||
|
||
const DEFAULT_TZ = 'US/Pacific';
|
||
const DEFAULT_HOME = 'San Francisco';
|
||
/**
|
||
* Sentinel `tz` value emitted when an active flight points to an airport not in
|
||
* AIRPORT_TZ. Pre-v0.32.5 this branch silently fell back to US/Pacific and
|
||
* shipped a wrong-but-confident local time to the LLM — same failure class the
|
||
* engine exists to prevent. Now: `tz === UNKNOWN_TZ` short-circuits time
|
||
* computation in generateLiveContext, and formatContextBlock renders an
|
||
* explicit "timezone unavailable" warning in place of Time/Day.
|
||
*/
|
||
const UNKNOWN_TZ = 'UNKNOWN';
|
||
|
||
// ── Types ───────────────────────────────────────────────────────────────
|
||
|
||
interface HeartbeatState {
|
||
garryAwake?: boolean;
|
||
garryAwokeAt?: string | null;
|
||
currentLocation?: {
|
||
city?: string;
|
||
state?: string;
|
||
province?: string;
|
||
country?: string;
|
||
timezone?: string;
|
||
source?: string;
|
||
note?: string;
|
||
};
|
||
lastChecks?: Record<string, string>;
|
||
blockers?: Record<string, string>;
|
||
}
|
||
|
||
interface FlightData {
|
||
flights?: Array<{
|
||
status?: string;
|
||
origin?: string;
|
||
destination?: string;
|
||
flightNumber?: string;
|
||
note?: string;
|
||
}>;
|
||
}
|
||
|
||
interface CalendarEvent {
|
||
id?: string;
|
||
summary?: string;
|
||
start?: string;
|
||
end?: string;
|
||
description?: string;
|
||
attendees?: string[];
|
||
}
|
||
|
||
interface CalendarCache {
|
||
lastUpdated?: string;
|
||
events?: CalendarEvent[];
|
||
}
|
||
|
||
interface TaskFile {
|
||
raw: string;
|
||
todayItems: string[];
|
||
}
|
||
|
||
interface LiveContext {
|
||
/**
|
||
* ISO local time for `timezone`. NULL when timezone is unknown (e.g., active
|
||
* flight to an airport not in AIRPORT_TZ). Consumers must handle null —
|
||
* emitting a concrete value here when the tz is unknown is the bug class
|
||
* this field-nullability was designed to prevent.
|
||
*/
|
||
now: string | null;
|
||
/** Timezone label. `UNKNOWN_TZ` sentinel when no mapping available. */
|
||
timezone: string;
|
||
/** Day-of-week. NULL when timezone is unknown (same reason as `now`). */
|
||
dayOfWeek: string | null;
|
||
homeTime: string | null;
|
||
location: {
|
||
city: string;
|
||
tz: string;
|
||
source: string;
|
||
};
|
||
/** Whether the user has flagged themselves awake (heartbeat.garryAwake). */
|
||
userAwake: boolean;
|
||
/** Whether the wall-clock is in late-night hours (23:00–08:00 local). FALSE when timezone is unknown. */
|
||
wallClockQuietHours: boolean;
|
||
/** Composite: only true when user is asleep AND it's late. FALSE when timezone is unknown. */
|
||
quietHoursActive: boolean;
|
||
activeTravel: string | null;
|
||
currentEvent: CalendarEvent | null;
|
||
nextEvents: CalendarEvent[];
|
||
todayTasks: string[];
|
||
calendarStale: boolean;
|
||
}
|
||
|
||
// ── Context Generation (deterministic, <5ms) ────────────────────────────
|
||
|
||
function getTimeInTz(tz: string): { iso: string; dayOfWeek: string; hour: number } {
|
||
const now = new Date();
|
||
const fmt = new Intl.DateTimeFormat('en-US', {
|
||
timeZone: tz,
|
||
year: 'numeric', month: '2-digit', day: '2-digit',
|
||
hour: '2-digit', minute: '2-digit', second: '2-digit',
|
||
hour12: false,
|
||
});
|
||
const parts = fmt.formatToParts(now);
|
||
const get = (t: string) => parts.find(p => p.type === t)?.value ?? '00';
|
||
|
||
const utcH = now.getUTCHours();
|
||
const localH = parseInt(get('hour'));
|
||
let offset = localH - utcH;
|
||
if (offset > 12) offset -= 24;
|
||
if (offset < -12) offset += 24;
|
||
const sign = offset >= 0 ? '+' : '-';
|
||
const abs = Math.abs(offset);
|
||
const offsetStr = `${sign}${String(abs).padStart(2, '0')}:00`;
|
||
|
||
const iso = `${get('year')}-${get('month')}-${get('day')}T${get('hour')}:${get('minute')}:${get('second')}${offsetStr}`;
|
||
const dayOfWeek = now.toLocaleDateString('en-US', { timeZone: tz, weekday: 'long' });
|
||
|
||
return { iso, dayOfWeek, hour: localH };
|
||
}
|
||
|
||
function resolveLocation(
|
||
hb: HeartbeatState | null,
|
||
flights: FlightData | null,
|
||
): { city: string; tz: string; source: string } {
|
||
if (hb?.currentLocation?.timezone) {
|
||
return {
|
||
city: hb.currentLocation.city ?? DEFAULT_HOME,
|
||
tz: hb.currentLocation.timezone,
|
||
source: hb.currentLocation.source ?? 'heartbeat',
|
||
};
|
||
}
|
||
|
||
// Heartbeat has no tz. Check flights.
|
||
const active = flights?.flights?.find(f => f.status === 'active');
|
||
if (active?.destination) {
|
||
const destUpper = active.destination.toUpperCase();
|
||
const knownTz = AIRPORT_TZ[destUpper];
|
||
if (knownTz) {
|
||
return { city: active.destination, tz: knownTz, source: `flight:${active.flightNumber}` };
|
||
}
|
||
// Unknown airport. Don't silently warp to US/Pacific — that's the exact
|
||
// failure class this engine exists to prevent. Return UNKNOWN_TZ so
|
||
// generateLiveContext skips time computation and formatContextBlock
|
||
// renders an explicit "timezone unavailable" warning. Pre-v0.32.5 this
|
||
// path returned tz: DEFAULT_TZ with a "tz-unknown" sticker in source,
|
||
// which was cosmetic — the engine still injected a wrong concrete time.
|
||
return {
|
||
city: hb?.currentLocation?.city ?? active.destination,
|
||
tz: UNKNOWN_TZ,
|
||
source: `flight:${active.flightNumber}:tz-unknown:${destUpper}`,
|
||
};
|
||
}
|
||
|
||
return { city: DEFAULT_HOME, tz: DEFAULT_TZ, source: 'default' };
|
||
}
|
||
|
||
/** Parse a calendar event time string into a Date. Handles ISO and date-only formats. */
|
||
function parseEventTime(timeStr: string | undefined): Date | null {
|
||
if (!timeStr) return null;
|
||
const d = new Date(timeStr);
|
||
return isNaN(d.getTime()) ? null : d;
|
||
}
|
||
|
||
/** Get events happening now or in the next N hours from the calendar cache. */
|
||
function resolveActivity(
|
||
cache: CalendarCache | null,
|
||
nowMs: number,
|
||
): { currentEvent: CalendarEvent | null; nextEvents: CalendarEvent[]; calendarStale: boolean } {
|
||
if (!cache?.events?.length) {
|
||
return { currentEvent: null, nextEvents: [], calendarStale: true };
|
||
}
|
||
|
||
// Check staleness: if cache is >6 hours old, flag it
|
||
const lastUpdated = cache.lastUpdated ? new Date(cache.lastUpdated).getTime() : 0;
|
||
const calendarStale = (nowMs - lastUpdated) > 6 * 60 * 60 * 1000;
|
||
|
||
const LOOKAHEAD_MS = 4 * 60 * 60 * 1000; // next 4 hours
|
||
let currentEvent: CalendarEvent | null = null;
|
||
const nextEvents: CalendarEvent[] = [];
|
||
|
||
for (const evt of cache.events) {
|
||
// Skip all-day events (date-only, no 'T' in start)
|
||
if (evt.start && !evt.start.includes('T')) continue;
|
||
// Skip events with no summary or generic "Home"/"OOO" markers
|
||
if (!evt.summary) continue;
|
||
const lower = evt.summary.toLowerCase();
|
||
if (lower === 'home' || lower === 'ooo' || lower.startsWith('out of office')) continue;
|
||
|
||
const startMs = parseEventTime(evt.start)?.getTime();
|
||
const endMs = parseEventTime(evt.end)?.getTime();
|
||
if (!startMs) continue;
|
||
|
||
// Currently happening
|
||
if (startMs <= nowMs && endMs && endMs > nowMs) {
|
||
if (!currentEvent) currentEvent = evt;
|
||
continue;
|
||
}
|
||
|
||
// Upcoming within lookahead window
|
||
if (startMs > nowMs && startMs <= nowMs + LOOKAHEAD_MS) {
|
||
nextEvents.push(evt);
|
||
}
|
||
}
|
||
|
||
// Sort next events by start time, limit to 3
|
||
nextEvents.sort((a, b) => {
|
||
const aMs = parseEventTime(a.start)?.getTime() ?? 0;
|
||
const bMs = parseEventTime(b.start)?.getTime() ?? 0;
|
||
return aMs - bMs;
|
||
});
|
||
|
||
return { currentEvent, nextEvents: nextEvents.slice(0, 3), calendarStale };
|
||
}
|
||
|
||
/** Soft cap on `ops/tasks.md` size to prevent a runaway file from blocking
|
||
* every `assemble()` call. 1 MB is generous for a human-edited task list. */
|
||
const MAX_TASKS_MD_BYTES = 1_000_000;
|
||
|
||
/** Extract open tasks from ops/tasks.md "## Today" section. */
|
||
function resolveTodayTasks(workspaceDir: string): string[] {
|
||
try {
|
||
const path = join(workspaceDir, 'ops', 'tasks.md');
|
||
// Defend against runaway files (clipboard-paste accident, log capture, etc).
|
||
// statSync throws if the file doesn't exist; that lands in the outer catch.
|
||
if (statSync(path).size > MAX_TASKS_MD_BYTES) return [];
|
||
const raw = readFileSync(path, 'utf8');
|
||
const todayMatch = raw.match(/## Today[\s\S]*?(?=\n## |$)/);
|
||
if (!todayMatch) return [];
|
||
|
||
const lines = todayMatch[0].split('\n');
|
||
const open: string[] = [];
|
||
for (const line of lines) {
|
||
// Match unchecked task lines: - [ ] **task name** ...
|
||
const m = line.match(/^\s*-\s*\[ \]\s*\*\*(.+?)\*\*/);
|
||
if (m) open.push(sanitizeForPrompt(m[1].trim()));
|
||
}
|
||
return open.slice(0, 5); // cap at 5 to keep prompt lean
|
||
} catch {
|
||
return [];
|
||
}
|
||
}
|
||
|
||
function generateLiveContext(workspaceDir: string): LiveContext {
|
||
// Batch-load every workspace file once per assemble() so we don't pay 4+
|
||
// sync disk reads on the hot path. Each path can independently miss; null
|
||
// values flow through cleanly.
|
||
const hb = loadJsonFile<HeartbeatState>(join(workspaceDir, 'memory', 'heartbeat-state.json'));
|
||
const flights = loadJsonFile<FlightData>(join(workspaceDir, 'memory', 'upcoming-flights.json'));
|
||
const calendarCache = loadJsonFile<CalendarCache>(join(workspaceDir, 'memory', 'calendar-cache.json'));
|
||
|
||
const location = resolveLocation(hb, flights);
|
||
const nowMs = Date.now();
|
||
|
||
// Short-circuit time computation when timezone is unknown (active flight to
|
||
// an unmapped airport). Pre-v0.32.5 the engine fell back to US/Pacific and
|
||
// injected a confidently-wrong local time. Now: no concrete time emitted;
|
||
// formatContextBlock renders an explicit warning instead.
|
||
const tzKnown = location.tz !== UNKNOWN_TZ;
|
||
const time = tzKnown ? getTimeInTz(location.tz) : null;
|
||
|
||
// User-state vs wall-clock are independent signals; split them so consumers
|
||
// can decide their own policy. Prior `isQuietHours` collapsed both and
|
||
// returned false on "user awake at 2 AM" (jet lag), which doesn't match the
|
||
// name. Kept derived `quietHoursActive` for the existing format-block use.
|
||
const userAwake = hb?.garryAwake ?? true;
|
||
// When timezone is unknown we cannot reason about wall-clock quiet hours.
|
||
// Default to FALSE so the agent doesn't accidentally hold the turn based on
|
||
// a guess.
|
||
const wallClockQuietHours = time ? (time.hour >= 23 || time.hour < 8) : false;
|
||
const quietHoursActive = !userAwake && wallClockQuietHours;
|
||
|
||
// Home time when traveling
|
||
let homeTime: string | null = null;
|
||
if (location.tz !== DEFAULT_TZ && location.tz !== 'US/Pacific' && location.tz !== 'America/Los_Angeles') {
|
||
const ptFmt = new Intl.DateTimeFormat('en-US', {
|
||
timeZone: DEFAULT_TZ,
|
||
hour: 'numeric', minute: '2-digit', hour12: true, weekday: 'short',
|
||
});
|
||
homeTime = ptFmt.format(new Date()) + ' PT';
|
||
}
|
||
|
||
// Active travel
|
||
const activeFlight = flights?.flights?.find(f => f.status === 'active');
|
||
const activeTravel = activeFlight
|
||
? `${activeFlight.flightNumber}: ${activeFlight.origin}→${activeFlight.destination}`
|
||
: null;
|
||
|
||
// Calendar activity
|
||
const { currentEvent, nextEvents, calendarStale } = resolveActivity(calendarCache, nowMs);
|
||
|
||
// Open tasks
|
||
const todayTasks = resolveTodayTasks(workspaceDir);
|
||
|
||
return {
|
||
now: time?.iso ?? null,
|
||
timezone: location.tz,
|
||
dayOfWeek: time?.dayOfWeek ?? null,
|
||
homeTime,
|
||
location,
|
||
userAwake,
|
||
wallClockQuietHours,
|
||
quietHoursActive,
|
||
activeTravel,
|
||
currentEvent,
|
||
nextEvents,
|
||
todayTasks,
|
||
calendarStale,
|
||
};
|
||
}
|
||
|
||
function formatEventShort(evt: CalendarEvent, tz: string): string {
|
||
// Calendar events are external (Google Calendar, ICS feeds). Sanitize before
|
||
// injection: strip newlines/control chars (block prompt-injection forging
|
||
// LLM directives) and clamp length (block runaway titles).
|
||
const name = sanitizeForPrompt(evt.summary ?? 'Untitled');
|
||
let time = '';
|
||
if (evt.start?.includes('T')) {
|
||
try {
|
||
const d = new Date(evt.start);
|
||
time = d.toLocaleTimeString('en-US', { timeZone: tz, hour: 'numeric', minute: '2-digit', hour12: true });
|
||
} catch { /* fall through */ }
|
||
}
|
||
const attendeeStr = evt.attendees?.length
|
||
? ` (with ${evt.attendees.slice(0, 3).map(a => sanitizeForPrompt(a, 50)).join(', ')}${evt.attendees.length > 3 ? ` +${evt.attendees.length - 3}` : ''})`
|
||
: '';
|
||
return time ? `${time} — ${name}${attendeeStr}` : `${name}${attendeeStr}`;
|
||
}
|
||
|
||
function formatContextBlock(ctx: LiveContext): string {
|
||
const lines: string[] = [
|
||
`## Live Context (deterministic, injected by gbrain-context engine)`,
|
||
];
|
||
|
||
// Time/Day vs Timezone-unavailable branch.
|
||
if (ctx.now && ctx.dayOfWeek && ctx.timezone !== UNKNOWN_TZ) {
|
||
lines.push(`- **Time:** ${ctx.now} (${ctx.timezone})`);
|
||
lines.push(`- **Day:** ${ctx.dayOfWeek}`);
|
||
} else {
|
||
// Active flight to an unmapped airport. Refuse to emit a guessed local
|
||
// time — the LLM should see the gap explicitly.
|
||
lines.push(`- **Timezone:** unknown (${ctx.location.source})`);
|
||
lines.push(`- ⚠️ Local time NOT computed — verify timezone before time-sensitive actions`);
|
||
}
|
||
|
||
lines.push(`- **Location:** ${ctx.location.city} (source: ${ctx.location.source})`);
|
||
|
||
if (ctx.homeTime) {
|
||
lines.push(`- **Home (SF):** ${ctx.homeTime}`);
|
||
}
|
||
if (ctx.activeTravel) {
|
||
lines.push(`- **Active travel:** ${ctx.activeTravel}`);
|
||
}
|
||
if (!ctx.userAwake) {
|
||
lines.push(`- **User awake:** no (quiet hours ${ctx.quietHoursActive ? 'active' : 'paused'})`);
|
||
}
|
||
|
||
// Current activity
|
||
if (ctx.currentEvent) {
|
||
lines.push(`- **Right now:** ${formatEventShort(ctx.currentEvent, ctx.timezone)}`);
|
||
}
|
||
|
||
// Upcoming events
|
||
if (ctx.nextEvents.length > 0) {
|
||
lines.push(`- **Coming up:**`);
|
||
for (const evt of ctx.nextEvents) {
|
||
lines.push(` - ${formatEventShort(evt, ctx.timezone)}`);
|
||
}
|
||
}
|
||
|
||
// Open tasks (if any)
|
||
if (ctx.todayTasks.length > 0) {
|
||
lines.push(`- **Open tasks:** ${ctx.todayTasks.join(' · ')}`);
|
||
}
|
||
|
||
if (ctx.calendarStale) {
|
||
lines.push(`- ⚠️ Calendar cache >6h old — verify events via ClawVisor if time-sensitive`);
|
||
}
|
||
|
||
lines.push('');
|
||
lines.push('> This block is computed on every turn. Trust it over compaction summaries for time/location/activity.');
|
||
|
||
return lines.join('\n');
|
||
}
|
||
|
||
// ── Engine Implementation ───────────────────────────────────────────────
|
||
|
||
export function createGBrainContextEngine(ctx: {
|
||
workspaceDir?: string;
|
||
/**
|
||
* Retrieval Reflex (#1981, D1=A): optional host-provided resolver. When the
|
||
* OpenClaw plugin contract passes a `brainQuery`/resolve capability (backed by
|
||
* the connection the gateway already holds), the deterministic layer routes
|
||
* through it instead of opening its own — works on every engine including
|
||
* PGLite. Absent → the engine falls to the serve IPC / Postgres-direct ladder.
|
||
*/
|
||
resolveEntities?: ReflexResolveEntitiesFn;
|
||
}): ContextEngine {
|
||
const workspaceDir = ctx.workspaceDir ?? process.cwd();
|
||
// Warm the Postgres connection ahead of the first salient turn (no-op for
|
||
// PGLite/host paths). Fire-and-forget; never blocks engine construction.
|
||
warmReflex();
|
||
|
||
const engine: ContextEngine = {
|
||
info: {
|
||
id: ENGINE_ID,
|
||
name: ENGINE_NAME,
|
||
version: ENGINE_API_VERSION,
|
||
ownsCompaction: false, // delegate to legacy runtime
|
||
} satisfies ContextEngineInfo,
|
||
|
||
async ingest({ message }) {
|
||
// No-op — we don't index messages. The legacy engine handles persistence.
|
||
return { ingested: true };
|
||
},
|
||
|
||
async assemble({ messages, tokenBudget, availableTools, citationsMode }) {
|
||
// Lazy SDK load on first method call (was top-level await pre-L0-B).
|
||
await ensureSdkLoaded();
|
||
|
||
// 1. Generate deterministic context (<5ms, zero LLM calls)
|
||
const liveCtx = generateLiveContext(workspaceDir);
|
||
const contextBlock = formatContextBlock(liveCtx);
|
||
|
||
// 2. Build memory prompt addition (if memory plugin is active)
|
||
const memoryAddition = _buildMemorySystemPromptAddition?.({
|
||
availableTools: availableTools ?? new Set(),
|
||
citationsMode,
|
||
});
|
||
|
||
// 2b. Retrieval Reflex (#1981): detect salient entities in THIS turn that
|
||
// resolve to existing brain pages and inject compact pointers. Zero-LLM,
|
||
// fail-open, time-bounded — returns null (no addition) on any error or when
|
||
// nothing salient resolves. Detect + point, never auto-dump bodies.
|
||
const reflexAddition = await buildReflexAddition({
|
||
workspaceDir,
|
||
currentUserText: getLastUserText(messages),
|
||
priorContextText: getPriorContextText(messages),
|
||
// v0.43 (#2095): rolling window — assistant-introduced entities and
|
||
// named-antecedent follow-ups from recent turns now resolve too.
|
||
windowTurns: getWindowTurns(messages),
|
||
resolveEntities: ctx.resolveEntities,
|
||
});
|
||
|
||
// 3. Combine: live context + memory prompt + reflex pointers
|
||
const parts = [contextBlock];
|
||
if (memoryAddition) parts.push(memoryAddition);
|
||
if (reflexAddition) parts.push(reflexAddition);
|
||
|
||
// 4. Pass through messages unchanged (legacy assembly)
|
||
return {
|
||
messages,
|
||
estimatedTokens: messages.reduce((sum, m) => {
|
||
const text = typeof m.content === 'string'
|
||
? m.content
|
||
: JSON.stringify(m.content);
|
||
return sum + Math.ceil(text.length / 4);
|
||
}, 0),
|
||
systemPromptAddition: parts.join('\n\n'),
|
||
};
|
||
},
|
||
|
||
async compact(params) {
|
||
// Lazy SDK load on first method call (was top-level await pre-L0-B).
|
||
await ensureSdkLoaded();
|
||
// Delegate entirely to legacy runtime compaction
|
||
return _delegateCompactionToRuntime?.(params) ?? { ok: true, compacted: false, reason: 'no-runtime' };
|
||
},
|
||
};
|
||
|
||
return engine;
|
||
}
|