193 Commits

Include renames

Author	SHA1	Message	Date
4 people	3aa064bcc6	fix(test): make resetGateway restore the test baseline instead of unconfiguring (#3554) (#3557) ... * fix(test): make resetGateway restore the test baseline instead of unconfiguring (#3554) bunfig.toml's legacy-embedding-preload pins the gateway once at process start to openai:text-embedding-3-large @ 1536, but resetGateway() wiped that pin to _config = null. The next test file's beforeAll engine-connect then reconfigured from the SHIPPED default (zembed-1 @ 1280) before the preload's per-test beforeEach could restore anything, and every 1536-d fixture in that file failed with `expected 1280 dimensions, not 1536`. Which file pairs collided depended on shard bin-packing, so adding ANY test file reshuffled the mines (this is what blocks #3545). Fix: the preload registers its config as a reset baseline via a new test-only seam (__setGatewayResetBaselineForTests); resetGateway() clears all module state as before, then re-applies the baseline. All 93 existing resetGateway() call sites get the correct behavior with zero edits. Production is untouched: nothing in src/ calls resetGateway() or the setter, so the baseline is never registered outside tests and resetGateway() still fully unconfigures there. Five tests genuinely need an unconfigured gateway (no_gateway_config diagnosis, isAvailable=false, the #2590 cold-gateway path, the registry builtin-default tier); they switch to the new __unconfigureGatewayForTests. Two files' hand-rolled "restore the legacy pin in afterAll/finally" workarounds for this exact bug are now redundant and simplified away. Guard test (test/ai/gateway-reset-baseline.test.ts) pins the contract: 1536/openai immediately after resetGateway(), transports still cleared, hard-unconfigure still available. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(test): restore preload's GBRAIN_AUDIT_DIR instead of deleting it (#3554 sibling) Same bug class as the gateway fix in this PR: state set once by a bunfig preload (audit-dir-preload's scratch GBRAIN_AUDIT_DIR), wiped by one file's cleanup, blast radius decided by shard bin-packing. In shard 6, test/minions-shell.test.ts (position 12) unconditionally deleted the var in afterAll; test/audit/audit-dir-preload.test.ts (position 93) then found it undefined and failed 3 tests — and every file in between wrote audit fixtures toward the operator's real ~/.gbrain/audit/. Fix: capture the prior value at file load and conditionally restore it, the same inline save/restore pattern 11 sibling files already use. test/e2e/skill-brain-first.test.ts had the identical unconditional delete in afterEach; fixed the same way. Sweep of every `delete process.env.GBRAIN_AUDIT_DIR` in test/ confirms all remaining sites are conditional restores. Ordered-pair proof (minions-shell.test.ts then audit/audit-dir-preload.test.ts, one process): 3 fail on master, 43/43 pass with this fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-07-28 17:08:27 -07:00
4 people	176836f84d	fix(embed): stamp the real model on chunk provenance and keep contextual prefixes on --stale (#3461, #3507) (#3538) ... #3461 — insertChunks stamped DEFAULT_EMBEDDING_MODEL on chunk provenance whenever the AI gateway was unconfigured: getEmbeddingModel() throws rather than returning falsy, so the reland's '|| resolvedModel' guard (e1919fab) was dead code and the catch path kept the compile-time constant. Both engines now fall back to the brain's own config.embedding_model row on the throw path, with the compiled default as last resort. Same-line sibling: the ON CONFLICT clause overwrote 'model' via COALESCE even when the existing vector was preserved — 'model' now mirrors the 'embedding' CASE branch-for-branch so the label always describes whichever vector wins the upsert. The initSchema sizing sites drop their dead '||' terms too. #3507 — every plain re-embed path (embed <slug>, --all, --stale, and the embed-backfill Minion loop) embedded raw chunk_text, silently stripping the contextual-retrieval prefixes the sync path applied — and embed --stale is the NORMAL post-model-migration path. All four sites now wrap through wrapChunkTextsForStoredMode(), reproducing the page's STORED convention (pages.contextual_retrieval_mode): title/per_chunk_synopsis pages get the title-tier prefix, fenced_code chunks stay unwrapped (D20-T4), unstamped pages stay raw. A fully re-embedded per_chunk_synopsis page is restamped to 'title' so the mode column keeps describing the vectors actually in the DB. Deliberately NOT folding the contextual mode into currentEmbeddingSignature(): the convention is already recorded per-page (mode + corpus_generation), pages legitimately differ per-page so a global signature cannot represent it, and bumping signature semantics would force a full-corpus re-embed on upgrade. No KNOBS_HASH_VERSION change (no collision with #3514). Tests fail on unmodified master (6) and pass with the fix; postgres.js path verified against a real pgvector instance in addition to the PGLite suite. Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-07-28 14:12:08 -07:00
4 people	b252acfce3	fix(test): correct the deadline inversion in the durability-hook serial tests (#2943) (#3537) ... Three layers, verified by 10-run tallies (master: 7 pass/3 fail; branch: 10/10): 1. Deadline inversion: the hook tests' internal 8s poll deadlines sat behind bun's 5000ms default because no third-arg timeout was passed — and bun 1.3.14 ignores bunfig.toml's `timeout` key, so bare `bun test` runs died at 5s with the 8s budget unreachable. All three tests now pass 60_000 explicitly; deadlines raised to 30s for loaded-shard headroom. Same class fixed in test/e2e/jsonb-roundtrip.test.ts (four tests had no explicit timeout). 2. Root cause of the CI assertion failures: the test's git() helper spawned git WITHOUT `env: process.env`. Bun snapshots process.env at startup (the #2747 quirk), so the post-commit hook under test resolved GBRAIN_HOME to the operator's real ~/.gbrain — writing its log there (polluting the real brain-push.log every run) while the test polled the temp log. The LOCAL-ONLY assertion only passed when beforeEach's scaffolding hook push happened to still be in flight, lose the ref race, and retry after origin pointed at the dead path — a load-dependent accident. env is now passed everywhere, making the intended signal deterministic (~1s). 3. index.lock race (the third observed CI form): beforeEach now waits for the scaffolding commit's detached brain_push to write its terminal log line before handing the repo to the test, so its pull-rebase fallback can't take .git/index.lock under the test body's own git calls. Poll loops untouched — they were already correct; the 150 is the poll interval. Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-07-28 13:58:09 -07:00
3 people	a104f98dca	fix(cycle): extract_facts guard counts only active legacy rows; reconcile preserves forget records (#2646) (#3474) ... master's empty-fence guard counts soft-expired legacy rows (row_num IS NULL, expired_at set), so forget_fact — the sanctioned removal path, which soft-expires rather than deletes — can never drain the backlog: apply-migrations no-ops (already marked applied) and the guard stays triggered forever, jamming extract_facts. Narrowed re-send of #3252-sibling #3234, scoped to exactly what the maintainer named reviewable: "one predicate plus the reconcile-preservation guard." - Guard predicate: the legacy COUNT adds `AND f.expired_at IS NULL`, so each forget_fact visibly drains the pending counter. - Reconcile preservation: listExistingFactsForPage excludes soft-expired legacy rows (they are never fence-owned, so they must neither read as perpetually stale nor mask a fence row), and the two wipe call sites pass `preserveExpiredLegacy: true` so deleteFactsForPage keeps the forget record. The option is the minimal seam for that guard — implemented identically in both engines (~6 lines each). Explicitly NOT included from #3234 (per the close review): the drift-repair lane, the re-runnable migration orchestrator path, and the race-accounting layer. Fence-is-canonical semantics are preserved and pinned by test: if the fence still carries an expired legacy row's claim, the reconcile reinserts it as a fresh active fence-owned row (legacy DB-only forgets are documented non-durable; the expired row survives as the audit record of the forget). Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>	2026-07-28 11:44:17 -07:00
4 people	56aac51a08	feat(extract): quarantine lane for auto-extracted entities from untrusted input (#160) (#3458) ... * feat(extract): quarantine lane for auto-extracted entities from untrusted input (#160) extractAndEnrich regex-extracts entity names from arbitrary ingested text and creates people/ + companies/ stub pages. Those writes are now trust- gated end to end: - src/core/extraction-review.ts: new marker module (sibling of quarantine.ts / embed-skip.ts, frontmatter-key pattern, no migration). Untrusted-input stubs carry `provenance: auto-extracted` + `status: unverified`; the shared unverifiedExtractionFragment() is the single SQL source of truth for every consumer. - enrichment-service: enrichEntity/enrichEntities/extractAndEnrich take EnrichmentTrustOptions; only an explicit trusted:true writes authoritative pages (fail-closed, mirrors the OperationContext.remote invariant). Also threads sourceId through the write path. - retrieval: unverified stubs rank as ordinary content — skipped by the compiled-truth fusion boost (stampUnverifiedExtractions pre-fusion on all three hybrid paths + keyword-only opt-out) and by the people// companies/ namespace source-boost (guard inside buildSourceFactorCase, shared by both engines' search SQL). Results carry `unverified: true`. New engine method getUnverifiedExtractionPageIds in BOTH engines. - ops (contract-first): extract_entities (direct write only for ctx.remote === false + --trusted-extraction; everything else quarantines), extraction_pending (read, source-scoped list), extraction_review (owner-only batch promote/reject; promote flips status to verified keeping provenance for audit, reject soft-deletes). - doctor: unverified_extractions check warns on stubs older than N days (default 7) with the exact review commands. Tests: test/extraction-review.test.ts (PGLite: fail-closed matrix incl. remote-unset, fusion boost skip, review queue, doctor, hostile-transcript e2e proving fake entities land quarantined and rank below a verified page of equal lexical relevance) + test/e2e/extraction-review-postgres.test.ts (live Postgres parity, verified against pgvector:pg16). sql-ranking expectations updated to current state. Docs: KEY_FILES + RETRIEVAL + llms rebuild. Closes #160 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(extract): close vector-arm source-boost gap + harden extract_entities (#160 review round) Adversarial review of the quarantine lane found the people//companies/ 1.2x source factor still applied to unverified stubs inside searchVector's pre-LIMIT re-rank (a different multiplier from the fusion-level 2.0x the lane already cancels — and applied early enough to evict legitimate pages from the candidate pool, which nothing downstream can restore). - buildSourceFactorCase gains an optional unverifiedGuardColumn for the bare-slug re-rank form; both engines' hnsw_candidates CTEs now project the guard predicate as `unverified_stub` and the factor CASE checks it first. Wrong "fusion covers the vector arm" comment corrected. - extract_entities resource guards: 200k-char input cap (loud reject), 200-entity cap surfaced as `truncated` + `entities_found`; the library extractAndEnrich gets the same default cap. (OperationContext has no abort signal field — caps are the bound.) - extraction_review promote is now a targeted JSONB-merge UPDATE instead of putPage, so non-carried columns (page_kind, content_hash) can't be reset by the upsert. - extraction_pending applies buildVisibilityClause (archived-source stubs no longer list). - Wording: op description + module header now state the marker-strip assumption plainly (markers are ordinary frontmatter; the boundary against wholesale rewrite is put_page write authz) and document the CREATE-only scope of the lane. Tests: vector-arm factor-1.0 pinned on BOTH engines (PGLite unit + live Postgres e2e, identical basis embeddings → score ratio is the factor); resource-guard test (oversize reject + 300-entity flood capped at 200); guard-column form pinned in the buildSourceFactorCase unit test. search/ suite (340), sql-ranking, searchvector-maxpool, title-retrieval- arm, rrf-source-key, doctor, ops, cli suites all green; JSONB guards clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-27 18:01:35 -07:00
4 people	0bbaed2e48	v0.42.67.0 feat(migrate): provider-agnostic embedding migration — the path off ZeroEntropy (#3390, fixes #3391) (#3459) ... * feat(migrate): provider-agnostic embedding migration service — the path off ZeroEntropy (#3390) - gbrain migrate embeddings --to <provider:model> (alias: retrieval-upgrade): plan + cost preflight, consent gate (--yes / TTY confirm / non-TTY exit 2), live probe against the target provider before any mutation, env-override gate, schema dimension transition via the shared runSchemaTransition, dual-plane config write, NULL-signature-inclusive invalidation, query-cache purge, resumable re-embed through the standard embed pipeline (single-flight locks, backoff, pacing, stderr progress). Killed runs resume by re-running the same command; the NULL-embedding column is the checkpoint. - #3391 root-cause fix (both engines): countStaleChunks / sumStaleChunkChars / invalidateStaleSignatureEmbeddings accept includeNullSignature to lift the v108 grandfather clause; embed --stale warns loudly when a model swap leaves NULL-signature pages in the old embedding space, and --include-null-signature re-embeds them. Default sweep behavior unchanged. - knobs_hash v=12 → v=13 (prov=default legacy callers must not be served pre-migration cache rows). - migrate_embeddings op: scope admin, localOnly, hidden cliHints, hard remote refusal, needs_confirmation without yes=true. - One-shot post-upgrade ZE-sunset banner (ze_sunset_notice_shown) for brains resolving to a zeroentropyai:* embedding model or reranker. - doctor's dimension-mismatch repair hint now names the real command. - Docs: docs/guides/embedding-migration.md, KEY_FILES entries, spend-controls gate row. Tests: PGLite unit + full-lifecycle flow (interrupted-run resume), real-Postgres e2e (pgvector DDL path + #3391 predicate parity). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): satisfy check:test-isolation + bump the remaining knobs_hash pins - test/migrate-embeddings-flow.test.ts → .serial.test.ts: the file holds a temp GBRAIN_HOME + an installed fake embed transport for its whole lifecycle (beforeAll → afterAll), which withEnv() can't wrap. This also fixes the CI shard-pollution failure in test/ai/recipes-existing-regression.test.ts (that file passes solo on both master and this branch; the flow test's configureGateway + provider-key deletion was leaking into it inside the same shard process). - test/embedding-migration.test.ts: env-override case now uses withEnv(). - Bump the three remaining KNOBS_HASH_VERSION pins to 13 (cross-modal-phase1, search-alias-resolved-boost, search/knobs-hash-reranker). - Docs + llms bundles follow the test rename. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(test): wire the new Postgres e2e into the smart e2e selector map Changes to embed.ts / embedding-migration.ts / retrieval-upgrade-planner.ts / postgres-engine.ts now trigger test/e2e/migrate-embeddings-postgres.test.ts — the #3391 stale predicates and runSchemaTransition's DDL path behave differently on real pgvector than on PGLite, so the smart selector has to know. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(migrate): consult spend.posture in the embedding-migration consent gate The brief asked the gate to honor spend.posture; it previously didn't read it at all. Now it does — but deliberately does NOT bypass on tokenmax: posture waives the spend CEILING, and this gate also guards a destructive schema rebuild (existing vectors dropped, retrieval degraded until the re-embed finishes). Under tokenmax the dollar figure is marked informational on stderr and the confirmation is still asked; --yes stays the single scripted bypass. Pinned by a new case in the flow test so a later refactor can't quietly turn posture into a bypass. Guide + spend-controls table updated to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * wip: blocker fixes --------- Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-27 17:48:37 -07:00
5 people	d2ac2aef49	fix(synthesize): dedupe successful transcripts after corpus moves (#3424) ... * fix(synthesize): dedupe across corpus moves * fix(synthesize): dedupe legacy CHUNKED completions; keep plain-completed suppression Repairs three gaps in the corpus-move dedupe (v2 content-hash keys): 1. Legacy chunked completions now suppress v2 resubmission. The scan previously matched only keys ending ':<hash16>' (legacy single-chunk), so every transcript synthesized under the pre-v2 chunked family 'dream:synth:<path>:<hash16>:c<i>of<n>' re-ran as a full paid v2 synthesis after upgrade. findLegacyCompletion now also matches the chunked family, counting a transcript as done only when the FULL chunk set c0..c(n-1) completed; partial sets fall through to a fresh v2 run (reason: already_synthesized_legacy_chunked for full sets). 2+3. Legacy suppression reverts to plain status='completed', dropping the result->>'stop_reason' = 'end_turn' filter. This restores the pre-v2 cost-safe semantics (queue-level idempotency blocks re-submission of completed jobs regardless of stop_reason, pinned in test/minions.test.ts) and sidesteps the double-encoded-jsonb result rows the naive ->> read missed. The tightening was not documented as intended in the PR. Tests: legacy chunked full-set suppression + partial-set resubmission; double-encoded jsonb result row still recognized. Co-authored-by: zsimovanforgeops <justin@caddolandworks.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Forge (Ron) <forge@zsimovan.dev> Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-27 14:17:06 -07:00
5 people	16782aee7f	fix(sources): recover corrupted config shapes (#3420) ... * fix(sources): recover corrupted config shapes (#3401) Use one canonical normalizer for nested string and array-shaped source configs across federation reads, config writes, archive/restore, and doctor remediation.\n\nFixes #3401\nFixes #3402\nFixes #3403 Signed-off-by: arisgysel-design <arisgysel-design@users.noreply.github.com> * fix(sources): bind restoreSource federated patch via ::text::jsonb (#2339 class) restoreSource bound a JS JSON string to a bare $1::jsonb placeholder; postgres.js double-encodes that into a jsonb string scalar, so on the Postgres engine the coerced object || string-scalar concat evaluates as array-concat and restore RE-CORRUPTS the exact config shape this PR repairs. PGLite masks the bug (its driver parses the bind natively). Fix: bind through $1::text::jsonb per the repo JSONB rule. Adds the DATABASE_URL-gated Postgres regression (test/e2e/restore-source-config-jsonb-postgres.test.ts): seeds a corrupted string-scalar config, runs archive -> restore, asserts jsonb_typeof(config) = 'object' with the federated flag applied and pre-existing keys preserved. Verified red on the bare ::jsonb bind (config became a jsonb array) and green on the fix against a real pgvector Postgres; skips cleanly without DATABASE_URL. Co-authored-by: arisgysel-design <arisgysel-design@users.noreply.github.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Signed-off-by: arisgysel-design <arisgysel-design@users.noreply.github.com> Co-authored-by: arisgysel-design <arisgysel-design@users.noreply.github.com> Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-27 14:16:05 -07:00
jared-vossandGitHub	d014707e3c	feat(admin): manage OAuth source grants (#3383)	2026-07-27 14:12:07 -07:00
3 people	14f0674bcf	fix(synthesize): normalize Postgres receipt job ids (#3414) ... Co-authored-by: Forge (Ron) <forge@zsimovan.dev>	2026-07-27 13:29:37 -07:00
3 people	4871ae0c05	fix(upgrade): detect every newer release (#3404) (#3418) ... Signed-off-by: arisgysel-design <arisgysel-design@users.noreply.github.com> Co-authored-by: arisgysel-design <arisgysel-design@users.noreply.github.com>	2026-07-27 13:29:07 -07:00
4 people	ef7351247a	fix(serve): boot-readiness deadline releases PGLite lock on wedged boot (#3335) ... * fix(serve): boot-readiness deadline releases PGLite lock on wedged boot (#3273) A serve process that wedges mid-boot (e.g. a boot step blocked on an unreachable upstream) held the PGLite write lock indefinitely — the post-#2348 lock discipline never steals from a live holder, so every CLI consumer timed out until the serve PID was manually killed. runServe (stdio path) now arms a boot-readiness deadline around startMcpServer: if the transport hasn't connected within GBRAIN_SERVE_BOOT_TIMEOUT_SECONDS (default 60, 0 disables), it logs the condition, awaits engine.disconnect() (raced against the existing 5s cleanup deadline so a wedged WASM close can't trap it either), and exits non-zero so supervisors restart with backoff. A completed boot clears the timer; the HTTP path is untouched (own lifecycle). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(e2e): 60s hook timeout for jsonb-parity setup/teardown The #2339 parity guard's beforeAll runs setupDB (full migration chain) under bun's default 5s hook timeout, which flaked on a slow CI runner (setupDB hit 5001ms). Other e2e suites already pass explicit hook timeouts; bring this file in line. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-24 12:11:10 -07:00
5 people	e1919fab9f	reland: fix(embed): stamp gateway-resolved model in content_chunks.model, not compiled default (#2846) (#3343) ... * fix(embed): stamp gateway-resolved model in content_chunks.model, not compiled default (#2846) upsertChunks fell back to the compile-time DEFAULT_EMBEDDING_MODEL ('zeroentropyai:zembed-1') when a ChunkInput carried no explicit `model`. The embed pipeline (src/commands/embed.ts) builds ChunkInputs without a `model` field, so rows whose vectors were produced by the config-resolved model (e.g. openai:text-embedding-3-large) were mislabeled with the hardcoded default — corrupting the provenance that signature-drift staleness and dimension-migration logic depend on. Both engines now resolve the gateway's runtime embedding model once per upsert and use it as the fallback, mirroring the existing resolve-then- default pattern used for schema sizing. Regression test added (pglite); verified via negative control that it fails against the old fallback. This is a write-path change (upsertChunks), not a search-path change, so retrieval eval replay is not applicable. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * test: Lane A.7 pins gateway-resolved chunk model, not compiled default #2846 changed upsertChunks' fallback from DEFAULT_EMBEDDING_MODEL to the gateway-resolved runtime model. Lane A.7 still pinned the old fallback, and the test preload (test/helpers/legacy-embedding-preload.ts) pins the gateway to openai:text-embedding-3-large for every test process — so the original #2846 landing failed this test deterministically and got batch- reverted. The test now asserts the resolved model (the intended #2846 semantics) while keeping the CDX2-4 bare-literal regression guard. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: SailorJoe6 <SailorJoe6@Gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Garry Tan <garrytan@gmail.com>	2026-07-23 18:48:22 -07:00
4 people	40d9b83d5c	fix(cycle): wire drift detection into the dream cycle — report-only v1 (#2653) (#3317) ... dream.drift.enabled has gated an unwired scaffold since v0.28: runPhaseDrift had zero call sites, the resolved model + BudgetMeter were discarded (void modelId; void meter), and no operator-readable output existed. - Wire 'drift' as a CyclePhase (default OFF via dream.drift.enabled), ordered after the calibration trio and before embed so the report page gets embedded same-cycle. PHASE_SCOPE=global, cycle-lock coordinated, --once (--phase drift --once) bypasses the gate for one run. - Implement the LLM judge: soft-band candidates (weight 0.3-0.85, active, unresolved, fresh timeline evidence) are judged against their page's recent timeline entries via gateway chat; BudgetMeter-gated (dream.drift.budget, default $1), capped by dream.drift.max_per_cycle (default 20). Judge model resolves models.drift -> reasoning tier -> sonnet fallback (unchanged from scaffold). - Report-only v1: judged candidates land on a reports/drift-<date> page. dream.drift.auto_update mutates NOTHING; the flag state is recorded in the report for operators. Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-23 17:28:47 -07:00
Garry Tan	97df1e78b7	Revert "fix(init): explicit --embedding-model overrides persisted --no-embedding sentinel (#3138)" ... This reverts commit df22c81996.	2026-07-23 15:26:33 -07:00
4 people	df22c81996	fix(init): explicit --embedding-model overrides persisted --no-embedding sentinel (#3138) ... * fix(init): explicit --embedding-model overrides the persisted --no-embedding sentinel (#2301) Pre-fix, once ~/.gbrain/config.json carried embedding_disabled: true (the --no-embedding deferred-setup sentinel), every re-init silently re-deferred embedding: resolveAIOptions honored the sentinel BEFORE the explicit --embedding-model flag and never cleared noEmbedding, and the persistence merge carried the sentinel forward via ...existingFile. Both recovery paths were dead ends — `gbrain config set embedding_model` is hard-refused (schema-sizing field), and re-init hit the sentinel. Fix: - resolveAIOptions: an explicit --embedding-model / --model flag clears the sentinel-derived noEmbedding (explicit --no-embedding on the same invocation still wins — that branch runs after). - initPGLite + initPostgres persistence: a resolved (model, dims) tuple drops the stale embedding_disabled key instead of inheriting it. - assertEmbeddingEnabled message no longer recommends the refused `gbrain config set embedding_model` command; the working re-init recipe leads. Test: test/e2e/init-reinit-after-deferred.test.ts — deferred init then re-init with an explicit model recovers (sentinel gone, model persisted); bare re-init still honors the sentinel. Fixes #2301 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * review: scrub remaining hard-refused `config set embedding_model` advice from init deferred-setup messages The PR fixed the recovery recipe in assertEmbeddingEnabled but the deferred-setup lines in initPGLite/initPostgres and the fail-loud defer hint still pointed users at the Lane C.2 hard-refused command. Point all three at the working re-init recipe instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-23 15:03:46 -07:00
MasaandGitHub	2b00b7abeb	fix(migrate): drop invalid CONCURRENTLY-build remnants without a DO block (#3191) ... * fix(migrate): drop invalid CONCURRENTLY-build remnants without a DO block Migration v66 (embed_stale_partial_index) pre-drops an invalid index left over from a previously interrupted CREATE INDEX CONCURRENTLY using DO $$ BEGIN ... EXECUTE 'DROP INDEX CONCURRENTLY IF EXISTS ...'; END $$. Postgres rejects CONCURRENTLY from any function/EXECUTE context, so the guard's EXISTS check passes but the EXECUTE inside it always throws "DROP INDEX CONCURRENTLY cannot be executed from a function" -- the migration only fails on brains carrying an invalid-index leftover. Add dropInvalidConcurrentIndex(): the validity probe runs as a plain application-level SELECT, and the DROP runs as its own top-level runMigration call instead of inside a DO block. Fixes #1178. * fix(migrate): address codex review — schema-safe index resolution + OID-based no-op assertion - dropInvalidConcurrentIndex(): resolve indexName via to_regclass() (search_path resolution, same as the unqualified DROP that follows) instead of matching pg_class.relname bare, which could hit a same-named index in a different schema on a non-default search_path. - e2e test: the no-op re-run case now compares index OID before/after, not just validity -- validity alone wouldn't catch a spurious drop+recreate.	2026-07-23 13:53:15 -07:00
3 people	18513c65be	fix(budget): make paid MCP spend atomic and fail closed (#3203) ... Co-authored-by: caterpillarC15 <caterpillarC15@users.noreply.github.com>	2026-07-23 13:53:11 -07:00
4 people	38f446bb6f	fix(test): isolate $HOME in mechanical.test.ts so E2E suite stops clobbering user config (#434) ... mechanical.test.ts shells out to `gbrain init --non-interactive`, `gbrain import`, and similar commands via Bun.spawnSync. The four `cliEnv()` helpers in this file forward `process.env` unchanged, so `gbrain init` ends up calling saveConfig() against the developer's real $HOME/.gbrain/config.json, overwriting their production database_url with the test container's URL on every `bun run test:e2e` invocation. Sibling test/e2e/migration-flow.test.ts already solved this with a module-level temp HOME and an afterAll restore. Mirror that pattern in mechanical.test.ts. Verified by md5'ing ~/.gbrain/config.json before and after running the Setup Journey, Init Edge Cases, Schema Idempotency, RLS Verification, Doctor Command, and Parallel Import describe blocks — config hash is identical pre and post (26 passing tests, 0 failures, 0 mutations to the user's real config). Co-authored-by: Seth Armbrust <setharmbrust@seth.local> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-07-23 13:24:40 -07:00
6 people	04e6b3af14	fix(cycle,lint): PGLite inline synth subagent drain + lint --exclude (takeover of #2699, #2649) (#3162) ... * fix(cycle): drain PGLite synth subagents inline (takeover of #2699) PGLite holds an exclusive file lock on its embedded data-dir, so no separate Minions worker can serve the subagent children the synthesize phase enqueues — they sat in 'waiting' until waitForCompletion timed out. Drain a private per-run child queue inline (claim → run → complete/fail, plus the promote/stall/timeout housekeeping a worker would perform). No-op on Postgres, where children stay on the shared 'default' queue. Rebased onto the reworked synthesize (config.subagentTimeoutMs, #1586 source scoping): the inline job context now carries deadlineAtMs from the claim-time timeout_at stamp, and opts.yieldDuringPhase is ticked on a 60s keepalive while each child runs so the 5-min cycle lock TTL refreshes across long (up to 30-min) children. Co-authored-by: TheRealMrSystem <TheRealMrSystem@users.noreply.github.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(lint): --exclude flag for mixed-content repos (takeover of #2649) Adds --exclude=a,b (and LintOpts.exclude) so mixed-content repos can skip software trees and repo metafiles by basename when collecting pages. The only built-in default is node_modules — vendored dependency trees are never knowledge pages; dot/underscore entries were already skipped by the walk. Diverges from #2649 deliberately: the original hardcoded an opinionated default list (README.md, CHANGELOG.md, CLAUDE.md, test/ dirs at any depth, plus fork-specific filenames), which silently changed lint counts for every existing repo. Those are repo policy — pass --exclude. Co-authored-by: ryangu00 <ryangu00@users.noreply.github.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cycle): enforce per-job timeout_ms in the PGLite inline subagent drain The inline drain claimed children with deadlineAtMs derived from timeout_at but never armed the worker's timeout timer — and the handleTimeouts sweep only runs between jobs, so nothing could stop a child that blew past its 30-min timeout_ms. A hung LLM call wedged the drain loop (and the whole cycle) indefinitely, with the 60s keepalive refreshing the cycle lock forever. Worker.ts parity: arm a timer from the claim-time timeout_at stamp, abort ctx.signal on fire, and dead-letter (never delayed-retry) timed-out children, mirroring handleTimeouts' stall→retry / timeout→dead split. Regression test: a child with timeout_ms=100 whose handler only ends on ctx.signal abort is dead-lettered with 'timeout exceeded'; pre-fix the test hangs to its 30s timeout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: TheRealMrSystem <TheRealMrSystem@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: ryangu00 <ryangu00@users.noreply.github.com>	2026-07-23 13:09:42 -07:00
SpinsirrandGitHub	d574e843a8	fix(mcp): source-scope hardening for remote callers (#2881) ... Three fixes in the same leak class (a remote caller reading or writing outside its granted sources), for multi-source / multi-tenant brains: 1. dispatchToolCall now refuses remote calls that arrive without a resolved sourceId (missing_source_scope error envelope) instead of silently falling back to the shared 'default' source. Every shipped transport already passes sourceId explicitly (serve-http from the OAuth client row, http-transport from the legacy token grant, stdio from GBRAIN_SOURCE); reaching the fallback remotely always meant a programmatic caller skipped scope resolution — the bug class behind #1924 / #1371. Trusted local callers (remote === false) keep the historical fallback. Direct-dispatch tests updated to carry an explicit sourceId, matching the real transport contract. 2. log_ingest threads ctx.sourceId (same pattern as get_chunks / get_page), so ingest events are attributed to the caller's source instead of piling into 'default'. Engines already accept entry.source_id (v0.31.2). 3. get_ingest_log is source-scoped for remote callers via the linkReadScopeOpts collapse rule (scalar grant → [scalar]; federated grant → granted array); it previously returned the whole brain's ingest log to any read-scoped remote client, and ingest summaries can carry another source's private context. Trusted local callers keep the whole-brain view. Tests: dispatch guard (refuse remote-without-source, keep local fallback, guard ordering after op lookup) and end-to-end ingest-log attribution + scoping over the real dispatch path, on PGLite.	2026-07-23 12:02:56 -07:00
3 people	d21f34e96d	fix(search): project email citation metadata (#2873) ... Co-authored-by: Amit Agarwal <5302320+amtagrwl@users.noreply.github.com>	2026-07-23 12:02:46 -07:00
Garry Tan	418357332f	Revert "fix(embed): stamp gateway-resolved model in content_chunks.model, not compiled default (#2846)" ... This reverts commit 5aa4795c04.	2026-07-23 12:02:36 -07:00
3 people	5aa4795c04	fix(embed): stamp gateway-resolved model in content_chunks.model, not compiled default (#2846) ... upsertChunks fell back to the compile-time DEFAULT_EMBEDDING_MODEL ('zeroentropyai:zembed-1') when a ChunkInput carried no explicit `model`. The embed pipeline (src/commands/embed.ts) builds ChunkInputs without a `model` field, so rows whose vectors were produced by the config-resolved model (e.g. openai:text-embedding-3-large) were mislabeled with the hardcoded default — corrupting the provenance that signature-drift staleness and dimension-migration logic depend on. Both engines now resolve the gateway's runtime embedding model once per upsert and use it as the fallback, mirroring the existing resolve-then- default pattern used for schema sizing. Regression test added (pglite); verified via negative control that it fails against the old fallback. This is a write-path change (upsertChunks), not a search-path change, so retrieval eval replay is not applicable. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-23 11:37:06 -07:00
Ziyang GuoandGitHub	7bbd087cb7	fix(pages): restore soft-deleted rows on putPage (#2779)	2026-07-23 11:25:00 -07:00
Aurora CapitalandGitHub	2c96787867	test(e2e): harden suite — kill flakes, no-op assertions, cross-test coupling (#1704) ... * test(e2e): drop flaky wall-clock bounds in minions-resilience The runaway-dead-letter and cascade-kill tests asserted tight real-clock upper bounds (<2000ms, <3000ms) on top of already-complete terminal-state checks. Those bounds carry no correctness signal — the dead/cancelled status and abortedChildren==10 assertions fully prove behavior — and flake on loaded CI runners where the stall/timeout sweep cadence varies. Removed both bounds, kept the diagnostic values, de-promised the test titles. * test(e2e): kill order-dependence + no-op assertions in mechanical - traverse_graph: self-contained (re-adds its own idempotent link) and asserts the linked company is reachable, instead of depending on a prior it() and only checking array shape. - file_list-without-slug: seeds its own >100 rows instead of relying on the previous test's 150 surviving in the DB; asserts the cap is exercised. - precision@5: add a loose floor (every known-item query surfaces >=1 truth doc in top-5) so a 0% retrieval regression no longer passes silently. - get_health: assert value bounds (page_count==16, embed_coverage 0..1) not just typeof; get_chunks: assert non-empty text, numeric non-decreasing chunk_index, and that the page name appears, instead of toBeTruthy on chunks[0]. * test(e2e): strengthen graph/search quality assertions + close coverage gaps graph-quality: truncate+reseed 'config' in truncateAll (kills config leak where a setConfig test throwing before its finally bleeds into later tests); replace toBeGreaterThan(0) link/timeline floors with fixture-derived minimums; assert exact attendee slugs are 'attended' instead of a vacuous .every; pin autoLinks.created to the provable 2 (Alice+Acme); add direction out/both + depth:2 multi-hop and a cycle-safety (A->B->A terminates) test. search-quality: fix the vacuous detail=low vector test; assert pedro returns >=2 chunks; assert detail=high includes the timeline chunk; add empty-query and zero-vector no-throw edge tests. * test(e2e): self-contain multi-source sync test + assert ledger cascade Break the sequential dependency where 'performSync no sourceId' relied on a prior test writing sync.repo_path — it now sets its own config. Add the missing file_migration_ledger COUNT(*)==0 cascade assertion. Tighten the source_id default check from toContain('default') to exact "'default'::text". * test(e2e): make migration-flow HOME/PATH swap throw-safe The suite repoints process.env.HOME/PATH to a temp dir and only restored them in afterAll, so a mid-test throw left HOME dead for the rest of the bun process and silently broke sibling suites. Wrap each test body in try/finally restore + a defensive restore at the top of beforeEach. * test(e2e): loud-skip jsonb-roundtrip + doctor-progress Both skipped silently with no DATABASE_URL, giving zero signal the regression guard never ran. Add the console.log skip line matching the sibling e2e files. * test(e2e): robust check-update contract + find_orphans tool coverage upgrade: the 'no-releases' test hard-asserted update_available===false, which flips to failing the moment the repo has a real release. Assert the JSON contract shape (boolean update_available, current_version===VERSION, typed optional fields) instead. mcp: add find_orphans to the asserted generated tool names.	2026-07-22 18:46:44 -07:00
5 people	e78ad9ff9e	fix(salience): exclude briefings/* from their own Brain Pulse (TIM-37) (#1202) ... The cron daily briefing writes 90_Briefings/<date>.md, which gets re-ingested on the next sync and then dominates tomorrow's getRecentSalience output as pure self-reference (observed: top result score 0.9956, everyone else clustered at 0.587). Filter `p.slug LIKE 'briefings/%'` out of getRecentSalience in both the PG and PGLite engines. Suppressed by default; callers can still opt in by passing `slugPrefix: 'briefings/'` (or `--kind briefings/` from the CLI). search and list_pages are unaffected. Co-authored-by: CTO <cto@timelycare.local> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>	2026-07-22 14:47:11 -07:00
5 people	62e009d192	fix(skillopt): emit proposed.md in no-mutate mode (#2635) (#3182) ... Takeover of #2719 (fork head; rebased onto origin/master). - writeProposed now writes both best.md (current-best pointer) and proposed.md (stable human-review artifact); returns the proposal path. - Orchestrator reports the real proposed.md path for accepted --no-mutate runs. - Tutorial updated; llms bundles regenerated (no content drift — tutorial is not inlined in the bundle). Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Ziyang Guo <121015044+RerankerGuo@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-22 00:23:10 -07:00
5 people	1fabbb9849	fix(links): resolve path-qualified wikilinks outside DIR_PATTERN in the DB/put_page path (#2866) ... The generic wikilink pass (issue #972) forwarded the raw literal to resolveBasenameMatches, whose index is keyed by final path segments — so [[notes/struktura]] (any dir outside DIR_PATTERN) silently produced zero edges from `extract links --source db` and put_page auto-link, while the FS extractor resolves the identical content (resolveSlugAll strips the dirname before its basename lookup). Query by the literal's final segment, then keep only matches whose slug ends with the written path — [[notes/struktura]] can resolve to vault/notes/struktura but never attach to wiki/struktura. Bare literals are untouched. Flag-gated by link_resolution.global_basename as before. Co-authored-by: YMYD <53603073+OJ-OnJourney@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>	2026-07-21 18:45:23 -07:00
5 people	d61808d806	v0.42.64.0 fix: harden confidential OAuth token revocation (takeover of #3017) (#3032) ... * fix(oauth): validate confidential revoke secrets * fix(oauth): harden confidential token revocation * chore: bump version and changelog (v0.42.64.0) Co-Authored-By: OpenAI Codex <noreply@openai.com> --------- Co-authored-by: Robin <rayme@boltdsolutions.com> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: Garry Tan <garrytan@gmail.com>	2026-07-21 12:44:46 -07:00
5 people	60125ee626	feat(dream): --once for one-shot phase runs without toggling config gates (takeover of #2983) (#3031) ... * feat(dream): --once for one-shot phase runs without toggling config gates Fixes the "toggle enabled true, run, toggle back to false" workaround that gbrain doctor's extract_atoms_backlog message implicitly recommends and that #2860's reporter had to script around: with an external orchestrator running `gbrain dream --phase patterns` on a cadence outside the autopilot, the only way to run patterns once was `config set dream.patterns.enabled true` -> run -> `config set ... false`. A crash between steps left the flag stuck true, and the autopilot (which polls the same flag) re-enqueued patterns every cycle -- 119 LLM jobs / ~$400 over 24h before it was caught. Root cause: `--phase X` only controls which phase FUNCTION cycle.ts calls; it does not bypass that phase's own `dream.<phase>.enabled` / `cycle.<phase>.enabled` config read. Each gated phase (patterns, synthesize, conversation_facts_backfill, enrich_thin, skillopt) reads its enabled flag internally and skips regardless of how the phase was selected -- confirmed by reading each phase module, not assumed. extract_atoms/synthesize_concepts are a DIFFERENT mechanism entirely (pack-declaration via packDeclaresPhase, not a config .enabled read) and already have a working one-shot escape hatch: `--drain`. The existing doctor message for extract_atoms already says `--phase extract_atoms --drain --window 120`, so no doctor text needed updating there -- verified by reading src/commands/doctor.ts directly rather than assuming the paraphrase in the issue was literal. Design: `gbrain dream --phase <name> --once`. Requires an explicit --phase (bare --once is a usage error, exit 2) so it can never force-enable every disabled phase at once in a full/default cycle -- that would recreate the same unbounded-spend risk the flag exists to prevent. Threaded through CycleOpts as `onceForPhase?: CyclePhase` (the literal phase name, not a boolean) so the bypass can never leak to a phase other than the one named, even if a future programmatic caller passes a wider `phases` array than the CLI does. Never reads or writes config -- the phase still evaluates its .enabled gate every call; --once only overrides the boolean OUTCOME for that one invocation, mirroring the existing --unsafe-bypass-dream-guard / --input precedents (stderr warning at the bypass point, no new config-touching code path). Rejected alternatives (documented per task instructions): - Making explicit --phase X always bypass .enabled: breaking change for existing crons that rely on the disabled flag as a cheap no-op; an upgrade would silently start running LLM/write phases. - A new subcommand: adds a whole dispatch/help/arg surface that internally routes through the same override anyway. - Extending --once to also bypass packDeclaresPhase for extract_atoms/synthesize_concepts: conflates two different gating mechanisms (config toggle vs. pack membership) under one flag; extract_atoms already has --drain, which is purpose-built for its batched/windowed execution model. Design was cross-validated by an independent second-model review (external design consultation) before implementation; its recommendation to also update the extract_atoms doctor message to `--once` was NOT adopted because that phase has no .enabled gate to bypass -- doing so would be a documented no-op, contradicted by reading src/commands/doctor.ts:3264 directly. Tests: 9 new (structural CLI-flag wiring in dream-cli-flags.test.ts; a real PGLite E2E test in dream-patterns-pglite.test.ts proving the bypass fires AND that dream.patterns.enabled is never written; 4 runCycle-level tests in cycle.serial.test.ts proving onceForPhase does not leak across phases). Verified 6 of 9 fail against the pre-fix source (via git stash of source-only changes) to confirm they're meaningful regressions, not tautologies. Closes #2860 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(dream): --help short-circuits before --once usage validation Codex review finding (P2): `gbrain dream --help --once` (no --phase) called process.exit(2) from the new --once usage-error check inside parseArgs before runDream's documented IRON RULE ("--help short-circuits BEFORE any engine-bearing work") ever got a chance to run -- parseArgs computes ALL its validations unconditionally before runDream checks opts.help. Repo precedent for this ordering already exists as a pinned regression test (test/dream.test.ts's "--help --source whatever prints help and exits 0"). Fix: compute wantsHelp once in parseArgs and exempt the --once validation when it's set, mirroring that precedent. Added the same class of pinned tests here: bare `--once` still exits 2 with the usage hint, `--help --once` prints help and exits 0, and a real --phase patterns --once run against a PGLite engine proves the bypass actually fires (falls through to insufficient_evidence instead of disabled) without writing dream.patterns.enabled. Also fixed the structural test in dream-cli-flags.test.ts that asserted the exact pre-fix guard-condition source text. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(dream): --once must require an EXPLICIT --phase, not a derived one Codex review finding (P3): the --once validation checked the derived `phase` value, but `phase` gets defaulted implicitly by --input (implies --phase synthesize) and --drain (implies --phase extract_atoms) BEFORE that check ran. So `gbrain dream --input <f> --once` and `gbrain dream --drain --once` both slipped past the "explicit --phase required" contract silently -- and --once became a true no-op in both cases: --drain returns from runDream before onceForPhase is ever read (the drain path doesn't call runCycle at all), and --input already bypasses the synthesize enabled-gate on its own via the existing opts.inputFile check, so onceForPhase would never even be consulted. Fix: capture `phaseWasExplicit = phaseIdx !== -1` at the very top of parseArgs, before the --input/--drain defaulting blocks run, and validate --once against that instead of the derived `phase`. Updated the usage-error message and --help text to say "an explicit --phase" so a user hitting this understands why `--input ... --once` doesn't count. Tests: 2 new pins in test/dream.test.ts exercising runDream directly (--input <file> --once exits 2; --drain --once exits 2), plus a structural test in dream-cli-flags.test.ts pinning that phaseWasExplicit is captured before both implicit-defaulting blocks. Updated the two existing structural/behavioral tests whose literal guard-condition / error-message assertions changed shape. Verified: dream-cli-flags.test.ts 27/27, dream.test.ts 31/31, typecheck clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: masashiono0611 <masashi.ono.0611@gmail.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Garry Tan <garrytan@gmail.com>	2026-07-21 12:29:35 -07:00
Anton SenkovskiyandGitHub	a93fcf504f	fix(chronicle): bound last-seen to <= asof/today so future events do not read as "seen today" (#2993) ... getLastSeen had no upper date bound, so a future-dated chronicle event (a scheduled calendar-event, a planned milestone) became the entity's "last seen" date; finalizeLastSeen's Math.max(0, ...) then clamped the negative delta, reporting days_ago: 0 -- the entity reads as seen-today. Recording future events is intended (eligibility ELIGIBLE_TYPES includes calendar-event); the reader just needs to stop counting them as "seen". Bound the query to te.date <= COALESCE(asof, current_date) in both engines, mirroring getOnThisDay's existing te.date < target bound. asof now reaches the WHERE clause (previously it only reached finalizeLastSeen), so as-of time-travel is honored for the date filter too. Regression test added: an entity with past events plus a future event -> last seen returns the most recent PAST event, not the future one; and as-of after the future date lets it through. Fails before, passes after.	2026-07-21 00:04:41 -07:00
3 people	11eebc3605	fix(conversation): extract iMessage facts with real timestamps (#2756) (#2958) ... Co-authored-by: Sameer Bopardikar <203024074+sameerbopardikar@users.noreply.github.com>	2026-07-20 16:57:53 -07:00
5 people	9ed53e4e1c	fix(code-edges): per-row $n::text::jsonb binds in addCodeEdges — Bun SQL mis-encodes jsonb[] arrays (#2968) (#3020) ... Bun SQL double-encodes ::jsonb[] array binds on the Postgres engine: every edge_metadata element landed as a jsonb string scalar instead of an object, so the resolver's `edge_metadata || jsonb_build_object(...)` UPDATE produced a jsonb array and resolved_chunk_id was never readable — code_callers / code_callees / code_blast returned nothing on Postgres-engine brains while the resolver logged edges_resolved > 0. PGLite was unaffected (per-row placeholders already). Rewrites both inserts (code_edges_chunk, code_edges_symbol) to per-row $n::text::jsonb placeholders via sql.unsafe — the same shape executeRawJsonb and the PGLite engine use. Adds the DATABASE_URL-gated Postgres regression test this class requires (PGLite cannot reproduce it): asserts jsonb_typeof(edge_metadata) = 'object' for resolved + unresolved inserts and that the resolver-style || UPDATE keeps object shape. Verified the test fails 3/3 against the pre-fix code and passes with the fix. Takeover of #2968 by @zsimovanforgeops with the missing regression test added. Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Forge (Ron) <forge@zsimovan.dev> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-20 14:51:47 -07:00
3 people	23e0541d9b	fix(cycle): budget the patterns subagent from remaining job time — one phase's fixed 35-min worst case defeats any interval-derived cycle budget (#2781) (#2959) ... * fix(cycle): budget the patterns subagent from remaining job time, not a fixed constant (#2781) The autopilot-cycle job gets an interval-derived timeout stamped at submit, but the patterns phase submits its subagent with a fixed 30-min job timeout and waits up to 35 min — one phase's worst case exceeds ANY interval-derived budget <= 35 min, so the parent job dead-letters mid-patterns and the tail phases (consolidate -> schema-suggest) starve for days (#2781, the deeper half left open by the #2852 dispatch-floor fix). - MinionJobContext.deadlineAtMs: absolute deadline from the claim-time timeout_at stamp (the DB ground truth handleTimeouts() sweeps against; re-stamped on every claim so retries get a fresh budget). Null when the job has no per-job timeout. - worker: the per-job abort timer now derives its delay from timeout_at when present, so the in-process timer, the DB sweeper, and the handler-visible deadline agree on ONE absolute instant. - autopilot-cycle + autopilot-global-maintenance handlers thread deadlineAtMs into runCycle; CycleOpts carries it to the patterns phase. - patterns: clampSubagentBudgets() derives BOTH the child job timeout and the wait timeout from the same child deadline (parent deadline minus a 60s stop-margin reserve — enough for the wait poll + force-evict grace + cleanup, deliberately NOT a promise that tail phases complete). Under a 2-min minimum the phase skips honestly (insufficient_cycle_budget) instead of submitting a guaranteed-kill LLM call; the next cycle retries with a fresh budget. - Direct callers (gbrain dream) pass no deadline and keep the configured timeouts unchanged. Follow-up (separate PR): synthesize has the same shape plus sequential per-child waits that accumulate N x subagent_wait_timeout_ms past any parent budget; it needs per-wait remaining-time recomputation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RaNDjecPPvhC7LRqkxkhGF * fix(cycle): address review — cancel timed-out patterns child; thread deadline through phase-wrapper handlers - P1: the child's timeout_ms clock starts at ITS claim, so a queued child could outlive the parent deadline the wait was clamped to. On wait timeout, cancelJob strips it (waiting -> cancelled; active -> lock stripped, worker abort fires next renew tick). - P2: makePhaseHandler (standalone patterns/synthesize/... minion jobs) now threads job.deadlineAtMs into runCycle like the autopilot handlers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RaNDjecPPvhC7LRqkxkhGF --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-20 13:56:00 -07:00
4 people	324c355318	test(e2e): production guard — setupDB refuses non-test databases (#2957) ... setupDB() TRUNCATEs every data table on whatever DATABASE_URL points at, and run-e2e.sh deliberately preserves an exported DATABASE_URL — one stray environment variable away from wiping a production brain, with no guard of any kind (found during an independent review, 2026-07-18). assertSafeE2eDatabaseUrl (pure, unit-tested) now runs before any connection: allowed when the database name carries "test" as a word segment (the gbrain_test convention used by CI and .env.testing.example), or when GBRAIN_E2E_ALLOW_DB names the exact database intentionally. Refusal is loud and actionable. 7 unit tests, no DB required. Co-authored-by: Aleksei Razsadin <razsoc.01@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-20 12:05:04 -07:00
4 people	184b6cb8a1	fix search: title candidate arm + gated OR fallback for lexical recall (#2956) ... Pages were unreachable by their own exact titles: FTS indexed only chunk body text while the title-weighted pages.search_vector (GIN-indexed since its introduction) was never queried by any search path, and websearch_to_tsquery AND-at-chunk-grain semantics meant one non-matching token zeroed keyword recall with no fallback — long or acronym-bearing titles (e.g. "IAWG ... AAR-LL deck") fell through to the vector arm alone and missed. - searchTitles (both engines): page-grain candidate arm over pages.search_vector (title 'A' + compiled_truth 'B' + timeline 'C'), ts_rank_cd ranked, representative-chunk LATERAL join, full filter parity with searchKeyword (visibility, soft-delete, source grants, hard-excludes, dates, types); fused as a weighted RRF list at the keyword arm's intent-effective k on all three hybrid return paths; fail-open with warnOncePerProcess. No schema changes — the index already existed, dark. - AND->OR one-retry fallback for the keyword arm, gated behind SearchOpts.orFallback (only hybridSearch opts in; countMentions, link resolution, eval, and keyword-only MCP callers keep the strict-AND contract). Refused for queries carrying websearch operators (negation, quoted phrases). searchTitles carries its own page-grain fallback. - Lexical arms parallelized (Promise.all) on the main path. Verified: typecheck clean; 18 hermetic PGLite tests + 2 engine-parity e2e cases (CI Postgres); consumer regression enrichment 18/0 + link-extraction 127/0; independent live QA on a 10,664-page brain — exact-title target miss -> rank 1 (exact_title_match), controls held, negation/quoted guards proven, strict-consumer contract pinned. Diagnosed from a 3-lane read-only diagnostic; adversarial review round closed findings on fallback scope, Postgres test coverage, and operator handling before this commit. Co-authored-by: Aleksei Razsadin <razsoc.01@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-20 11:57:31 -07:00
4 people	42375bded5	fix(sync): stop the sync data-loss family — ops/ prune, DB-only write-through, full-sync gate drift (#2404, #2426, #2607) (#2938) ... Three verified-open defects, one family: sync silently destroying or diverging on content it should preserve. #2404 (P0) — 'ops' was hardcoded in PRUNE_DIR_NAMES (a v0.2.0-era carve-out), so any path with an ops segment was 'pruned-dir': committed ops/*.md never imported, and modified ops/* files hit the unsyncableModified delete loop (whose #1433 guard only spared 'metafile'), silently deleting put-created pages like the bundled daily-task-manager's canonical ops/tasks on every sync. Fix: remove 'ops' from the prune list (ordinary user content; the vendor/generated entries stay), and harden the delete loop to also skip 'pruned-dir' — a page under a pruned dir can only exist via a deliberate put_page. #2426 (P0) — write-through content stayed DB-only and was deleted by sync --full. All three compounding bugs fixed: 1. writePageThrough now best-effort commits the artifact (path-limited git commit) on durability-hardened repos, so the post-commit hook can push it; result carries committed?: boolean. 2. scripts/brain-commit-push.sh stages+commits BEFORE any pull — the old fetch+pull-rebase-first order aborted on any dirty tree, so the helper could never commit a MODIFIED page; brain_push's rebase-on-reject already handles an advanced remote. 3. The full-sync delete-reconcile partitions stale pages by git history (listEverCommittedPaths): never-committed source_paths are DB-only write-through — pages are KEPT and re-exported to the working tree instead of soft-deleted. Builds on the #2828 mass-delete valve (covers the below-valve cases). #2607 — the sync --full git ls-files fast path bypassed pruneDir, so a full pass imported (and resurrected soft-deleted) pages under dot-dirs and vendored trees that incremental sync excludes. Fix: isCollectibleForWalker applies the same segment-level pruneDir gate as classifySync, so full and incremental enumeration agree. One regression test per defect (all verified failing against master src): test/sync-ops-pages.serial.test.ts, test/write-through-commit.serial.test.ts, the #2426 helper-order test in test/brain-durability-hook.serial.test.ts, test/sync-reconcile-db-only.serial.test.ts, test/import-git-fastpath-prune.test.ts. Fixes #2404 Fixes #2426 Fixes #2607 Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-17 14:41:25 -07:00
5 people	a0ef951586	fix(dream): gate patterns phase on gateway provider reachability, not ANTHROPIC_API_KEY (takeover of #2279) (#2936) ... Absorbs PR #2279's intent (drop the hardcoded ANTHROPIC_API_KEY gate) with the end-state the gateway world actually wants: the patterns phase now probes the RESOLVED patterns model through probeChatModel(normalizeModelId) — the same semantics as think/index.ts and synthesize's makeJudgeClient. Fixes two misclassifications of the old env gate: - Non-Anthropic stacks (litellm, deepseek, openrouter, ...) were skipped as "no upstream" even though the subagent routes them through the gateway (agent.use_gateway_loop). They now pass the gate; their auth is checked lazily at dispatch and surfaces in the job outcome. - Anthropic keys set via `gbrain config set anthropic_api_key` (stdio MCP launches without shell env) were treated as missing. hasAnthropicKey inside probeChatModel reads both sources. Skip reason renames no_api_key → no_provider (carrying the probe's detail). Both pinning tests updated: the structural test asserts the probe wiring; the PGLite E2E swaps its env-only helper for the shared hermetic withoutAnthropicKey (env + config file) so it can't flip to a live LLM call on a dev machine with a config-file key. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: brettdavies <brettdavies@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-17 14:18:56 -07:00
5 people	26d2f8abfc	fix(calibration,takes,cli): calibration CLI routing, source-scoped takes reads, BigInt-safe outputs (takeover of #2452) (#2892) ... Rebase-port of #2452 (spinsirr:fix/calibration-profile-scope-and-cli) onto current master after tonight's merges made the fork branch conflict. - cli: add 'calibration' to CLI_ONLY so dispatch reaches its existing handler instead of falling through to "Unknown command" (#2035); honor --source / GBRAIN_SOURCE in the calibration CLI. - takes: route takes_list / takes_search / takes_scorecard / takes_calibration through sourceScopeOpts(ctx) (federated array > scalar > nothing) and scope engine reads via the take's page.source_id — JOIN filter for list/search, EXISTS for scorecard/curve — on both engines (#2200-class). - bigint: shared takeHitRowToHit coercion in searchTakes / searchTakesVector (both engines) + bigintToStringReplacer on the cli.ts output normalizer and the `gbrain call` exit, so int8/BIGSERIAL columns no longer crash JSON.stringify (#2450); calibration profile id BIGSERIAL → string. - calibration: default model ids route through TIER_DEFAULTS (provider-prefixed) instead of bare model strings; admin calibration chart endpoints fixed (takes has no page_slug column; month-precision since_date; Date generated_at; bigint id in drill-down). The think/gather source-scope slice of the original PR was dropped: it already landed on master via #2739. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: spinsirr <ID+spinsirr@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-16 21:20:12 -07:00
+515 people	86962b242b	fix(gateway): consolidate tool-loop resume + provider fixes (fix-wave A) (#2820) ... Collector branch superseding the gateway tool-loop duplicate cluster and adjacent provider fixes. Re-implemented from the best of each PR (deduped by content, not file-overlap); every fix carries test coverage. Gateway tool-loop resume (supersedes #1934 #2062 #2065 #2112 #2274 #2487 #2336 #2257 #2499 #2491, test #2063): - toolLoop now persists the tool-result user turn per round (onToolResultTurn), so a resumed subagent job reloads a balanced transcript instead of dangling assistant tool-calls that non-Anthropic providers reject with AI_MissingToolResultsError. - runSubagentViaGateway reconciles an already-corrupted transcript on resume: it heals every dangling assistant tool-call turn (not just the tail) from the settled subagent_tool_executions rows, re-dispatching idempotent-pending tools and throwing on non-idempotent, mirroring the legacy Anthropic path. Terminal early-return for a transcript that already reached end_turn. - repairToolPairing() is a last-resort normalization at the chat() boundary (from #2336): back-fills error stubs for any assistant tool-call still unanswered (partial turns, provider-duplicated/dropped IDs on local models, length-truncated batches). No-op on balanced input. - toModelMessages is Date-safe (Postgres timestamptz -> ISO via a JSON round trip at the SDK boundary, never a ::jsonb cast; degrades bigint/circular to a string instead of throwing) and drops non-string text blocks reasoning models emit that AI SDK v6 rejects (#2488). Adjacent provider fixes: - Model-aware default max output tokens: thinking-by-default Claude 5 models get headroom (gateway 32000, think 16000) while everything else stays 4096/4000, so DeepSeek/OpenAI subagents don't exceed provider caps (#2614 #2806). - DeepSeek: promote reasoning_content into content when content is empty, via a fail-open recipe fetch shim (#2617). - OpenRouter: map openrouter_api_key (config + env) into OPENROUTER_API_KEY through buildGatewayConfig; register agent.use_gateway_loop, zeroentropy and openrouter keys in KNOWN_CONFIG_KEYS so `config set` accepts them (#2572, config key from #2112). Preserves JSONB (no JSON.stringify into ::jsonb), engine parity, source isolation, and trust-boundary invariants. Verified with the gbrain-pr-test-env consumer matrix (clawlancer/Postgres, gstack + hivemindos/PGLite) baseline-FAIL -> candidate-PASS on the same repro, plus bun run verify (31/31) and 439 targeted unit + e2e tests. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: thomaskong119 <thomaskong119@hotmail.com> Co-authored-by: maxpetrusenkoagent <max.petrusenko.agent@gmail.com> Co-authored-by: brettdavies <brettdavies@users.noreply.github.com> Co-authored-by: ivandebot <187176982+ivandebot@users.noreply.github.com> Co-authored-by: Rafael Reis <rafael.reis@contabilizei.com.br> Co-authored-by: fbal23 <fbal.public@gmail.com> Co-authored-by: javieraldape <javieraldape@users.noreply.github.com> Co-authored-by: David Carolan <david@joyrestart.com> Co-authored-by: Masashi-Ono0611 <masashi.ono.0611@gmail.com> Co-authored-by: spiky02plateau <155588579+spiky02plateau@users.noreply.github.com> Co-authored-by: psam-717 <mphilannorbah@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-16 16:31:53 -07:00
vinsewandGitHub	d0447a597b	fix(files): normalize bigint sizes before JSON serialization (#472) ... Postgres returns BIGINT file sizes as native BigInt values. Returning those values directly from file_list makes JSON serialization fail, and using them in CLI arithmetic can also throw. Convert size_bytes to Number at the operation boundary and in the CLI display path. File sizes remain exact well beyond any practical attachment size. Add a unit regression that exercises a native BigInt row and proves the operation result is JSON-serializable, plus a real-Postgres E2E assertion for the file_list response.	2026-07-16 16:27:16 -07:00
Time AttakcandGitHub	010847c020	fix(think): enforce source scope across gather (#2200) (#2739) ... Carry the caller's scalar or federated source scope through every think gather stream: hybrid page retrieval, takes keyword/vector retrieval, and graph traversal. Adds source predicates to the takes retrieval methods in both engines. Part of #2200 (the think slice; #2200 stays open as the tracking issue for the remaining by-slug read ops).	2026-07-13 00:09:27 -07:00
Time AttakcandGitHub	8e84c5b4a1	fix(facts): parse escaped pipes in facts fence round-trips (#2726) (#2738) ... parseRowCells now splits on unescaped pipes only and decodes escaped pipes while preserving ordinary backslashes and empty cells, so facts whose text contains literal | survive the fence->DB reconcile instead of being silently deleted. Fixes #2726.	2026-07-13 00:09:16 -07:00
Time AttakcandGitHub	42ab0956a4	fix(migrate): preserve sources and scope resume targets (#2677) (#2736) ... migrate --to now copies the complete source catalog before pages (fixes the pages_source_id_fkey failure on multi-source brains), and resume manifests carry an opaque target identity so a checkpoint from one target is discarded for a different target. Fixes #2677.	2026-07-13 00:08:52 -07:00
3 people	dde1132a29	v0.42.55.0 fix(security): dotfile/skills/slug confinement + DCR consent default + schema-lint migration (#418 #419 #245 #1353 #1647 #171 #1385) (#2399) ... * fix(security): confine routing dotfiles, skills dir, slugs, and transcription exec Shared src/core/path-confine.ts consolidates the realpath-containment idiom (moved from sources-ops.ts) and adds isTrustedDotfile + isWriteTargetContained. - .gbrain-source (source-resolver) and .gbrain-mount (brain-resolver) walk-up dotfiles are now lstat trust-gated: a symlink, foreign-owned, or world-writable file is refused on multi-user hosts (#418), fail-closed on stat error. - resolveWorkspaceSkillsDir + every skills-dir tier (env, cwd_walk_up, repo_root, cwd_skills, install_path) route through realpath containment so a symlinked workspace/skills can't escape the declared workspace (#419). - resolveSourceId/resolveBrainId realpath both sides of the registered local_path / mount prefix match so a symlinked cwd can't misattribute source/brain. - validateSlug rejects NUL/control, bidi/RTL overrides, backslashes, and URL-encoded path separators at the shared putPage/updateSlug chokepoint; write-through confirms the file path stays within the source tree. - transcribeLargeFile uses execFileSync arg-arrays + fs.rmSync (no shell), so a path with shell metacharacters is never parsed by a shell (#245). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(security): default dynamic-registration clients to authorization_code Self-registered DCR clients (the unauthenticated network registration path) previously defaulted to the client_credentials grant, which bypasses the /authorize consent screen. They now default to authorization_code; an explicit client_credentials request is rejected with invalid_client_metadata unless the operator opts in with the new --enable-dcr-insecure flag. A loud stderr WARNING prints at startup whenever DCR is enabled (#1353). Manual CLI/admin client registration is unchanged (operator-trusted). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(security): schema-lint hardening migration (search_path + view security_invoker) Migration v120 brings existing brains to the same posture as fresh installs: - ALTER VIEW page_links SET (security_invoker = on) on Postgres so the view honors the caller's RLS instead of the owner's (the view-through-RLS bypass). - ALTER FUNCTION ... SET search_path on the gbrain-owned trigger/event functions (both engines, IF EXISTS so engine-only functions are skipped; body untouched, so the load-bearing auto_enable_rls event trigger is unchanged). Closes #171. - Broaden the BYPASSRLS preflight in the historical RLS migration gates to honor superuser and inherited-role BYPASSRLS, so a superuser-connected fresh install no longer aborts (#1385). Fresh-install function definitions in schema.sql / pglite-schema.ts are born-correct (regenerated schema-embedded.ts). scripts/check-search-path.sh is a new CI guard (wired into verify) that fails if a trigger function in the schema base files is added without SET search_path. Postgres-only assertions live in the bootstrap E2E; the PGLite path is covered by test/migration-v120.test.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * v0.42.55.0 fix(security): dotfile/skills/slug confinement, DCR consent default, schema-lint migration Bump VERSION + package.json to 0.42.55.0 and add the CHANGELOG entry for the security-hardening wave (#418 #419 #245 #1353 #1647 #171 #1385). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(security): note the DCR consent default in SECURITY.md (#1353) The "disable client_credentials, only allow authorization_code" guidance is now the built-in DCR default; document the new --enable-dcr-insecure escape hatch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(todos): add takes_search + code_def to the federated by-slug P1 (#2200) The v0.42.55.0 eng-review codex pass flagged takes_search (holder-allowlist only) and code_def (brain-wide raw SQL over content_chunks) as remaining same-class surfaces. Noted on the existing #2200 P1 follow-up, with the caveat that the #2399 close-list deliberately keeps #1371/#2200 open until this lands. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(security): correct plpgsql alias collision in #1385 BYPASSRLS gate (real-PG) The broadened BYPASSRLS preflight aliased `pg_roles r`, but several RLS DO-blocks already declare `r record` for their backfill FOR loop, so plpgsql resolved `r.oid`/`r.rolbypassrls` to the unassigned record variable → "record \"r\" is not assigned yet" on real Postgres (PGLite tolerated it; the DATABASE_URL-gated e2e jobs are the backstop). Renamed the subquery alias to `pr` at all 10 migrate.ts sites; also broadened the schema.sql base RLS gate the same way (with the `pr` alias) for #1385 consistency on superuser fresh installs, and regenerated schema-embedded.ts. Also fixes a PRE-EXISTING engine-parity bug (confirmed failing on clean origin/master): the relationalFanout shape compared `canonical_chunk_id`, a serial id that diverges between a fresh PGLite engine and a shared Postgres DB (setupDB TRUNCATEs without RESTART IDENTITY). Compare its presence, not the exact value. Validated on real Postgres (pgvector/pg16): migration v120 applies, the v35 RLS backfill runs, and engine-parity + postgres-bootstrap + jsonb-parity are green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-02 08:00:58 -07:00
3 people	814258dda6	v0.42.53.0 fix(sync,db): #2339 op_checkpoints jsonb double-encode + bug-class sweep + CI guard (#2375) ... * fix(sync): op_checkpoints pin write double-encodes jsonb — every sync aborts (#2339) recordCompleted bound JSON.stringify(array) to a $3::jsonb param via postgres.js .unsafe(), double-encoding it into a jsonb string scalar that violates the v119 op_checkpoints_completed_keys_array CHECK — aborting every multi-source sync on real Postgres at the first checkpoint write. PGLite parses the string silently, which is why unit tests stayed green and it shipped. Cast through $3::text::jsonb so the text->jsonb cast parses a genuine array. Adds a DATABASE_URL-gated parity test + a dedicated Postgres CI job so the guard can never silently skip. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(db): sweep positional jsonb double-encode sites + AST CI guard (#2324) Every executeRaw/.unsafe site that bound JSON.stringify(x) to a bare positional jsonb cast double-encodes on real Postgres (same class as #2339). Sweep them all to the text::jsonb form across query-cache, sources-ops, llm-base, calibration-profile, impact-capture, subagent, receipt-write, traversal-cache, symbol-resolver, and the agent/sources commands. Adds scripts/check-jsonb-params.mjs (AST-lite scanner for the positional form the legacy template grep misses, incl. generic-typed calls), wired into check-jsonb-pattern.sh, with a self-test. PGLite's native db.query is not scanned — it parses text to jsonb natively, so the bug can't occur there. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(search,eval): alias-hop injected results carry page_id (contradiction-probe crash) applyAliasHop injected synthetic SearchResults without page_id (the `as SearchResult` cast hid the missing field), so listActiveTakesForPages bound undefined/NaN into ANY($1::int[]) and crashed the whole contradiction probe on real Postgres. Stamp page_id=page.id at the injection site and add a finite-id filter in generateIntraPagePairs as a defensive backstop (mirrors hybrid.ts:63). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(engines): positional jsonb binding rule (text::jsonb vs the double-encode trap) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * v0.42.53.0 fix(sync,db): #2339 op_checkpoints jsonb double-encode + bug-class sweep + CI guard Bumps VERSION + package.json to 0.42.53.0, adds the CHANGELOG entry, and regenerates llms-full.txt. Ships the #2339 sync-abort hotfix, the repo-wide positional jsonb double-encode sweep, the alias-hop contradiction-probe crash fix, and the new positional-form CI guard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: post-ship sync — jsonb invariant now covers the positional form + new guard CLAUDE.md JSONB invariant + KEY_FILES (sql-query, check-jsonb-pattern, op-checkpoint) now describe the #2339 positional double-encode class, the $N::text::jsonb fix, and the new check-jsonb-params.mjs guard. Regenerates llms-full.txt. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-24 06:05:16 -07:00
3 people	bb2e88c42a	v0.42.52.0 fix(reliability): autopilot dead-job storm + supervisor wedge + sync/status/minion reliability (#2194 #2227 #1994 #1737 #1738 #1950 #1984) (#2287) ... * test(supervisor): pin LOCK_HELD fence-exit is never counted as a crash (#2227) A duplicate supervisor loses the queue-scoped DB singleton lock (#1849) and exits LOCK_HELD before spawning a worker or emitting 'started'. summarizeCrashes counts only worker_exited, so the fence path is structurally uncountable. Pin it so a future refactor that logs worker_exited on the fence path fails here instead of silently re-introducing the crash-budget breaker-trip loop. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autopilot): per-source cycle binds FS phases to source.local_path, not global repo (#2194 #2227) A per-source autopilot-cycle inherited the global sync.repo_path as brainDir while stamping DB freshness for source_id — mixed scope. FS phases (sync/lint/extract) ran against the wrong tree, so the failure-cooldown and freshness gates would attribute work to the wrong source. Resolve the source's local_path in the handler (reuse the archive-recheck SELECT) and bind brainDir to it; a pure-DB source gets null (FS phases skip) instead of falling through to the global checkout. Legacy no-source dispatch keeps the global repoPath. Prerequisite for the cooldown/split commits (codex outside-voice #8). Resolves TODOS:634. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(supervisor): detect a live supervisor via the DB lock under split $HOME (#2227) jobs supervisor status + doctor read the HOME-derived pidfile, so a supervisor started under a different $HOME (keeper=/root vs ops=/data) read as 'not running' while healthy — the false signal that drives an operator to spawn a duplicate. Both surfaces now fall back to the queue-scoped DB singleton lock (#1849), the HOME-independent authority, when the pidfile shows nothing. New isLockHolderLive keys on lock freshness (ttl + heartbeat steal-grace), never process.kill, so PID reuse can't false-positive (pid-liveness-alone-pid-reuse). Status surfaces the holder host/pid + recorded concurrency/max-rss from the latest started event. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(supervisor): degraded retry instead of permanent give-up on crash storm (#1994 #2227) max_crashes_exceeded gave up forever, so a transient DB-pooler blip that tripped the soft budget wedged the queue until a human restart (#2227's breaker-trips tail). Crossing the soft budget now enters degraded mode: keep respawning with capped exponential backoff (60s cap — a paced retry, not a hot loop) and emit a loud crash_budget_degraded health_warn. The existing stable-run reset clears the count once a respawn survives >5min, so a recovered DB self-heals. Permanent give-up fires only at a much-higher hard ceiling (maxCrashes × 10), tunable/disablable via GBRAIN_SUPERVISOR_HARD_STOP_CRASHES (0 = never). Resolves TODOS:92. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autopilot): clamp fan-out to worker concurrency + doctor warning (#2194) Fan-out resolved to 4 (Postgres) regardless of worker --concurrency, so surplus cycles queued behind the worker and raced the stalled-sweeper. Two fixes for the same mismatch: - resolveEffectiveFanoutMax clamps to max(1, concurrency-1) (reserve a slot), gated on a LIVE DB-lock holder so a stale started-audit row can't shrink throughput (codex #9/D5); no live holder → unknown → unclamped base. Escape hatch autopilot.fanout_clamp_to_concurrency. - doctor's autopilot_fanout_concurrency check warns when fan-out exceeds effective slots — the misconfig was silent before. Advisory (started-event concurrency), wired into both doctor surfaces. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autopilot): per-source failure cooldown — break the dead-job storm (#2194) Only SUCCESS gated dispatch, so a source whose cycle kept failing/timing-out re-fanned-out every 5-min tick forever (200+ dead jobs/24h). Now a failed source backs off with bounded exponential cooldown (10→120min). Read at DISPATCH from minion_jobs dead/failed rows (timeouts/RSS-kills dead-letter via SQL and never run handler code, so a write-only hook would miss them) AND re-checked at CLAIM time in the handler (codex #5: already-queued/retrying jobs). A success clears it (codex #7); null-source rows excluded (codex #6); engine-parity via executeRaw. Disable with autopilot.failure_cooldown_min=0. Fail-open if config/history reads error. Surfaced via fanout_cooldown_skipped + the fanout summary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autopilot): split the cycle — per-source phases + one global-maintenance job (#2194 #2227) N per-source cycles each ran the brain-wide global phases (embed-all/orphans/ purge/…) concurrently, thrashing the same rows and taking the worker 4→10GB in <60s → RSS-kill → orphaned stalls. Split them: per-source jobs now run only source-scoped (+ mixed) phases and stamp last_source_cycle_at; a new autopilot-global-maintenance job runs the global phases ONCE per window (idempotency_key + maxWaiting:1 = structural single-flight) and stamps autopilot.last_global_at. This is the codex-endorsed design that replaced the rejected skip-and-stamp-fresh approach (codex #1/#2): no freshness poisoning, no starvation — global work always runs as its own job, never marked done when it wasn't. PHASE_SCOPE is now a runtime partition (GLOBAL ∪ NON_GLOBAL == ALL). last_full_cycle_at still written for doctor/legacy (no longer a global gate). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(doctor): guard nullable engine in supervisor DB-lock fallback (#2227) Follow-up to the supervisor-visibility commit: doctor's engine binding is BrainEngine | null, so the inspectLock fallback must guard on a non-null engine (tsc TS2345). No behavior change — a null engine simply skips the DB-lock probe and falls back to the pidfile reading, as before. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(doctor): categorize autopilot_fanout_concurrency check as ops (#2194) Follow-up to the fan-out/concurrency commit: the doctor-categories drift guard requires every check name in doctor.ts to belong to exactly one category set. Add the new autopilot_fanout_concurrency check to OPS_CHECK_NAMES (infrastructure liveness, alongside wedged_queue/supervisor). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: update KEY_FILES for the autopilot cycle split + supervisor degraded-retry (#2194 #2227) Post-ship document-release: refresh the KEY_FILES current-state entries that drifted — cycle.ts (GLOBAL/NON_GLOBAL phase split + last_source_cycle_at / autopilot.last_global_at), jobs.ts (per-source local_path brainDir, claim-time cooldown, autopilot-global-maintenance handler), supervisor.ts + child-worker (degraded retry instead of permanent give-up; hard ceiling), db-lock.ts (isLockHolderLive), handler-timeouts (new handler). Regenerated llms bundle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(minions): handleTimeouts counts the timed-out run as a spent attempt (#1737) The per-job timeout_at dead-letter (handleTimeouts) set status='dead' without incrementing attempts_made, unlike the wall-clock and stall dead-letter siblings. It is the FIRST killer to fire for the long-lane handlers (subagent / embed-backfill / autopilot-cycle) because timeout_ms is stamped at submit, so a timed-out long job reported `attempts: 0/N (started: N)`. Mirror the siblings with attempts_made + 1 (terminal, no retry). Safe against double-count: the worker sweep runs handleStalled -> handleTimeouts -> handleWallClockTimeouts sequentially and awaited, each guarded on status='active', so the first to dead-letter excludes the row from the rest. Regression assertions added (test/minions.test.ts + e2e/minions-resilience.test.ts) so the increment can't be silently dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agent): recognize trailing switches in `agent run`, keep prompts freeform (#1738) parseRunFlags() broke flag parsing at the first positional token, so any flag after the prompt (`gbrain agent run "do X" --detach`) was swallowed into the prompt string and silently ignored. Now the no-value switches --detach/--follow/ --no-follow are hoisted when they trail the prompt, while everything else stays verbatim: an unknown --word is treated as prompt text (no "unknown flag" throw), a --switch mid-prompt is preserved, and `--` suppresses hoisting entirely for a literal escape. Value-flags now reject a missing or flag-shaped value (and --max-turns/--timeout-ms a non-number) instead of capturing undefined/NaN. Contract change: a prompt that starts with or trails an unguarded --word no longer errors; a literal trailing --detach needs `--`. Help text updated; tests revised + extended (test/agent-cli.test.ts). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sync): honest live-sync status + progress-aware stall-abort (#1950) Finishes the #2255 honest-freshness story for two gaps it left. (a) `gbrain sources status` printed "idle" while a sync proc held the per-source lock (the reported bug). New shared liveSyncStatus() helper in db-lock.ts reads the SAME live-lock signal `gbrain doctor` uses; runStatus now shows "running" (BACKFILL column + a sync_running field in --json) and suppresses the misleading "never synced" warning while a sync is live. One helper, so the surfaces can't drift (doctor/status retrofit tracked as a follow-up). (b) A sync wedged-but-alive kept refreshing its lock heartbeat (it fires on its own timer) and hadn't hit the wall-clock deadline, so only a manual pkill freed it. New in-band stall watchdog keys off FORWARD IMPORT PROGRESS (progress.tick), not the heartbeat: if no file completes for GBRAIN_SYNC_STALL_ABORT_SECONDS (default 900s), it aborts via a controller composed into opts.signal, so the drain returns partial() (last_commit unchanged, next run resumes from the checkpoint) and withRefreshingLock releases the lock. Limits, documented in code: a single file slower than the window trips it; a fully starved event loop won't fire the timer (the wall-clock hard deadline is that backstop). Tests: liveSyncStatus (live/expired/none/per-source) in db-lock-inspect; the resolveStallAbortSeconds env matrix in sync-hard-deadline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(status): version field + per-section --deadline-ms budget (#1984) `gbrain status` had no version in its JSON envelope and could hang on a slow connection with no way to get a partial answer. Two additions: - version: the StatusReport JSON now carries the local gbrain CLI version so a poller can pin behavior to a build. Thin-client also surfaces remote_version (the brain server's version), and the get_status_snapshot MCP op reports its version for that parity. - --deadline-ms=N / --fast: a shared wall-clock budget. Each section is raced against the REMAINING budget via Promise.race (NOT process-watchdog, which SIGKILLs and can't return partial output), so one slow/hung section can't strand the snapshot — it's marked stale and the rest still return. The envelope gains partial:true + stale_sections[]; exit code stays 0 (a snapshot was produced). Invalid --deadline-ms → exit 2. Tests: parseDeadlineFlag + withSectionDeadline (hermetic), the usage-error exit, version presence in the PGLite envelope, and the op's version key. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sync): report stall_timeout distinctly + document in-flight limit (#1950) Pre-landing review (codex + adversarial): the stall watchdog aborted opts.signal but the per-iteration abort checks returned partial('timeout'), collapsing a wedge-reap into a user --timeout/SIGINT so JSON consumers couldn't tell them apart. Add a 'stall_timeout' reason (set via a stallAborted flag) on the three import-loop abort sites; deletes/renames-phase and checkpoint sites stay 'timeout'. Sharpen the watchdog comment: the abort is observed BETWEEN files, so a hang inside a single importFile is not interrupted until it returns (TODO: thread a cancellation signal through importFile). * fix(agent): `--` escape suppresses trailing-switch hoisting anywhere (#1738) Pre-landing review: the leading-flag loop breaks at the first positional, so the `escaped` flag only fired for a leading `--`. A `--` placed after a positional left trailing-switch hoisting active, so `agent run note -- body --detach` silently detached and dropped the `--` as junk. Suppress hoisting whenever a literal `--` appears in the prompt. Regression test added. * fix(status): deadline-ms usage-error + scoped stale_sections + cancel losing remote call (#1984) Pre-landing review (codex): (1) bare `--deadline-ms` with no value silently fell through to no-budget/--fast instead of a usage error; (2) thin-client timeout reported both sync+cycle stale even under `--section sync`, naming a section the caller excluded (local path was already correct); (3) the section race abandoned the remote promise locally but didn't cancel the in-flight MCP call — pass the budget as timeoutMs so the losing side actually cancels. Regression test added. * v0.42.52.0 fix(reliability): autopilot dead-job storm + supervisor wedge + sync/status/minion reliability (#2194 #2227 #1994 #1737 #1738 #1950 #1984) Bundles the already-reviewed autopilot/supervisor stabilization (#2194 #2227 #1994: cycle split, per-source failure cooldown, fan-out clamp, degraded supervisor retry, DB-lock live-supervisor detection) with four operational fixes: minion timeout attempt-accounting (#1737), agent-run trailing-flag parsing (#1738), honest live-sync sources status + progress-aware stall watchdog (#1950), and status version + --deadline-ms partial result (#1984). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: document GBRAIN_SYNC_STALL_ABORT_SECONDS env knob (#1950) Post-ship doc sync (/document-release): add the sync stall watchdog env var to the CLAUDE.md sync-tuning table (Five → Six knobs) + regenerate the llms bundle. * test: quarantine #2249 fanout tests as *.serial (R1 env-isolation) (#2194) The cherry-picked autopilot-fanout-clamp + doctor-autopilot-fanout-concurrency tests mutate process.env.GBRAIN_AUDIT_DIR in beforeEach/afterEach, which the check:test-isolation R1 lint flags (parallel shards load multiple files per process). Rename to *.serial.test.ts (sanctioned quarantine — they run under --max-concurrency=1) instead of restructuring the reviewed test bodies. No logic change; both files stay green (9 tests). Fixes the failing verify CI check. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-21 06:36:43 -07:00
3 people	c023a6041d	v0.42.46.0 fix(engine): federated read scope reaches by-slug reads (#2200) (#2239) ... * fix(engine): federated sourceIds[] scope on by-slug secondary reads (#2200) getTags/getLinks/getBacklinks/getTimeline (both engines) + TimelineOpts now accept a federated `sourceIds[]` read grant, precedence over scalar sourceId, filtering `source_id = ANY($::text[])` — mirroring getPage from v0.42.37.0. - getTags: `page_id = (subquery)` -> `IN (subquery)` + DISTINCT so a slug present in >1 granted source unions tags instead of throwing on a multi-row subquery. - getLinks/getBacklinks: federated branch scopes ALL THREE page endpoints (from, to, AND the authoring origin) so a cross-source link can't disclose a foreign slug. Scalar/unscoped branches unchanged (trusted internal callers keep the cross-source view). - getTimeline: Postgres 8-branch cartesian tree collapsed to one fragment-composed query; PGLite adds the sourceIds branch to its dynamic WHERE. * fix(ops): route by-slug reads through the federated source scope (#2200) get_page resolves tags against the concrete page's source; get_tags/get_links/ get_backlinks/get_timeline route through sourceScopeOpts(ctx) (replacing the copy-pasted scalar `ctx.sourceId ? {sourceId} : {}`). New linkReadScopeOpts promotes an UNTRUSTED remote scalar scope to sourceIds[] so legacy/pre-federated tokens also get all-endpoint link scoping; trusted local CLI keeps cross-source. * test: federated read scope on by-slug reads + engine parity (#2200) Per-op federated reads, isolation (out-of-grant -> empty), cross-source decoy guard, far-endpoint + origin leak guards (F1), same-slug union (D3A), empty-array contract, scalar-remote promotion (D1), getTimeline date-window after the Postgres fragment refactor (D5A), and engine-parity arms for all four methods. * chore: bump version and changelog (v0.42.46.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(key-files): document federated by-slug read scope + linkReadScopeOpts (#2200) The #2200 fix routed get_page tags + get_tags/get_links/get_backlinks/ get_timeline through the federated source scope and added sourceIds[] to the engine read methods + TimelineOpts. Bring KEY_FILES.md to current state: the operations.ts entry's sourceScopeOpts read-op list now includes the by-slug reads and documents the linkReadScopeOpts helper (three-endpoint link scoping + untrusted-remote scalar promotion); the engine.ts entry notes the by-slug read methods + TimelineOpts carry the same sourceIds[] federated axis. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 22:32:38 -07:00
3 people	a81f7e05e8	v0.42.43.0 feat(context): push-based context (#2095) + teardown-exit hardening (#2084) (#2175) ... * fix(cli): exit deliberately after bounded teardown instead of riding the 10s backstop (#2084) Root cause: bounded teardown (endPoolBounded, #2015) RESOLVES, but lingering sockets — embedding-provider fetch keep-alive, PgBouncer txn-mode sockets the bound raced past — keep Bun's event loop alive, so every `gbrain query` paid a flat 10s tax exiting via the hard-deadline force-exit banner. Three changes, one contract: - flushStdoutThenExit (cli-force-exit.ts): when main() resolves and the command is not a daemon, exit deliberately — after stdout AND stderr drain (writableLength===0, 'drain'-event + poll loop, 2s unref'd guard for a blocked pipe). Incident #1959 (force-exit truncating piped stdout) is the regression class; pinned by a 256KB real-pipe subprocess test. - drainThenDisconnect (cli.ts): ONE owner-disconnect helper at all 8 sites (op-dispatch, CLI_ONLY fall-through, search dashboard, doctor remediation x3, ze-switch, dream, read-only timeout path). Drains the background-work registry, then disconnect (best-effort), bounded by the 10s unref'd hard-deadline — which is now armed around the TEARDOWN window only, not before the op handler (the old placement would have force-killed any op slower than 10s). Closes the filed TODOS P3 drain-hoist: six sites previously skipped the drain entirely and had no hang timer at all. - Inner process.exit sweep: mid-handler exits in engine-owning/output-bearing paths (status, friction, claw-test, smoke-test, eval cross-modal / takes-quality replay / conversation-parser / whoknows-thin, status-thin) become process.exitCode + return so they flow through the drains and the flush-exit. Pre-engine usage/parse/refusal exits stay as-is. BrainRegistry.disconnectAll deliberately unchanged: zero production callers in src/, per-engine disconnects already bounded, and the kernel reclaims sockets on exit (src/core/timeout.ts doctrine). DAEMON_COMMANDS gains 'watch' ahead of the #2095 push transport. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(e2e): PgBouncer transaction-mode pooler in CI + teardown e2e (#2084) Three consecutive waves (#1972 → #2015 → #2084) fixed pooler-teardown bugs verified only against one production deployment — CI had no transaction-mode pooler and could never see the class. Now it can: - docker-compose.ci.yml: `pgbouncer` service (transaction pooling) fronting postgres-1, mirroring the production split-pool topology (direct :5432 + pooled :6543). AUTH_TYPE=plain (pg16 SCRAM verifiers need the plaintext password in the userlist) + IGNORE_STARTUP_PARAMETERS for the statement_timeout/idle_in_transaction_session_timeout startup params gbrain's client sets (the Supabase pooler whitelists the same). - test/e2e/pgbouncer-teardown.test.ts: schema + fixture via the DIRECT url into a dedicated `gbrain_pgbouncer` database (never races shard TRUNCATEs), then spawns the real CLI against the POOLED url and asserts: exit 0, stdout intact (the #1959 truncation class), and NO "did not return within 10000ms — force-exiting" banner (pre-#2084 it printed on 100% of query-shaped ops on this topology). Class bound, not exact timing. Skips gracefully without GBRAIN_PGBOUNCER_URL. - scripts/ci-local.sh: threads GBRAIN_PGBOUNCER_URL + GBRAIN_PGBOUNCER_DIRECT_URL into all three e2e phases. Verified live: both tests green against pgbouncer 1.25.2 in transaction mode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(schema): context_volunteer_events table (v116) — push-context feedback log (#2095) One row per page the brain volunteers (op / reflex / watch channels). "Used" is DERIVED, never written: pages.last_retrieved_at > volunteered_at (the existing bumpLastRetrievedAt write-back is the open/cite signal), so there is no second tracking path. session_id/turn are nullable caller-supplied attribution; rationale is a deterministic template string, never raw conversation text. - Migration v116 (idempotent) + mirrors in src/schema.sql + src/core/pglite-schema.ts + regenerated schema-embedded.ts (regen also folds in pre-existing comment-only drift from the v114 links edits). - src/core/context/volunteer-events.ts: insertVolunteerEvents (ONE multi-row parameterized INSERT — never per-row awaited round-trips) + purgeStaleVolunteerEvents (90-day GC, returns 0 on pre-v116 brains). - Dream cycle purge phase prunes stale events alongside op_checkpoints / brainstorm checkpoints / batch-retry audit files. - RLS on Postgres comes from the v35 auto_rls_on_create_table event trigger (the same mechanism that covered v110 page_aliases and v115 op_checkpoint_paths); the volunteer Postgres e2e pins it. - No ::jsonb anywhere; no bootstrap probe needed (nothing references the table pre-creation; writers guard with try/catch). Tests: v116 shape + columns + indexes + live insert/purge round-trip on PGLite (test/migrate.test.ts, 161 pass); schema-bootstrap-coverage green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(context): multi-turn window extraction + confidence-scored volunteer core (#2095) - entity-salience.ts: extractCandidatesFromWindow(turns) — runs the existing per-turn extractor across the last N turns (oldest→newest), merges by the normalizeAlias form with occurrence/newest-turn/user-mention metadata, and orders by salience (recency > frequency > user-role) so the MAX_CANDIDATES cap drops stale assistant chatter, not the entity the user just named. Closes the filed assistant-introduced-entities recall TODO; true pronoun coreference (never-named antecedents) stays out of scope. - retrieval-reflex.ts: ReflexPointer gains source_id + arm + confidence + matchedNorm. ARM_CONFIDENCE (alias 0.9 / title 0.8 / slug-suffix 0.6) lives next to the arm definitions so identity and score can't drift. Arm-2 provenance is classified in JS (codex D8 — the combined OR can't report which predicate matched). Federated sourceIds[] scope (alias arm loops per source; arm 2 uses source_id = ANY — no engine-interface change). Suppression gains 'slug-only' mode (codex D7, REQUIRED for windowing): the legacy title-whole-word rule would suppress every entity merely MENTIONED in a prior window turn, breaking the feature by construction — slugs only enter context when a pointer/page was actually surfaced. Default stays 'slug-and-title' for the window=1 legacy path. - volunteer.ts (new): parseWindow (lenient user:/assistant: prefixes, CRLF, unprefixed → one user turn), volunteerContext (zero-LLM: extract → resolve → +0.05 multi-turn/newest-turn boost → min_confidence 0.7 gate → cap 3/5; deterministic rationale strings, never raw conversation text), and volunteerUsageStats (per-arm/channel precision from the last_retrieved_at join, labeled approximate — 5-min throttle false negatives, unrelated-read false positives; codex D9). Tests: 35 green across volunteer-context (window parsing, pronoun follow-up via assistant-introduced entity, confidence gating, slug-only suppression, takes-fence privacy, multi-source scope, caps, stats join math) + retrieval-reflex back-compat + resolve-ipc. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ops): volunteer_context op — CLI (stdin) + MCP, drained event sink (#2095) New read-scope op on the contract surface (CLI `gbrain volunteer-context` with stdin → window, MCP tool for free): takes a rolling conversation window, returns confidence-gated page pointers with rationales + synopses. `window` is optional-unless-stats (validated in the handler, codex D9); `stats: true` returns the volunteered-vs-used precision summary, labeled APPROXIMATE (the 5-min last-retrieved throttle and unrelated reads both bias the join). Source scope threads through sourceScopeOpts — federated grants narrow the volunteer to the granted sources. Event logging is fire-and-forget through a new `volunteer-events` background-work sink (volunteer-events.ts, mirrors last-retrieved: tracked dangling promise set + bounded drain + snapshot-drop on timeout so a long-lived process never accumulates ghosts). ONE batched INSERT per call, drained on every exit path by the commit-1 drain hoist; failure never fails the op (pinned by an injected failing-engine test). cli formatResult renders both shapes (pointer lines with confidence/arm/ rationale; the stats summary with per-arm precision). Tests: op contract surface, window-required validation, sink round-trip with session_id/turn attribution, failing-engine fail-open, federated grant scoping, stats mode (26 green on PGLite) + a real-Postgres e2e proving the op + sink + stats join AND that context_volunteer_events has RLS enabled (keeps the auto-RLS event-trigger mechanism honest for v116). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(context): reflex consumes the rolling window + ambient-channel logging (#2095) The default-on retrieval reflex now extracts entities from the last N turns (retrieval_reflex_window_turns, default 4; env GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS; window=1 reproduces the legacy current-turn-only behavior exactly). assemble() passes the recent user/assistant turns (hard cap 12); the reflex slices to the configured window. Assistant-introduced entities and "what did she invest in?" follow-ups whose antecedent was NAMED in the window now surface pointers — the issue's "zero agent-initiated queries" success criterion on the ambient path. Under windowing, suppression switches to slug-only (codex D7): the legacy title-whole-word rule would suppress every entity merely MENTIONED in a prior window turn, breaking the feature by construction. Slugs only enter prior context when a pointer/page was actually surfaced, so already-surfaced pages still suppress. The suppression mode flows through all three resolver rungs (host opts, serve IPC request, direct Postgres). Ambient-channel feedback (codex D11): the server-side resolver paths (serve IPC + direct Postgres) log volunteered pointers with channel: 'reflex' through the drained volunteer-events sink, so `gbrain volunteer-context --stats` measures the default-on path where most volunteering happens. Host-injected resolvers (no gbrain engine) can't log — documented gap. Precision gates, 1.5s ceiling, fail-open, and the pointer cap are unchanged. Tests: prev-assistant-turn entity fires; window=1 legacy parity; slug-only vs already-surfaced suppression; throwing resolver stays fail-open (16 green). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(cli): gbrain watch — push transport over stdin (#2095) The issue's headline: the brain volunteers pages as the conversation flows, instead of waiting to be asked. `some-transcript-feed | gbrain watch` reads turns line-by-line ('user:'/'assistant:' prefixes set the role; unprefixed lines are user turns), keeps a rolling window (--window-turns, default 4), and streams confidence-gated pointers with rationales to stdout (--json for JSONL). Session dedupe rides the core's slug-only suppression — a slug is volunteered at most once per session. Events log on channel 'watch' with session_id + turn through the drained sink. Lifecycle: watch BLOCKS in the stdin iteration (like `jobs work`) — an interactive TTY stays alive until Ctrl-C/Ctrl-D, piped input ends at EOF — so it is deliberately NOT in DAEMON_COMMANDS (reverts the commit-1 placeholder): when main() resolves the work is over, the CLI_ONLY finally drains volunteer events via drainThenDisconnect, and the entrypoint flush-exit ends the process. Keeping it in the daemon set would have made the piped EOF path hang on lingering sockets — the exact #2084 class. SIGINT closes the stream and flows through the same drain path instead of killing mid-write. Per-turn resolution failures are fail-open (the stream never dies on a transient DB error). Full wiring (eng-review D12): CLI_ONLY + CLI_ONLY_SELF_HELP (WATCH_HELP) + THIN_CLIENT_REFUSED_COMMANDS (thin clients use the volunteer_context MCP op) + main --help entry. Tests: 18 green — help, per-turn volunteering + clean EOF return, rolling window via assistant-introduced entity, session dedupe, --json shape with turn attribution, channel-watch event rows, --min-confidence gate, CRLF/ blank tolerance, daemon-gate semantics. Live smoke: piped `gbrain watch` on a fresh PGLite brain exits 0 at EOF with no force-exit banner. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: KEY_FILES + push-context guide + TODOS for the #2084/#2095 wave - docs/architecture/KEY_FILES.md (current-state): context entries gain the window extractor, arm provenance/confidence, suppression modes, volunteer + volunteer-events modules; background-work entry now lists FIVE sinks and the drainThenDisconnect owner-disconnect contract; new entries for src/core/cli-force-exit.ts (the exit contract) and src/commands/watch.ts. - docs/guides/push-context.md (new): the three channels (reflex/op/watch), the confidence model, CLI usage, config keys, and the approximate-stats caveat. Linked from CLAUDE.md's reference map. - CLAUDE.md: ops line mentions volunteer_context + the guide link; bun run build:llms regenerated in the same commit (freshness test green). - TODOS.md: #2095 deferrals filed (SSE push channel, policy skill + doctor check, structured messages[] param); the #1981 entity-detection TODO narrowed (window extraction covered assistant-introduced entities + named-antecedent follow-ups); the drain-hoist P3 marked DONE by the #2084 wave. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(e2e): truncate context_volunteer_events in setupDB (#2095) The new feedback-log table wasn't in ALL_TABLES, so volunteered-event rows persisted across e2e runs on a reused database and poisoned count/stats assertions in volunteer-context-postgres on the second run. No FK to pages (slug join), so position before pages is for hygiene only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): own the exit verdict — never trust ambient process.exitCode (#2084) Caught by the full unit suite: `gbrain apply-migrations` on PGLite started exiting 99. Root cause: PGLite's Emscripten runtime writes the WASM backend's proc_exit status into process.exitCode (initdb at create-time, the postmaster at close-time — `exitCode=status` in pglite's dist), and the writes land ASYNCHRONOUSLY, outside any snapshot/restore window around create/close (a guarded attempt verified this). The pre-#2084 success path never read process.exitCode, so the pollution was invisible; the new deliberate flush-exit propagated it faithfully. Fix: gbrain records its own verdict. setCliExitCode(n)/getCliExitCode() in cli-force-exit.ts — every gbrain-owned exit-code assignment routes through the setter (still mirrored to process.exitCode for outside readers), and both exit paths (entrypoint flushStdoutThenExit + the drainThenDisconnect hard-deadline backstop) read the getter. Swept all assignment sites: cli.ts (op error, friction, claw-test, smoke-test, eval runners, status, import errors) + reindex/transcripts/brainstorm/frontmatter/autopilot. Also updates the v0.42.20 structural pins to the drainThenDisconnect shape (ordering invariant asserted INSIDE the helper + >=8 helper call sites, superseding the two-inline-pairs assertion). Verified: apply-migrations spawn test green; `init --migrate-only` exits 0; an errored op still exits 1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: re-pin the teardown-arming invariant at its post-#2084 home Master's v0.42.41.0 triage wave and the #2084 wave fixed the same pre-armed-timer bug independently; the merge keeps #2084's shape (arming inside the shared drainThenDisconnect helper, covering all 8 exit paths). The structural pin now asserts the same invariant — no pre-try arming; gated, unref'd, before-drain, cleared — at the helper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: coverage for ambient reflex-channel logging + watch window/cap flags Ship coverage audit (85%, gate PASS) named five gaps; the two substantive cheap ones close here: the codex-D11 logChannel='reflex' path now has a behavioral pin (events land on channel 'reflex' through the drained sink; no logChannel → no events), and gbrain watch's --window-turns / --max-pages flags are exercised (turn-1 attribution under window=1; cap to one page). Remaining flagged-not-blocking: the wallclock-timeout branch (untestable without >10s real-clock flake — same rationale as the arming pin), formatResult's volunteer case (module-private), and the cycle purge wiring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: close the remaining plan-audit gaps — formatResult rendering + watch SIGINT formatResult exported for tests (same import-safety contract as cliAliases); test/cli-format-volunteer.test.ts pins the pointer lines, empty-gate message, and approximate stats summary. test/watch-command.test.ts gains a real subprocess SIGINT test: piped stdin that never reaches EOF, SIGINT mid-stream, assert exit 0 with no force-exit banner — the drain-then-exit lifecycle under the actual signal, not just the shared exit path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: doctor's FAIL verdict was zeroed by the owned exit — sweep stragglers + class pin The merged-state suite caught it: doctor --fast --json reported FAIL but exited 0. Master's v0.42.41.0 brought raw `process.exitCode =` writes (doctor.ts hasFail ternary, extract.ts) that the #2084 verdict-owning exit silently zeroes — getCliExitCode() deliberately never reads ambient process.exitCode (the PGLite-Emscripten pollution defense), so any setter that bypasses setCliExitCode reports success on failure. Swept both sites and added the structural class pin: a test greps src/ for raw `process.exitCode =` outside cli-force-exit.ts, so the next merge that introduces one fails loudly instead of lying about exit codes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: bump version and changelog (v0.42.43.0) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: quarantine the watch SIGINT subprocess test to the serial lane The parallel unit shards flake on concurrent CLI subprocess spawns (failed at 7ms in-suite, green solo) — same isolation rationale as apply-migrations-pglite-spawn.serial.test.ts and #2141's R3 quarantine. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: update project documentation for v0.42.43.0 Post-ship doc verification against the release diff (#2095 push-based context + #2084 superset hardening), with a cross-model doc review: - push-context.md: version tag corrected to v0.42.43.0; per-call knobs now cover prior_context/days and watch's flag surface accurately; feedback-log writes described as best-effort; synopsis fence-strip described as unconditional. - CLAUDE.md: stale operation count (~47 -> ~90); volunteer_context release reference corrected to v0.42.43.0. - KEY_FILES.md: ci-local entry rewritten to current topology (4-shard parallel default, four Postgres services, transaction-mode PgBouncer + GBRAIN_PGBOUNCER_URL/_DIRECT_URL exports); stale E2E file counts dropped from the selector entry. - TESTING.md: inventory entries for the new #2084 structural pins (cli-exit-verdict-pin, cli-pipe-truncation), the push-context test suite (volunteer-context, watch-command, watch-sigint.serial, cli-format-volunteer), migrate v117 coverage, and the two new E2E files (pgbouncer-teardown env gating, volunteer-context-postgres RLS pin); check:all row corrected (not a superset of verify). - AGENTS.md + RELEASING.md: ci:local descriptions updated to the sharded + pooler topology. - CHANGELOG (wording only, entry preserved): "retrieved" instead of "opened" for the used-signal, pooler scoped to the local CI gate, feedback log labeled best-effort. - llms-config.ts: index the new push-context guide; bundles regenerated (build:llms) and freshness test green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(test): correct the v116 reference — the table shipped as migration v117 * fix: pre-landing review hardening — federated alias parallelism, trust-boundary clamps, shared protocol helpers (#2095) Five specialist reviewers (testing/maintainability/security/performance/ data-migration) on the reconciled diff; every finding applied: Performance: the alias arm now resolves all granted sources CONCURRENTLY (a federated caller paid M sequential RTTs per turn — ~355ms at 5 sources cross-region, inside the reflex's 1.5s budget); watch's session dedupe is O(1) Set membership instead of a monotonically growing priorContext string (O(T²) over a long-lived session); getWindowTurns iterates from the tail (per-turn cost no longer grows with session length); the resolver's provenance maps fold into the existing candidate pass. Security: volunteer_context clamps caller-supplied attribution at the trust boundary — session_id capped at 256 chars (a read-scoped token could bank ~1MiB TEXT per request, retained 90 days), turn logged only when a safe integer (a non-integer threw inside the batched INSERT and silently dropped the whole batch). The privacy comments now state precisely what rationale may contain (the matched entity's surface form — which by construction resolved to an existing alias/title/slug — never free conversation text). Maintainability: TURN_PREFIX_RE + formatVolunteeredPage exported from volunteer.ts and shared by watch/cli (the two surfaces can no longer drift); volunteerEventRowsFrom is the single VolunteerEventRow assembly site for all three channels; watch's window default now honors the same retrieval_reflex_window_turns config knob the reflex reads; the stale pre-v116 comments swept to pre-v117. Testing: the two flake-class CRITICALs fixed (pipe test asserts the backstop banner instead of a cold-CI-hostile 9s wall bound; the SIGINT test waits on watch's new machine-readable ready line instead of a fixed 15s sleep — 2.5s and deterministic now); new coverage for the sink's timeout branch + ghost-reference drop, watch per-turn fail-open, untrusted knob clamps (min_confidence/max_pages/days), window-cap ordering (newest user mention survives), serve-IPC suppression passthrough + channel=reflex logging, windowTurnCount edge semantics, and structural pins for the sink registration + cycle purge wiring. The exit-verdict pin's grep is now operator/whitespace-tolerant. Deferred with TODOs: resolver index shapes for the per-turn query; batched first-prune after a long dream-cycle gap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(context): red-team hardening — pre-cap dedupe, delivery-side reflex logging, window clamp Four red-team findings on the #2095 push-context surface: - RT1 starvation: watch's session-dedupe Set filtered AFTER volunteerContext's cap, so a recurring already-pushed entity burned cap slots every turn and starved fresh pages behind it. VolunteerOpts.excludeSlugs now skips inside the pointer loop BEFORE the confidence gate and the cap. - RT3 honest stats: reflex-channel event logging moved from inside the resolver to the DELIVERY point — serve's resolve-IPC onDelivered hook fires only after the response write succeeds, and buildReflexAddition logs only after the per-turn timeout admits the block. A block the client's 250ms budget abandoned was never injected and no longer counts as volunteered. (logChannel resolver opt removed; logDeliveredReflexPointers is the seam.) - RT5 unbounded window: --window-turns is clamped to [1, 64] so a config typo can't reintroduce the re-scan-everything-per-turn cost class. - RT2/RT4 documented + filed: PGLite watch connection monopoly (WATCH_HELP, push-context guide, TODO to route watch via serve IPC); host-resolver suppression contract at ResolveEntitiesFn (TODO for a capability gate). Tests: starvation guard (watch + volunteerContext unit), window clamp floor + ceiling, delivery-side logging (helper writes channel=reflex through the drained sink; bare resolver writes nothing; empty list no-op), IPC wiring test rewired to onDelivered. KEY_FILES.md + push-context.md updated; build:llms run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(context): env-plane window knob works config-less; harden two gateway-state-leak victims Three CI-only check failures, two root causes: 1. windowTurnCount ignored GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS when loadConfig() returned null (no config file AND no DATABASE_URL — a clean CI shard with no brain). loadConfig drops its env→config mapping in that case, so the documented escape hatch silently died and the window fell back to 4 → windowed extraction widened when the test set window=1 → prior-turn entity leaked. Fixed: read the env var directly in windowTurnCount, mirroring reflexEnabled's direct process.env read. This is a real product bug, not just a test artifact — any config-less host using the env hatch was affected. Regression test pins it. 2. sync-cost-preview + doctor-federation-health failed only IN-SHARD: a sibling test configured a non-legacy (ZeroEntropy 1280-d / $0.05) gateway and never reset it. The legacy-embedding preload only restores the OpenAI/1536 default when the gateway slot is EMPTY, so a non-empty foreign config survives into the next file — and a file's beforeAll runs BEFORE the preload's restoring beforeEach, so federation-health built a vector(1280) column and its 1536-d fixture hit CheckExpectedDim. My new test files reshuffled the deterministic file→shard assignment, exposing this latent ordering bug. Hardened both victims to establish the gateway state they assert (sync-cost-preview resets to the unconfigured fallback; federation-health pins legacy 1536 before initSchema) so they're order-independent. Verified against a simulated leaker run before them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(context): use withEnv() in the window env-hatch test (test-isolation guard) The regression test added in 82cc7fff mutated process.env directly, which check:test-isolation (R1) forbids — use the withEnv() helper that restores on exit, same as the rest of this file. Behavior identical; guard green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-06-14 09:32:58 -07:00