mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-28 14:59:47 +00:00
b0f74017d713828d7191e048374652a03e4e0116
150
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
b0f74017d7 |
fix(calibration): resolve owner holder via config (default 'self') (#3077)
* fix(calibration): resolve owner holder via config (default 'self'), fixes #2464 Takeover of #2467 (rebased onto master). consolidate writes owner takes with holder='self' while calibration-profile, calibration CLI/op, think's calibration block, emotional-weight, and doctor's calibration_freshness all defaulted to a hardcoded 'garry' — so getScorecard returned 0 resolved and the calibration profile never built on non-upstream brains. New src/core/owner-holder.ts is the single source of truth: resolveOwnerHolder({override, configValue}) = override > emotional_weight.user_holder config > 'self'. All six call sites route through it; doctor's freshness SQL is parameterized (). Upgrade note: upstream-owner brains with historical holder='garry' profiles should `gbrain config set emotional_weight.user_holder garry` to keep reading them. Co-authored-by: devty <devty@users.noreply.github.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(calibration): replace real-name holder fixture with charlie-example placeholder Privacy iron rule: no real people's names in checked-in code. The sanctioned placeholder mapping uses people/charlie-example. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: devty <devty@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Garry Tan <garrytan@gmail.com> |
||
|
|
d21f34e96d |
fix(search): project email citation metadata (#2873)
Co-authored-by: Amit Agarwal <5302320+amtagrwl@users.noreply.github.com> |
||
|
|
d9eb027bdd |
fix(openclaw): declare gbrain plugin manifest entry (takeover of #2551) (#3185)
Add the OpenClaw-required top-level id to openclaw.plugin.json, export a direct register(api) entrypoint from src/openclaw-context-engine.ts, add a manifest regression test, and document that skillpack harvest must preserve OpenClaw-native manifest fields (id, configSchema, contracts). llms bundles regenerated (bun run build:llms) — no content drift. Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Filip <FilipHarald@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
62e009d192 |
fix(skillopt): emit proposed.md in no-mutate mode (#2635) (#3182)
Takeover of #2719 (fork head; rebased onto origin/master). - writeProposed now writes both best.md (current-best pointer) and proposed.md (stable human-review artifact); returns the proposal path. - Orchestrator reports the real proposed.md path for accepted --no-mutate runs. - Tutorial updated; llms bundles regenerated (no content drift — tutorial is not inlined in the bundle). Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Ziyang Guo <121015044+RerankerGuo@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
d61808d806 |
v0.42.64.0 fix: harden confidential OAuth token revocation (takeover of #3017) (#3032)
* fix(oauth): validate confidential revoke secrets * fix(oauth): harden confidential token revocation * chore: bump version and changelog (v0.42.64.0) Co-Authored-By: OpenAI Codex <noreply@openai.com> --------- Co-authored-by: Robin <rayme@boltdsolutions.com> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: Garry Tan <garrytan@gmail.com> |
||
|
|
11eebc3605 |
fix(conversation): extract iMessage facts with real timestamps (#2756) (#2958)
Co-authored-by: Sameer Bopardikar <203024074+sameerbopardikar@users.noreply.github.com> |
||
|
|
2934c53c1d |
fix(sources): validate --path is a git repo with committed content at registration (#2707) (#2975)
* fix(sources): validate --path is a git repo at registration time (#2707) `sources add --path <dir>` accepted any existing non-git directory with zero validation, deferring the failure to the first `gbrain sync` ("Not inside a git repository: ..."). By the time that surfaces, the source has been silently stale for however long nobody read the sync logs. Add a registration-time check (git-remote.ts:isInsideGitRepo, mirroring sync.ts's discoverGitRoot walk-up so subdir-of-git-repo sources still pass) that rejects an existing-but-non-git --path directory with an actionable error pointing at `git init && git add -A && git commit`. Non-existent paths are unaffected (out of scope — different, pre-existing failure mode) and `--force` opts out for callers who want to register before git-init exists. This is registration-time validation ONLY — it never auto-`git init`s the directory, preserving the consent boundary #2967 established for sync-time self-heal (a --path source is the user's own external directory; gbrain must not mutate it without explicit ask). Also documents the git requirement (docs/guides/multi-source-brains.md), including the "files must be committed, not just present" gotcha and that a stale/unreachable sync anchor already self-heals on plain `gbrain sync` (verified manually against HEAD — no reset-anchor command needed). * fix(sources): require a committed HEAD + shell-quote remediation cmd (codex round 1) Codex review round 1 on #2707 found two real gaps: 1. isInsideGitRepo alone accepts a `git init`ed-but-never-committed directory (rev-parse --show-toplevel succeeds with no HEAD), so registration would still pass a source that fails sync's own "No commits in repo ... Make at least one commit before syncing." Add hasGitCommits (git rev-parse HEAD) as a second required check. 2. The remediation command in the error message interpolated the raw path unquoted — spaces, $(), backticks, etc. would break or, worse, execute unintended shell syntax if pasted. POSIX single-quote it (mirrors src/commands/connect.ts:shellQuote; duplicated locally rather than imported, since commands/ depends on core/ not the reverse). * fix(sources): require tracked content in HEAD, not just a resolvable HEAD (codex round 2) Codex review round 2 P1: hasGitCommits (rev-parse HEAD) accepted a repo with an empty commit (git commit --allow-empty) followed by untracked files — HEAD resolves fine (to git's well-known empty-tree object), so registration passed, but the first sync would "succeed" importing nothing and then silently never notice the untracked files change. The exact same gap applied to an untracked subdirectory of an otherwise- real git repo (monorepo case). Replace hasGitCommits with hasTrackedContent (`git ls-tree HEAD -- .`, non-recursive — one entry is enough, no need to walk the whole subtree). `-C path` + pathspec `.` scopes correctly to both a repo toplevel and a subdirectory-of-a-repo source, and an empty tree lists zero entries where a bare `rev-parse HEAD` would still succeed. Also subsumes the "no commits at all" case hasGitCommits covered (ls-tree on an unborn repo fails the same way), so this is one check instead of two. Updated the error copy and docs/guides/multi-source-brains.md to match what's actually verified now. * fix(sources): O(1)-output tree-emptiness probe, avoid maxBuffer overflow (codex round 3) Codex review round 3 found the round-2 `git ls-tree HEAD -- .` listing buffers the whole (non-recursive) tree — a real repo with ~17-20K directly-tracked entries exceeds execFileSync's default 1 MiB maxBuffer, throws ENOBUFS, and the catch-all incorrectly rejects a perfectly valid registration. Replace the listing with `git rev-parse --verify HEAD:./` (resolves the tree object for `path` specifically, correct for both toplevel and subdirectory sources same as before) compared against git's canonical empty-tree SHA-1 (4b825dc6...) — a fixed ~40-byte read regardless of how many entries the tree has, structurally immune to this class of bug rather than just raising the threshold. Added a 300-file regression test locking this in. Declined a second round-3 finding (P1: reject a tree if ANY untracked file exists anywhere under the path, not just when the tree is entirely empty) — untracked files never being synced is standard, existing git-source behavior throughout this codebase (identical for --url managed clones), not a bug specific to this validation. Enforcing zero-untracked-files at registration would reject ordinary repos with gitignored build output, .DS_Store, editor swapfiles, etc. Out of scope relative to what #2707 actually asks for (a directory with real, committed content that will sync) and how every other git source in this system already behaves. * fix(sources): derive empty-tree OID per repo instead of hardcoding SHA-1 (codex round 4) Codex review round 4 P2, confirmed by directly testing against a `git init --object-format=sha256` repo: the hardcoded SHA-1 empty-tree constant only matches SHA-1 repositories. An empty SHA-256 repo's real empty-tree OID is a different (64-char) hash, so the SHA-1 comparison silently mismatched and let an empty/untracked SHA-256 source through — exactly the case this validation exists to catch. Replace the constant with `emptyTreeOid()`: `git hash-object -t tree --stdin < /dev/null` computed in the target repo's own context, so it returns the correct empty-tree OID for whichever object format that repo actually uses, without gbrain needing to know or care which one. Added gated regression tests (git 2.29+ / --object-format=sha256, test.skipIf on older git) for both the empty-repo-rejected and real-content-registers-fine cases. Converging here (4 review rounds; this is the last outstanding finding from round 4, and round 4 raised only this one issue). * fix(test): --force the incidental non-git second source in #1434 routing test (#2707) CI caught a real regression from this PR's registration-time git validation: test/sync-sole-non-default-routing.test.ts's "2+ non-default sources" case registers a bare mkdtempSync temp dir (no git init) as a second source purely to have 2 sources present — the directory's content was never meant to be exercised, only its existence as a distinct local_path. #2707's new validation correctly rejects that dir at registration time, since nothing else in the test suite told it otherwise. --force is the right fix, not adding unnecessary git-init/commit boilerplate to secondRepo: it documents that this specific registration intentionally doesn't care about git-validity, matching what a real caller opting into the legacy lenient behavior would do. Verified: the specific test (3/3 pass), plus every other test file in the repo using `sources add --path` (sources.test.ts, sources-ops.test.ts already covered by the PR's own commits; repos-alias.test.ts, sync-cost-gate.serial.test.ts — 11/11 pass, no similar fixture gap). typecheck clean, verify 31/31. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
|
8b325041ee |
v0.42.63.0 fix: preserve configured PGLite schema database path (#3016)
* fix(schema): preserve configured PGLite database path * chore: bump version and changelog (v0.42.63.0) Co-Authored-By: OpenAI Codex <noreply@openai.com> --------- Co-authored-by: OpenAI Codex <noreply@openai.com> |
||
|
|
f72de97943 |
feat(sync): --src-subpath + --exclude for monorepo subdir-source support (#753, supersedes #774) (#2942)
* feat(sync): --src-subpath + --exclude for monorepo subdir-source support (#753) Rebased port of PR #774 onto current master. A single git repo can hold N logical sources at subdirectories: git operations run at the discovered repo root (git rev-parse --show-toplevel — worktrees and submodules resolve natively) while file walking, imports, deletes and renames are scoped to the subpath. Passing the subdirectory directly as the repo path (auto-discovery) works through the same code path. Path-containment guards (the point of the feature): - NAV-1/NAV-2: the realpath-resolved scope must live inside the realpath-resolved git root — ../-traversal and symlinked subdirs pointing outside the repo are rejected before any git op. - NAV-1 TOCTOU: per-file realpath re-validation during the incremental import drain and rename reimport; symlink-escape files are recorded as failures (fail-closed — the bookmark cannot advance past an escape). - NAV-4: an --exclude set that filters out every candidate warns loudly. Scoped syncs use git-root-relative slugs + source_path in BOTH the full and incremental paths (runImport gains slugRoot), fixing the original PR's full/incremental slug divergence in the auto-discovery flow. --exclude matches scope-relative paths in both paths; exclusion never deletes previously-imported pages. The full-sync delete reconcile is scope-restricted and relativizes against the slug base so a healthy scoped source can't trip the #2828 mass-delete valve. .gitignore management resolves to the git root at every call site. Preserves all master-side sync work since the original branch: the #2828 mass-delete safety valve, #1794 resumable checkpoints + pinned targets, #1950 stall watchdog, #2335 heartbeat bump, #1970 bookmark reachability, and the git-ls-files walker (#2315/#2462/#2678, whose symlink/cycle hardening is untouched). Co-authored-by: Jeremy Knows <jeremy@veefriends.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: listEverCommittedPaths uses gitContextRoot after #753 root-triple refactor Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Jeremy Knows <jeremy@veefriends.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
f8d11f67a3 |
feat(search): configurable FTS language + reindex command (lands #580/#581/#582) (#2941)
Squashed superset takeover of the FTS-language trilogy by @rafaelreis-r (#580 env-var language for query+write side, #581 migration backfill, #582 gbrain reindex-search-vector), rebased onto current master with the security review's required fixes applied: - Migration renumbered v116 -> v123 (master's v116 was already claimed by code_edges_source_backfill; master is at v122). - Restored the v120/#1647 search_path hardening: all four CREATE OR REPLACE trigger-function bodies (migration handler + reindex command) now carry SET search_path = pg_catalog, public, since CREATE OR REPLACE resets proconfig and would otherwise strip the hardening on upgrade. - reindex-search-vector: shared progress reporter (stderr phases reindex_search_vector.pages/.chunks), id-keyset batched backfill (5000 rows/UPDATE) instead of single whole-table statements, and --json no longer bypasses the --yes/TTY confirmation gate. - Allowlist validation regex + injection tests kept exactly as authored. - Stale v33/v116 comments swept; docs/guides/multi-language-fts.md written (README referenced it but no PR added it); llms bundles regenerated via bun run build:llms. - Trilogy tests quarantined as *.serial.test.ts (env mutation, per check-test-isolation). Verified live on PGLite: fresh init with GBRAIN_FTS_LANGUAGE=portuguese produces portuguese-stemmed vectors with search_path pinned; the reindex command retokenizes an english brain to portuguese in place. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Rafael Reis <rafael.reis@contabilizei.com.br> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
9fe4628d02 |
feat(engine): opt-in Postgres RLS source-scope binding (lands #2387) (#2940)
Takeover of community PR #2387 with the security review's required fixes applied. Original work by @harrisali0101. With GBRAIN_RLS_SCOPE_BINDING=1, source-scoped Postgres read methods wrap their queries in a transaction that binds set_config('app.scopes', $1, true) (federated sourceIds CSV > scalar sourceId > '*') so operator-managed RLS policies can filter rows at the SQL layer — defense-in-depth layer 2 under the mandatory app-layer source filters. Review fixes on top of the original PR: - Flag-off is now a TRUE pass-through: no new per-read transaction wrap (the #1794 PgBouncer pool-exhaustion class). Only the three search methods keep a transaction when off — exactly the sql.begin() + SET LOCAL statement_timeout wrap they already had on master — via the helper's alwaysTransaction option. - Preserved the PR's latent setseed fix: listCorpusSample pins setseed() + SELECT to one connection when seeded (alwaysTransaction gated on opts.seed), so the deterministic path can't split across pooled connections. - Updated the two postgres-engine shape tests to pin the new invariant (search methods route through withScopedReadTransaction with alwaysTransaction; helper owns the sql.begin(); flag-off path is callback(this.sql)). - New behavioral tests (test/postgres-engine-rls-scope.test.ts): flag-off pass-through, flag-off alwaysTransaction, flag-on set_config emission, federated > scalar > '*' precedence, and the CSV as a bound parameter (never interpolated). - Fixed the helper header comment to match the actual branching behavior. - Operator docs in docs/ENGINES.md: env var, policy SQL, the ALTER ROLE ... SET app.scopes='*' default requirement, FORCE ROW LEVEL SECURITY for owner roles, and the honest caveat about unwrapped paths. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Harris <79081645+harrisali0101@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
1833d95896 |
fix(dream,chronicle): synthesize/concepts output family — source scope, output root, retrieval reach, durable provenance, honest judge failures (#1586 #2415 #2163 #2569 #2606) (#2939)
Five verified-open fixes to the dream/synthesize output family: - #1586: thread the cycle's resolved sourceId (cycleSourceId) through runPhaseSynthesize -> SubagentHandlerData.source_id -> subagent tool OperationContext, and stamp collected refs + summary page with the same source, so synthesized pages stop landing in 'default'. - #2415: new config knob dream.synthesize.output_root (default 'wiki', zero behavior change unless set) drives the synthesize prompt slug templates, the patterns reflection lookup + prompt, and remaps the filing-rule allow-list globs. Registered in KNOWN_CONFIG_KEYS. - #2163: synthesize_concepts writes concept pages through importFromContent (put_page's parse->chunk->embed pipeline) instead of bare engine.putPage, so concepts/ pages are chunked + embedded and reachable by retrieval. - #2569: stampDreamProvenance persists dream_generated + dream_cycle_date into pages.frontmatter (JSONB merge via executeRawJsonb) at write time, so generated pages are DB-queryable and put_page write-through can't erase the marker. - #2606: chronicle judge detects stopReason 'length' truncation and no-JSON-array parse failures as distinct skipped reasons (judge_truncated / judge_parse_failed) instead of a false terminal no_events; output cap raised to 4000 and configurable via chronicle.judge_max_tokens. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Garry Tan <garrytan@gmail.com> |
||
|
|
42375bded5 |
fix(sync): stop the sync data-loss family — ops/ prune, DB-only write-through, full-sync gate drift (#2404, #2426, #2607) (#2938)
Three verified-open defects, one family: sync silently destroying or diverging on content it should preserve. #2404 (P0) — 'ops' was hardcoded in PRUNE_DIR_NAMES (a v0.2.0-era carve-out), so any path with an ops segment was 'pruned-dir': committed ops/*.md never imported, and modified ops/* files hit the unsyncableModified delete loop (whose #1433 guard only spared 'metafile'), silently deleting put-created pages like the bundled daily-task-manager's canonical ops/tasks on every sync. Fix: remove 'ops' from the prune list (ordinary user content; the vendor/generated entries stay), and harden the delete loop to also skip 'pruned-dir' — a page under a pruned dir can only exist via a deliberate put_page. #2426 (P0) — write-through content stayed DB-only and was deleted by sync --full. All three compounding bugs fixed: 1. writePageThrough now best-effort commits the artifact (path-limited git commit) on durability-hardened repos, so the post-commit hook can push it; result carries committed?: boolean. 2. scripts/brain-commit-push.sh stages+commits BEFORE any pull — the old fetch+pull-rebase-first order aborted on any dirty tree, so the helper could never commit a MODIFIED page; brain_push's rebase-on-reject already handles an advanced remote. 3. The full-sync delete-reconcile partitions stale pages by git history (listEverCommittedPaths): never-committed source_paths are DB-only write-through — pages are KEPT and re-exported to the working tree instead of soft-deleted. Builds on the #2828 mass-delete valve (covers the below-valve cases). #2607 — the sync --full git ls-files fast path bypassed pruneDir, so a full pass imported (and resurrected soft-deleted) pages under dot-dirs and vendored trees that incremental sync excludes. Fix: isCollectibleForWalker applies the same segment-level pruneDir gate as classifySync, so full and incremental enumeration agree. One regression test per defect (all verified failing against master src): test/sync-ops-pages.serial.test.ts, test/write-through-commit.serial.test.ts, the #2426 helper-order test in test/brain-durability-hook.serial.test.ts, test/sync-reconcile-db-only.serial.test.ts, test/import-git-fastpath-prune.test.ts. Fixes #2404 Fixes #2426 Fixes #2607 Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e1cefd0654 |
fix(subagent): orchestration fix-wave G — configurable timeouts/caps, honest child outcomes, fenced timeline writes (#2937)
Four verified-open issues in the subagent-orchestration family, one PR: - #1594: dream synthesize subagent job/wait timeouts promoted from hardcoded 30/35-min constants to config keys dream.synthesize.subagent_timeout_ms / subagent_wait_timeout_ms. Approach ported from stale PR #1596 (credit @ai920wisco). - #2778: add_timeline_entry joins the subagent brain-tool allowlist, fenced fail-closed by the shared enforceSubagentSlugFence (extracted from put_page's inline check — same trusted-workspace allow-list / wiki/agents/<id>/ namespace policy). The per-turn output cap is now resolveMaxOutputTokens (data.max_tokens → agent.max_output_tokens → 8192, was hardcoded 4096); a max_tokens stop surfaces as stop_reason 'max_tokens' instead of a silent end_turn, and a mid-tool-round cap hit injects a truncation note so the model re-issues the dropped call. - #2782: patterns phase status now reflects the child outcome — non-complete outcome with zero writes → fail (PATTERNS_CHILD_<OUTCOME>), partial writes → warn. Patterns timeouts get the same config-key pair (dream.patterns.subagent_timeout_ms / subagent_wait_timeout_ms). - #2113: facts extraction cap is config facts.extraction_max_tokens (default 4000, was hardcoded 1500); stopReason 'length' is checked, retried once at 2x the cap, and surfaced on stderr instead of silently extracting zero facts. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: ai920wisco <ai920wisco@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
da1bab532a |
fix(import): make checkpoints staging-first — canonical dir identity + self-describing metadata (#2935)
Port of #1731 (diazMelgarejo) onto current master. gbrain import wrote ~/.gbrain/import-checkpoint.json with the caller's raw dir argument, so a checkpoint left behind by an interrupted run (e.g. SIGTERM) could carry "." or a symlinked spelling — an identity that resolves to whatever CWD the next consumer happens to run from. Downstream tooling that treated the checkpoint dir as an owned staging boundary could then act on the wrong directory. - runImport captures the import target ONCE via resolveImportTargetDir (resolve + realpathSync) and threads that canonical value through collection, checkpoint load/save, and resume filtering - checkpoints are self-describing (schema_version: 1, owner: "gbrain", kind: "import"); loadCheckpoint tolerates absent metadata (legacy path-based files) but rejects present-and-wrong metadata and any relative dir - checkpoint contract documented in docs/guides/live-sync.md (llms bundle regenerated) - test/import-resume.test.ts fixture now realpaths its tmpdir so planted checkpoints match the canonicalized dir (macOS /var -> /private/var) Fixes #1728 Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Lawrence Melgarejo <Lawrence@cyre.me> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
fe6838ffac |
docs: add macOS 26.x Tahoe PGLite WASM workaround + native Postgres setup guide (#1671)
PGLite's embedded WASM engine crashes on macOS 26.x (Tahoe) on Apple Silicon during engine initialization. This adds: - A Troubleshooting section in docs/INSTALL.md with step-by-step instructions for using native Homebrew PostgreSQL 17 + pgvector as a workaround - A callout in README.md's Troubleshooting section pointing users to the detailed setup guide Tested on macOS 26.5 (arm64), Bun 1.3.14, gbrain 0.41.29.0, PostgreSQL 17.10 (Homebrew), pgvector 0.8.0. All 102 schema migrations pass. gbrain doctor green. Co-authored-by: Saurav Roy <roysaurav@users.noreply.github.com> |
||
|
|
ff2eb6ff3a |
docs: post-release reference-doc sync for v0.42.59.0 (#2798)
Cross-referenced the v0.42.59.0 five-fix rollup (#2735-#2739) against the reference docs and updated every entry that no longer described current behavior: - KEY_FILES.md: migrate-engine.ts (source-catalog copy + target-aware resume manifest), pglite/postgres bootstrap probe set (timeline_entries.event_page_id), searchTakes/searchTakesVector source scope, think op scope threading through runGather via thinkSourceScopeOpts, new fence-shared.ts entry (escape-aware parseRowCells as escapeFenceCell's inverse). - TESTING.md: one-liners for the three new e2e suites (think-source-isolation-pglite, facts-fence-reconcile-postgres, migrate-engine-sources-postgres) + the new multi-source-bug-class case. - TODOS.md: new v0.42.59.0 follow-ups section (6 items); refreshed the two existing items the wave partially resolved (think gather scope plumbing, #2200 takes_search engine-layer scope). Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a7b0ae80a9 |
v0.42.60.0 chore(release): eleven verified community fixes — changelog + version bump (#2888)
* v0.42.60.0 chore(release): eleven verified community fixes — changelog + version bump Windows full-sync mass-delete fix, gateway tool-loop resume consolidation (fix-wave A), two source-isolation closes, search-cache exclude-policy keying, and six more verified community fixes. Files the take-writes fail-open source fallback and the #2112 doctor hunk as follow-up TODOs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: update reference docs for v0.42.60.0 - KEY_FILES.md: unpin stale KNOBS_HASH_VERSION number in the autocut entry (mode.ts is the single source of truth); document the full-sync reconcile path-separator normalization + mass-delete safety valve (planReconcileDeletes, GBRAIN_ALLOW_MASS_RECONCILE); describe the TTY-gated admin bootstrap token banner (--print-admin-token, env-sourced always hidden) - docs/mcp/DEPLOY.md + docs/tutorials/company-brain.md: bootstrap token is now hidden on non-TTY starts; document GBRAIN_ADMIN_BOOTSTRAP_TOKEN and --print-admin-token for headless deploys Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(docs): regenerate llms bundle after KEY_FILES/deploy-doc sync Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Time AttakcandGitHub
|
68ed7bafa4 |
fix(facts): quarantine ambiguous entity matches (#2723) (#2737)
Bare names resolve only when prefix expansion finds exactly one canonical candidate; ambiguous collisions and low-specificity multi-token fuzzy matches fall through to the guarded holding path instead of confident wrong attribution. Fixes #2723. |
||
|
|
a25209bbb2 |
v0.42.58.0 fix(ai): provider-agnostic gateway — env clobber, base-URL /v1, embedding dims (#1249 #1250 #1292 #2271 #2209) (#2627)
* fix(ai): drop empty-string env values before merge so they can't clobber config keys (#1249) Claude Code injects ANTHROPIC_API_KEY='' to neuter subprocess LLM calls; an unconditional process.env spread let that empty string override a valid config.json key, breaking every gateway op with NO_ANTHROPIC_API_KEY. Filter '' / undefined before the merge; '0' and 'false' are preserved. * fix(ai): normalize native provider base URLs + replace embedding guard with a dims-presence check (#1250, #1292) #1250: createAnthropic/createOpenAI were called with no baseURL, so an env-injected bare host (e.g. ANTHROPIC_BASE_URL without /v1) 404'd. Add a shared resolveNativeBaseUrl and pass a normalized baseURL at all anthropic + openai native sites (google deferred until its suffix is verified). #1292/D6: the user_provided_model_unset guard was structurally unreachable as a no-model check (parseModelId throws on a bare provider) and only ever false-positived for litellm:<model>, silently disabling vector search. Replace it with a real dims-presence check for user-provided/zero-default recipes and delete the dead branch in both consumers. Also stop configureGateway from fabricating a default embedding_dimensions, so 'no dims set' stays honest. * fix(ai): trust user-declared embedding dims for local recipes + litellm /v1 hint (#2271, #2209) #2271: a new trust_custom_dims flag adds a passthrough tier so ollama / llama-server / litellm accept a user-supplied --embedding-dimensions instead of being hard-rejected. Fail-closed for fixed-dim providers (openai/voyage/ zeroentropy) and excludes openrouter (declares dims_options). Register modern ollama embed model names. #2209: litellm setup_hint now states the /v1 path convention and the docs pointer is corrected to docs/integrations/embedding-providers.md. * docs+test(ai): KEY_FILES current-state for provider-agnostic gateway + embed-preflight dims-unset test (#1249, #1250, #1292) * fix(ai): point user_provided_dims_unset remediation at 'gbrain init' (config set rejects it) + coverage Pre-landing adversarial review (P1): the new dims-unset guard told users to run 'gbrain config set embedding_dimensions <N>', which config.ts hard-rejects (it's a schema-sizing field). Both consumer messages now point at 'gbrain init --embedding-dimensions'. Adds: pgvector-cap-still-fires regression for the trust_custom_dims passthrough, and a configureGateway backfill-invariant test. * chore: bump version and changelog (v0.42.57.0) Provider-agnostic plumbing wave: #1249 empty-env clobber, #1250 native baseURL normalization, #1292 embedding dims-presence guard, #2271 trust_custom_dims passthrough, #2209 litellm /v1 hint. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: sync embedding-providers guide for provider-agnostic gateway wave (v0.42.57.0) Post-ship doc drift fix for the v0.42.57.0 AI-gateway wave: - LiteLLM section now names the /v1 base-URL convention (#2209). - Ollama section lists the newly-registered modern embedders qwen3-embed-8b + snowflake-arctic-embed-l-v2, and notes dims-trust for local recipes (#2271). - llama-server section notes gbrain trusts the user-declared dimension (#2271). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: post-ship doc sweep for v0.42.57.0 provider-agnostic gateway wave - KEY_FILES.md types.ts entry: document EmbeddingTouchpoint.trust_custom_dims (#2271 passthrough tier, runs after dims_options + Matryoshka allowlists) - ENGINES.md: embedding design-choice note now names the provider-agnostic gateway delegation instead of the stale OpenAI-only parenthetical - embedding-providers.md: drop an exact-duplicate doctor-8c paragraph - llms-full.txt regenerated (ENGINES.md is inlined in the bundle) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: apply codex doc-review findings for v0.42.57.0 (base-URL env note, litellm multimodal) - embedding-providers.md OpenAI section: document OPENAI_BASE_URL / ANTHROPIC_BASE_URL bare-host /v1 normalization (#1250 user-facing surface) - TL;DR table: litellm multimodal is backend-permitting (recipe declares supports_multimodal: true, routed via the openai-compat multimodal path), not "no" Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: pin engine-find-trajectory schema to 1536 + stop gateway-state leaks across shard files CI shard 5 failed 7 findTrajectory tests with 'expected 1280 dimensions, not 1536': engine-find-trajectory hardcodes 1536-d vectors but sizes its schema from AMBIENT gateway state in beforeAll — which runs before the legacy-embedding-preload's per-test 1536 restore. A preceding file that ends with a dimensionless configureGateway (facts-extract-silent-no-op) or a bare resetGateway poisons the next fresh initSchema down to 1280-d columns. The new test files in this PR reshuffled shard bin-packing and exposed the trap. - engine-find-trajectory: pin OpenAI/1536 explicitly before initSchema (the pattern bunfig's preload documents) — deterministic regardless of neighbors - facts-extract-silent-no-op, diagnose-embedding-dims, embed-preflight: restore the legacy 1536 pin in afterAll instead of ending reset/dimensionless Reproduced: synthetic dimensionless-gateway file + old victim = the exact 7 CI failures; with the pin = 0. Verified in-process pair runs both orders. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
058f448b9a |
v0.42.57.0 fix(pglite): incident — never steal a live data-dir lock + corrupted-store recovery hint (#2348) (#2400)
* fix(pglite): never steal the data-dir lock from a live holder (#2348) A busy `gbrain dream`/`embed` holder whose 30s heartbeat lapsed (the JS event loop is blocked during long synchronous WASM imports/CHECKPOINTs) used to get its lock reaped past the steal-grace window. PGLite/WASM is strictly single-writer, so a second OS process then opened the same data dir and corrupted the catalog + pgvector extension state (58P01 / internal_load_library / "type vector does not exist"), recoverable only by wipe+restore. Reap ONLY a dead PID; a live holder is never stolen — a wedged-but-alive or PID-reused holder makes the acquire time out with a message naming the PID. Removes the GBRAIN_PGLITE_LOCK_STEAL_GRACE_SECONDS knob (no longer meaningful). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pglite): point a corrupted store at reinit-pglite recovery (#2348) classifyPgliteInitError gains a `corrupt` verdict for the 58P01 / internal_load_library / "vector does not exist" / "content_chunks does not exist" signature (beating the generic wasm-runtime match), so an already-damaged store gets actionable recovery (gbrain reinit-pglite / restore a backup) instead of the wrong macOS-WASM hint. Updates KEY_FILES.md to current lock behavior. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * v0.42.55.0 fix(pglite): incident — never steal a live lock + corrupted-store recovery hint (#2348) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * v0.42.56.0 chore: re-bump past #2399 version collision + refresh ownerToken comment #2399 (security wave) claimed 0.42.55.0; take the next slot. Also updates the LockHandle.ownerToken JSDoc to current #2348 behavior (live holders are never reaped, so reap-then-reacquire is dead-holder-only; token guard stays as defense-in-depth). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
646179047a |
v0.42.56.0 feat(chronicle): Life Chronicle — temporal timeline + thought diary + bi-temporal per-entity ontology (#2390) (#2533)
* feat(chronicle): register event + diary page types (#2390) Life Chronicle Phase A.1. Adds `event` (timeline atom) and `diary` (first-person interiority) as temporal-primitive page types under life/events/ and life/diary/, extractable:false — registered in ALL_PAGE_TYPES, both base schema packs, and the inferType prefix table, with parity fixtures. Also lands the chronicle read result types (ChronicleTimelineRow/ChronicleTimelineOpts/LastSeenResult) consumed by Phase A.2. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): event_page_id timeline projection + day/since/last-seen reads (#2390) Life Chronicle Phase A.2. Adds a nullable event_page_id FK to timeline_entries (migration v120; mirrored in schema.sql, pglite-schema, schema-embedded) so a type:event page projects ONE date-index row keyed to its depth page; a partial UNIQUE(event_page_id, date) makes re-extraction with a changed summary an update, not a duplicate. Dual-engine getTimelineForDate / getSince / getLastSeen filter the depth page on deleted_at, hide soft-deleted event projections at READ time (not just doctor), order by event effective_date for intra-day sequence, and honor source scope (sourceIds[] > sourceId). Ops surface as `gbrain day <date> [--week]`, `gbrain since <date> [--kind]`, `gbrain last-seen <entity>`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): auto-emit extractor — backstop + chronicle_extract job (#2390) Life Chronicle Phase A.3. A put_page backstop (gated on status==='imported', the auto-link/timeline trust gate, and the default-OFF auto_chronicle flag; diary + event pages never eligible) enqueues a chronicle_extract minion job. The job runs the extractor off the write path: deterministic when/who, an injectable LLM judge (default = chat gateway; no-op when no gateway), an all-or-nothing parse barrier (a malformed proposal writes NOTHING), then content-addressed event pages + a timeline projection via the new dual-engine upsertEventProjection (idempotent — re-run yields one event + one projection). New: src/core/chronicle/{eligibility,config,extract-events,backstop}.ts, engine.upsertEventProjection (both engines), the jobs.ts handler + a 10-min timeout. 14 unit tests (eligibility, idempotency, parse barrier, backstop gating + enqueue) green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): quick-capture for diary + manual events (#2390) Life Chronicle Phase A.5. `gbrain capture` now routes the default slug by type (diary → life/diary/, event → life/events/, else inbox/) and accepts --who/--what/--where/--kind/--depth to assemble the `event:` frontmatter block for `--type event`. User-declared event keys win per-key over the flags. Goes through the existing put_page → write-through → embed path. 6 new unit tests; existing capture tests stay green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): bi-temporal per-entity ontology on the facts table (#2390) Life Chronicle Phase B.10 — the feature's differentiator. Rather than a parallel store, the open-world per-entity ontology EXTENDS the existing `facts` table (eng-review G1): migration v121 adds dimension/value/value_hash /dim_status columns + a deterministic partial-UNIQUE dedup key + an asof read index. facts already supplies bi-temporal validity, supersession (superseded_by), visibility (remote redaction), confidence, provenance, and embedding — all inherited. Dual-engine methods: mergeOntologyFact (deterministic value_hash dedup → idempotent retry; same value corroborates; a different value forward-closes the prior row's valid_until + superseded_by; a BACKDATED conflicting value is recorded WITHOUT rewriting, surfaced by findOntologyConflicts), getOntology with `--asof` valid-time travel (expired_at + status + validity-window in the predicate so quarantined/superseded never leak), discoverOntologyDimensions, findOntologyConflicts. Novel/LLM-proposed dimensions quarantine; a seed lexicon canonicalizes name drift (job_role → role). 9 unit tests cover the full lifecycle; typecheck pins both engines to the interface. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): ontology ops — get/add/dimensions/contradictions (#2390) Life Chronicle Phase B.11. Exposes the bi-temporal ontology over CLI + MCP (contract-first, auto-generated): `gbrain ontology <entity> [--asof]`, `gbrain ontology-add <entity> <dim> <value>`, `gbrain ontology-dimensions` (meta-ontology rollup), `gbrain ontology-contradictions`. All reads route through sourceScopeOpts. Privacy: ontology_get redacts diary-sourced observations (source under life/diary/) for untrusted (remote) callers. 3 op tests (incl. the remote-redaction path); 47 op-registry/tool-def/description tests stay green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): agent-context loader — volunteer_chronicle (#2390) Life Chronicle Phase B.12. `loadChronicleContext` hands an agent the recent timeline (last N days) + the validity-resolved current ontology for the entities in play, in one zero-LLM payload, so it orients before acting — the exact gap behind fumbled chronology. Pure composition over getSince + getOntology (no new SQL). Exposed as the `volunteer_chronicle` read op (`gbrain orient [--days] [--entities a,b]`); diary-sourced ontology is redacted for remote callers. 2 loader tests + op-registry green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): backfill op — sweep existing meetings into events (#2390) Life Chronicle Phase A.8. `chronicle_backfill` (local-only admin op; `gbrain chronicle-backfill [--since] [--limit] [--dry-run]`) lists existing meeting/conversation/calendar pages (source-scoped via listPages), filters through the chronicle eligibility predicate, and enqueues one chronicle_extract job per eligible page so existing brains populate the timeline. --dry-run counts only; per-page enqueue failures are surfaced in `errors`, never swallowed. 2 op tests (dry-run count + enqueue). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): delight — on-this-day + narrative rendering (#2390) Life Chronicle Phase A.6 (delight). Dual-engine getOnThisDay (events from the same month-day in prior years; `gbrain on-this-day [--date]`) reusing the chronicle JOIN shape (deleted-event hiding + source scope). A pure renderTimelineNarrative turns timeline rows into prose; `gbrain day --narrative` returns it alongside the events. 5 tests. (Coverage gap-detection ships with the advisor collectors next.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): proactive advisor collector (#2390) Life Chronicle Phase A.7. A brain-state advisor collector surfaces two proactive signals in `gbrain advisor`: unresolved ontology conflicts (warn, → `gbrain ontology-contradictions`) and recent meetings with no timeline coverage (info, → `gbrain chronicle-backfill`). Advisory-only (no dispatch_id); runs over MCP too (not workspace-dependent); tolerant of pre-migration brains. 3 tests + advisor-op-gate green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): doctor chronicle_projection_health probe (#2390) Life Chronicle Phase B.13. An always-run doctor check (keyed off the event_page_id schema, NOT a migration verify-hook) counts timeline projections whose event page is soft-deleted — hidden at read time, surfaced here as a cleanup backlog (`gbrain integrity auto`). Tolerant of pre-migration brains. 1 detection test. (auto_chronicle / chronicle.tz flags already work via getConfig defaults; their docs land with document-release at ship.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): E1 temporal recall — chronicle-type boost on temporal queries (#2390) Life Chronicle Phase A.4 (E1, ambient temporality). Rather than a separate RRF arm (which needs chunk hydration + risks the fusion path), E1 is a bounded post-fusion boost: applyChronicleTypeBoost lifts `event`/`diary` results on temporal queries, wired INSIDE runPostFusionStages' `recency !== 'off'` branch so it fires ONLY on temporal intent — non-temporal search is bit-for-bit unchanged (proven by 110 passing search-path tests). Bounded ([1.0,1.25]) + floor-gated like the other metadata stages; attribution via `chronicle_boost`. 3 unit tests. (Empirical precision/negative measurement lands with the eval.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): feature eval — gbrain eval chronicle (PRIMARY proof) (#2390) Life Chronicle Phase A.9, the North-Star proof. A deterministic, CI-safe eval (brings its own in-memory PGLite; no LLM, no gateway) builds a synthetic month corpus with a known gold chronology + a planted ontology supersession + a planted conflict, then scores the chronicle layer on six gold tasks: day reconstruction (intra-day order), last-seen exact date, ontology supersession, --asof valid-time travel, contradiction surfacing, and source isolation. `gbrain eval chronicle [--json]` exits 0 iff all pass — currently 6/6. The OFF baseline (raw meeting pages) structurally can't order intra-day events or time-travel ontology; the ON path does. (The live-LLM OFF-vs-ON agent arm + LongMemEval temporal slice are a follow-up; this deterministic bar gates CI.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(chronicle): pre-landing review — conflict validity, parse-barrier date, remote conflict redaction (#2390) Three bugs caught by the codex pre-landing review on the diff: 1. findOntologyConflicts ignored valid_until, so a normal forward supersession (founder→advisor from two sources) falsely reported as a live conflict. Now restricted to currently-open rows (valid_until IS NULL) in both engines. 2. The extractor parse barrier accepted any when-string >= 4 chars; a non-date value slipped past, wrote the event page, then threw on the projection's ::date cast (partial write). isValidProposal now requires a real parseable date. 3. The ontology_conflicts op had no remote diary redaction (ontology_get did); remote callers now get diary-sourced values filtered, and conflicts that lose their disagreement after redaction are dropped. Three regression tests added; 29 chronicle tests + eval 6/6 green; typecheck clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * v0.42.56.0 feat(chronicle): Life Chronicle — temporal timeline + diary + bi-temporal ontology (#2390) Bump VERSION + package.json to 0.42.56.0 and add the CHANGELOG entry for the Life Chronicle feature (#2390, closes duplicate #2388). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(chronicle): register #2390 surfaces with the five CI guard suites (#2390) CI caught five guard tests that pin registration invariants my diff tripped: - schema-bootstrap-coverage: the four v122 facts ontology columns join COLUMN_EXEMPTIONS (facts is migration-created; the partial indexes live inside v122; every reader filters dimension IS NOT NULL — same precedent as facts.claim_metric et al). - no-valid-until-write (R8): both engines' mergeOntologyFact forward- supersession is a deliberate, documented valid_until write authority (engine-layer, dimension IS NOT NULL only; the contradiction probe still never mutates). - doctor-categories: chronicle_projection_health registered under BRAIN_CHECK_NAMES (same class as child_table_orphans). - checkTypeProliferation: the test is now threshold-relative (computes declared from the active pack, seeds declared+6) so base-pack growth can't silently move the fixed threshold again. - schema-cli: gbrain-base page-type count 25 → 27 (event + diary), with assertions on both new types. All 41 guard tests green; typecheck clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: file Life Chronicle follow-up TODOs (v0.42.56.0, #2390) Eight deferred items from the CEO/eng review decisions (auto-emit default-flip fast-follow, live eval arm + LongMemEval slice, passive diary consent, interval-splitting, federation, place-as-entity, meta-ontology dashboard, materialized daily pages), each with decision provenance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(architecture): KEY_FILES entry for the Life Chronicle module (#2390) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
814258dda6 |
v0.42.53.0 fix(sync,db): #2339 op_checkpoints jsonb double-encode + bug-class sweep + CI guard (#2375)
* fix(sync): op_checkpoints pin write double-encodes jsonb — every sync aborts (#2339) recordCompleted bound JSON.stringify(array) to a $3::jsonb param via postgres.js .unsafe(), double-encoding it into a jsonb string scalar that violates the v119 op_checkpoints_completed_keys_array CHECK — aborting every multi-source sync on real Postgres at the first checkpoint write. PGLite parses the string silently, which is why unit tests stayed green and it shipped. Cast through $3::text::jsonb so the text->jsonb cast parses a genuine array. Adds a DATABASE_URL-gated parity test + a dedicated Postgres CI job so the guard can never silently skip. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(db): sweep positional jsonb double-encode sites + AST CI guard (#2324) Every executeRaw/.unsafe site that bound JSON.stringify(x) to a bare positional jsonb cast double-encodes on real Postgres (same class as #2339). Sweep them all to the text::jsonb form across query-cache, sources-ops, llm-base, calibration-profile, impact-capture, subagent, receipt-write, traversal-cache, symbol-resolver, and the agent/sources commands. Adds scripts/check-jsonb-params.mjs (AST-lite scanner for the positional form the legacy template grep misses, incl. generic-typed calls), wired into check-jsonb-pattern.sh, with a self-test. PGLite's native db.query is not scanned — it parses text to jsonb natively, so the bug can't occur there. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(search,eval): alias-hop injected results carry page_id (contradiction-probe crash) applyAliasHop injected synthetic SearchResults without page_id (the `as SearchResult` cast hid the missing field), so listActiveTakesForPages bound undefined/NaN into ANY($1::int[]) and crashed the whole contradiction probe on real Postgres. Stamp page_id=page.id at the injection site and add a finite-id filter in generateIntraPagePairs as a defensive backstop (mirrors hybrid.ts:63). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(engines): positional jsonb binding rule (text::jsonb vs the double-encode trap) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * v0.42.53.0 fix(sync,db): #2339 op_checkpoints jsonb double-encode + bug-class sweep + CI guard Bumps VERSION + package.json to 0.42.53.0, adds the CHANGELOG entry, and regenerates llms-full.txt. Ships the #2339 sync-abort hotfix, the repo-wide positional jsonb double-encode sweep, the alias-hop contradiction-probe crash fix, and the new positional-form CI guard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: post-ship sync — jsonb invariant now covers the positional form + new guard CLAUDE.md JSONB invariant + KEY_FILES (sql-query, check-jsonb-pattern, op-checkpoint) now describe the #2339 positional double-encode class, the $N::text::jsonb fix, and the new check-jsonb-params.mjs guard. Regenerates llms-full.txt. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
bb2e88c42a |
v0.42.52.0 fix(reliability): autopilot dead-job storm + supervisor wedge + sync/status/minion reliability (#2194 #2227 #1994 #1737 #1738 #1950 #1984) (#2287)
* test(supervisor): pin LOCK_HELD fence-exit is never counted as a crash (#2227) A duplicate supervisor loses the queue-scoped DB singleton lock (#1849) and exits LOCK_HELD before spawning a worker or emitting 'started'. summarizeCrashes counts only worker_exited, so the fence path is structurally uncountable. Pin it so a future refactor that logs worker_exited on the fence path fails here instead of silently re-introducing the crash-budget breaker-trip loop. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autopilot): per-source cycle binds FS phases to source.local_path, not global repo (#2194 #2227) A per-source autopilot-cycle inherited the global sync.repo_path as brainDir while stamping DB freshness for source_id — mixed scope. FS phases (sync/lint/extract) ran against the wrong tree, so the failure-cooldown and freshness gates would attribute work to the wrong source. Resolve the source's local_path in the handler (reuse the archive-recheck SELECT) and bind brainDir to it; a pure-DB source gets null (FS phases skip) instead of falling through to the global checkout. Legacy no-source dispatch keeps the global repoPath. Prerequisite for the cooldown/split commits (codex outside-voice #8). Resolves TODOS:634. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(supervisor): detect a live supervisor via the DB lock under split $HOME (#2227) jobs supervisor status + doctor read the HOME-derived pidfile, so a supervisor started under a different $HOME (keeper=/root vs ops=/data) read as 'not running' while healthy — the false signal that drives an operator to spawn a duplicate. Both surfaces now fall back to the queue-scoped DB singleton lock (#1849), the HOME-independent authority, when the pidfile shows nothing. New isLockHolderLive keys on lock freshness (ttl + heartbeat steal-grace), never process.kill, so PID reuse can't false-positive (pid-liveness-alone-pid-reuse). Status surfaces the holder host/pid + recorded concurrency/max-rss from the latest started event. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(supervisor): degraded retry instead of permanent give-up on crash storm (#1994 #2227) max_crashes_exceeded gave up forever, so a transient DB-pooler blip that tripped the soft budget wedged the queue until a human restart (#2227's breaker-trips tail). Crossing the soft budget now enters degraded mode: keep respawning with capped exponential backoff (60s cap — a paced retry, not a hot loop) and emit a loud crash_budget_degraded health_warn. The existing stable-run reset clears the count once a respawn survives >5min, so a recovered DB self-heals. Permanent give-up fires only at a much-higher hard ceiling (maxCrashes × 10), tunable/disablable via GBRAIN_SUPERVISOR_HARD_STOP_CRASHES (0 = never). Resolves TODOS:92. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autopilot): clamp fan-out to worker concurrency + doctor warning (#2194) Fan-out resolved to 4 (Postgres) regardless of worker --concurrency, so surplus cycles queued behind the worker and raced the stalled-sweeper. Two fixes for the same mismatch: - resolveEffectiveFanoutMax clamps to max(1, concurrency-1) (reserve a slot), gated on a LIVE DB-lock holder so a stale started-audit row can't shrink throughput (codex #9/D5); no live holder → unknown → unclamped base. Escape hatch autopilot.fanout_clamp_to_concurrency. - doctor's autopilot_fanout_concurrency check warns when fan-out exceeds effective slots — the misconfig was silent before. Advisory (started-event concurrency), wired into both doctor surfaces. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autopilot): per-source failure cooldown — break the dead-job storm (#2194) Only SUCCESS gated dispatch, so a source whose cycle kept failing/timing-out re-fanned-out every 5-min tick forever (200+ dead jobs/24h). Now a failed source backs off with bounded exponential cooldown (10→120min). Read at DISPATCH from minion_jobs dead/failed rows (timeouts/RSS-kills dead-letter via SQL and never run handler code, so a write-only hook would miss them) AND re-checked at CLAIM time in the handler (codex #5: already-queued/retrying jobs). A success clears it (codex #7); null-source rows excluded (codex #6); engine-parity via executeRaw. Disable with autopilot.failure_cooldown_min=0. Fail-open if config/history reads error. Surfaced via fanout_cooldown_skipped + the fanout summary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autopilot): split the cycle — per-source phases + one global-maintenance job (#2194 #2227) N per-source cycles each ran the brain-wide global phases (embed-all/orphans/ purge/…) concurrently, thrashing the same rows and taking the worker 4→10GB in <60s → RSS-kill → orphaned stalls. Split them: per-source jobs now run only source-scoped (+ mixed) phases and stamp last_source_cycle_at; a new autopilot-global-maintenance job runs the global phases ONCE per window (idempotency_key + maxWaiting:1 = structural single-flight) and stamps autopilot.last_global_at. This is the codex-endorsed design that replaced the rejected skip-and-stamp-fresh approach (codex #1/#2): no freshness poisoning, no starvation — global work always runs as its own job, never marked done when it wasn't. PHASE_SCOPE is now a runtime partition (GLOBAL ∪ NON_GLOBAL == ALL). last_full_cycle_at still written for doctor/legacy (no longer a global gate). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(doctor): guard nullable engine in supervisor DB-lock fallback (#2227) Follow-up to the supervisor-visibility commit: doctor's engine binding is BrainEngine | null, so the inspectLock fallback must guard on a non-null engine (tsc TS2345). No behavior change — a null engine simply skips the DB-lock probe and falls back to the pidfile reading, as before. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(doctor): categorize autopilot_fanout_concurrency check as ops (#2194) Follow-up to the fan-out/concurrency commit: the doctor-categories drift guard requires every check name in doctor.ts to belong to exactly one category set. Add the new autopilot_fanout_concurrency check to OPS_CHECK_NAMES (infrastructure liveness, alongside wedged_queue/supervisor). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: update KEY_FILES for the autopilot cycle split + supervisor degraded-retry (#2194 #2227) Post-ship document-release: refresh the KEY_FILES current-state entries that drifted — cycle.ts (GLOBAL/NON_GLOBAL phase split + last_source_cycle_at / autopilot.last_global_at), jobs.ts (per-source local_path brainDir, claim-time cooldown, autopilot-global-maintenance handler), supervisor.ts + child-worker (degraded retry instead of permanent give-up; hard ceiling), db-lock.ts (isLockHolderLive), handler-timeouts (new handler). Regenerated llms bundle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(minions): handleTimeouts counts the timed-out run as a spent attempt (#1737) The per-job timeout_at dead-letter (handleTimeouts) set status='dead' without incrementing attempts_made, unlike the wall-clock and stall dead-letter siblings. It is the FIRST killer to fire for the long-lane handlers (subagent / embed-backfill / autopilot-cycle) because timeout_ms is stamped at submit, so a timed-out long job reported `attempts: 0/N (started: N)`. Mirror the siblings with attempts_made + 1 (terminal, no retry). Safe against double-count: the worker sweep runs handleStalled -> handleTimeouts -> handleWallClockTimeouts sequentially and awaited, each guarded on status='active', so the first to dead-letter excludes the row from the rest. Regression assertions added (test/minions.test.ts + e2e/minions-resilience.test.ts) so the increment can't be silently dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agent): recognize trailing switches in `agent run`, keep prompts freeform (#1738) parseRunFlags() broke flag parsing at the first positional token, so any flag after the prompt (`gbrain agent run "do X" --detach`) was swallowed into the prompt string and silently ignored. Now the no-value switches --detach/--follow/ --no-follow are hoisted when they trail the prompt, while everything else stays verbatim: an unknown --word is treated as prompt text (no "unknown flag" throw), a --switch mid-prompt is preserved, and `--` suppresses hoisting entirely for a literal escape. Value-flags now reject a missing or flag-shaped value (and --max-turns/--timeout-ms a non-number) instead of capturing undefined/NaN. Contract change: a prompt that starts with or trails an unguarded --word no longer errors; a literal trailing --detach needs `--`. Help text updated; tests revised + extended (test/agent-cli.test.ts). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sync): honest live-sync status + progress-aware stall-abort (#1950) Finishes the #2255 honest-freshness story for two gaps it left. (a) `gbrain sources status` printed "idle" while a sync proc held the per-source lock (the reported bug). New shared liveSyncStatus() helper in db-lock.ts reads the SAME live-lock signal `gbrain doctor` uses; runStatus now shows "running" (BACKFILL column + a sync_running field in --json) and suppresses the misleading "never synced" warning while a sync is live. One helper, so the surfaces can't drift (doctor/status retrofit tracked as a follow-up). (b) A sync wedged-but-alive kept refreshing its lock heartbeat (it fires on its own timer) and hadn't hit the wall-clock deadline, so only a manual pkill freed it. New in-band stall watchdog keys off FORWARD IMPORT PROGRESS (progress.tick), not the heartbeat: if no file completes for GBRAIN_SYNC_STALL_ABORT_SECONDS (default 900s), it aborts via a controller composed into opts.signal, so the drain returns partial() (last_commit unchanged, next run resumes from the checkpoint) and withRefreshingLock releases the lock. Limits, documented in code: a single file slower than the window trips it; a fully starved event loop won't fire the timer (the wall-clock hard deadline is that backstop). Tests: liveSyncStatus (live/expired/none/per-source) in db-lock-inspect; the resolveStallAbortSeconds env matrix in sync-hard-deadline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(status): version field + per-section --deadline-ms budget (#1984) `gbrain status` had no version in its JSON envelope and could hang on a slow connection with no way to get a partial answer. Two additions: - version: the StatusReport JSON now carries the local gbrain CLI version so a poller can pin behavior to a build. Thin-client also surfaces remote_version (the brain server's version), and the get_status_snapshot MCP op reports its version for that parity. - --deadline-ms=N / --fast: a shared wall-clock budget. Each section is raced against the REMAINING budget via Promise.race (NOT process-watchdog, which SIGKILLs and can't return partial output), so one slow/hung section can't strand the snapshot — it's marked stale and the rest still return. The envelope gains partial:true + stale_sections[]; exit code stays 0 (a snapshot was produced). Invalid --deadline-ms → exit 2. Tests: parseDeadlineFlag + withSectionDeadline (hermetic), the usage-error exit, version presence in the PGLite envelope, and the op's version key. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sync): report stall_timeout distinctly + document in-flight limit (#1950) Pre-landing review (codex + adversarial): the stall watchdog aborted opts.signal but the per-iteration abort checks returned partial('timeout'), collapsing a wedge-reap into a user --timeout/SIGINT so JSON consumers couldn't tell them apart. Add a 'stall_timeout' reason (set via a stallAborted flag) on the three import-loop abort sites; deletes/renames-phase and checkpoint sites stay 'timeout'. Sharpen the watchdog comment: the abort is observed BETWEEN files, so a hang inside a single importFile is not interrupted until it returns (TODO: thread a cancellation signal through importFile). * fix(agent): `--` escape suppresses trailing-switch hoisting anywhere (#1738) Pre-landing review: the leading-flag loop breaks at the first positional, so the `escaped` flag only fired for a leading `--`. A `--` placed after a positional left trailing-switch hoisting active, so `agent run note -- body --detach` silently detached and dropped the `--` as junk. Suppress hoisting whenever a literal `--` appears in the prompt. Regression test added. * fix(status): deadline-ms usage-error + scoped stale_sections + cancel losing remote call (#1984) Pre-landing review (codex): (1) bare `--deadline-ms` with no value silently fell through to no-budget/--fast instead of a usage error; (2) thin-client timeout reported both sync+cycle stale even under `--section sync`, naming a section the caller excluded (local path was already correct); (3) the section race abandoned the remote promise locally but didn't cancel the in-flight MCP call — pass the budget as timeoutMs so the losing side actually cancels. Regression test added. * v0.42.52.0 fix(reliability): autopilot dead-job storm + supervisor wedge + sync/status/minion reliability (#2194 #2227 #1994 #1737 #1738 #1950 #1984) Bundles the already-reviewed autopilot/supervisor stabilization (#2194 #2227 #1994: cycle split, per-source failure cooldown, fan-out clamp, degraded supervisor retry, DB-lock live-supervisor detection) with four operational fixes: minion timeout attempt-accounting (#1737), agent-run trailing-flag parsing (#1738), honest live-sync sources status + progress-aware stall watchdog (#1950), and status version + --deadline-ms partial result (#1984). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: document GBRAIN_SYNC_STALL_ABORT_SECONDS env knob (#1950) Post-ship doc sync (/document-release): add the sync stall watchdog env var to the CLAUDE.md sync-tuning table (Five → Six knobs) + regenerate the llms bundle. * test: quarantine #2249 fanout tests as *.serial (R1 env-isolation) (#2194) The cherry-picked autopilot-fanout-clamp + doctor-autopilot-fanout-concurrency tests mutate process.env.GBRAIN_AUDIT_DIR in beforeEach/afterEach, which the check:test-isolation R1 lint flags (parallel shards load multiple files per process). Rename to *.serial.test.ts (sanctioned quarantine — they run under --max-concurrency=1) instead of restructuring the reviewed test bodies. No logic change; both files stay green (9 tests). Fixes the failing verify CI check. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
9bf96db807 |
v0.42.51.0 fix(sync): contention-free clock + checkpoint integrity + honest sync freshness (#2255)
* fix(sync): contention-free page-generation clock — sequence swap The page-generation clock backed the query-cache Layer-1 bookmark via a FOR EACH STATEMENT trigger running `UPDATE page_generation_clock SET value=value+1 WHERE id=1`. That took a transaction-length RowExclusiveLock on one tuple, so every concurrent page writer serialized on the prior writer's COMMIT — sync ran at ~0.8 cores regardless of worker count. Swap to a SEQUENCE bumped by nextval() (a microsecond LWLock, never a row lock). The clock's only contract is monotonic advancement on any page INSERT/UPDATE/DELETE; last_value is non-transactional, so rolled-back or concurrent-uncommitted writers only OVER-invalidate the cache (lose a hit), never serve stale. - migration v118: CREATE SEQUENCE + load-bearing 2-arg setval (is_called= true, floor 1, seeded >= old clock and MAX(generation)) + repoint the trigger function body + DELETE query_cache so no old-clock bookmark survives the swap. v107 left immutable. - query-cache-gate.ts: 3 readers -> SELECT last_value FROM page_generation_clock_seq. - schema.sql + pglite-schema.ts (+ regenerated schema-embedded.ts) ship the sequence on fresh install; table + trigger names retained. - tests: clockValue reads last_value; mechanism proof (trigger fn uses nextval not the row UPDATE); rollback-advances-clock safety pin; real PGLite sequence round-trip (is_called gotcha); shape test requires _seq. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sync): op_checkpoints array-shape guard — CHECK + repair + defensive loader completed_keys is JSONB and the checkpoint loader runs jsonb_array_elements_text over it. A non-array (scalar) value makes that throw "cannot extract elements from a scalar", which takes down the whole UNION load — including the valid op_checkpoint_paths child rows — and loses all checkpoint progress for that key. No current writer produces a scalar, but an older binary / external script / future bug could. Make the corruption class structurally impossible and self-healing: - migration v119: LOCK TABLE (so an out-of-band scalar can't land between repair and constrain; no-op on single-connection PGLite), repair any pre-existing scalar to '[]' (op_checkpoint_paths child rows are the append-only source of truth, so the reset loses nothing), then add the named CHECK (jsonb_typeof(completed_keys) = 'array') via a pg_constraint IF NOT EXISTS guard. A DB-enforced always-on guard — the correct pattern vs a migration verify-hook, which never runs on already-stamped brains. - schema.sql + pglite-schema.ts (+ regenerated schema-embedded.ts) ship the same NAMED inline CHECK so fresh installs match migrated brains and v119 skips the duplicate. - op-checkpoint.ts loader: gate the legacy arm on jsonb_typeof = 'array' so a scalar parent is skipped (children still load) instead of throwing the whole union, and log a specific corruption warning when one is seen. - tests: CHECK rejects a scalar (exactly one constraint, no blob+migration dupe); loader survives a scalar parent and returns the children; v119 repair converts a scalar to '[]'. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(doctor): report actively-running sync via live lock, not stale freshness A slow source that makes partial progress every cycle but never fully completes used to read as permanently "stale" / "never synced" because last_sync_at only advances on a full successful sync. The naive fix (treat recent checkpoint banking as "in progress") is unsafe: a blocked sync banks the good files then writes no anchor, so banking can't tell in-progress from wedged. Use the only honest signal: a LIVE, non-expired per-source sync lock (inspectLock + syncLockId against gbrain_cycle_locks). Every non-skipLock sync holds it and refreshes it; a blocked/failed sync's process has exited (no lock row) and a wedged holder stops refreshing (TTL lapses), so either correctly falls through to the stale path and is NEVER masked. An actively-syncing source (including a never-synced source doing its first sync) counts as synced_recently, preserving the pinned 3-bucket invariant. The lock lookup reuses doctor's existing dynamic db-lock import and swallows any throw (stub engine, pre-lock-table brain) to false, so it can only ADD an in-progress verdict, never suppress a real stale one. Tests (real PGLiteEngine + real lock rows): stale+no-lock -> fail; stale+live-lock -> ok; never-synced+live-lock -> ok; never-synced+no-lock -> fail; expired-TTL lock -> fail (wedged not masked); blocked source with banked checkpoint rows but no lock -> still fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sync): honest --force-break-lock diagnostic when no lock is held --force-break-lock used to emit the same terse "Lock ... is not held (nothing to break)" line and exit 0 even when a sync was genuinely wedged, sending the operator down a dead end — the wedge was not a held lock. Keep rc=0 (breaking a non-existent lock is idempotently successful; flipping the exit code would break automation), but under --force say plainly that nothing was broken and point at the real next step (gbrain sync / gbrain doctor) plus a `wedge_hint` field in --json output. The non-force path is byte-for-byte unchanged. runBreakLock is exported for the test. Tests: force+no-lock -> wedge_hint JSON + human hint, rc 0; non-force+no-lock -> unchanged terse line, no hint. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(doctor): surface the in-progress sync holder in the freshness message Plan-completion follow-up to the BUG 4 live-lock signal: when a source is actively syncing, name the holder (pid + host) in the check message instead of silently folding it into synced_recently. The note is appended only when something is in progress, so steady-state messages stay byte-for-byte unchanged (the pinned exact-message + 3-bucket-invariant tests still pass). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sync): pre-landing review fixes — monotonic clock seed, scoped CHECK guard Adversarial (codex) review of the implementation diff caught three: - P1 (correctness): the fresh-schema setval was not monotonic. initSchema replays the schema blob, and the unconditional setval(MAX(generation)) could move page_generation_clock_seq.last_value BACKWARD on an already-upgraded brain, letting a stored query_cache bookmark serve stale rows. Seed via GREATEST over the sequence's OWN last_value (+ old table value + MAX(generation)) in all 3 fresh schemas and migration v118, so a replay is idempotent — mirrors the old table's ON CONFLICT DO NOTHING. Pinned by a new monotonic regression test. - P2: v119's CHECK-exists guard keyed on conname only (not globally unique). Scope it to conrelid = 'op_checkpoints'::regclass. - P3: in-progress note ran into the prior sentence in fail/warn doctor messages; separate it with '. '. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: make Anthropic/ZE no-key tests hermetic against a dev config key These "no key" tests cleared only ANTHROPIC_API_KEY / ZEROENTROPY_API_KEY from the env, but hasAnthropicKey() and checkZeEmbeddingHealth() also read the key from ~/.gbrain/config.json. On a dev machine whose real config holds a key, the no-key assertions flipped and the tests failed locally (they passed only in key-less CI). Add a shared with-env emptyHome() helper and point GBRAIN_HOME at an empty dir in every no-key path so loadConfig finds nothing — matching the already-hermetic anthropic-key / gateway-probe tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.44.1.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(key-files): sync doctor + op-checkpoint entries to v0.44.1.0 truth checkSyncFreshness now reports an actively-running sync via the live per-source lock (names holder pid+host, counts as synced_recently) instead of flagging it stale; loadOpCheckpoint gates the legacy union arm on jsonb_typeof = 'array' so a scalar parent can't take down the whole load, and migration v119's CHECK constraint makes the corruption class structurally impossible. Reference docs describe current behavior only — both entries updated in place, no release-clause appends. Guard + llms freshness test green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: re-version to v0.42.51.0 (natural next-off-master) Maintainer override of the queue allocator's leap to 0.44.1.0 (it jumped past in-flight sibling PR claims at 0.42.50/0.43.0/0.44.0). Take the natural next slot in the 0.42.x line above the immediate sibling claim (0.42.50.0); a merge re-bump resolves any collision if a cathedral PR lands first. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci(e2e): bound + retry the OpenClaw install so a transient npm hang can't burn the Tier 2 budget The Tier 2 (LLM Skills) job failed at 30m16s — the `npm install -g openclaw@2026.4.9` step hung on a transient npm/registry stall (orphan `npm install openclaw` was still running at cancel time) and consumed the entire 30m job budget that v0.42.50.0 (#2254) introduced. The install normally finishes in under a minute (Tier 2 is ~4m end to end on master), so this is flaky-install infra, not a test failure. Wrap the install in `timeout 120` + a 3-attempt retry loop with an 8-minute step backstop: a hung attempt is killed in 2 min and retried instead of eating the whole job. Same bound-the-hang philosophy as #2254's job timeouts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
7ea92d602c |
v0.42.48.0 feat(durability): auto-harden brain repos for git durability on PAT+URL (#2241)
* feat(git): divergence-safe pull, push-probe, default-branch detection for brain durability Add GIT_ENV_AUTH + divergenceSafePull (skip-on-dirty, conflict-abort-clean, never-mid-rebase), detectDefaultBranch, pushProbe, and an env-gated GBRAIN_GIT_ALLOW_FILE_TRANSPORT escape hatch. Export GIT_ENV. pullRepo's --ff-only contract is unchanged. * feat(durability): brain-repo hardening core (hook, helper, cron, PAT, AGENTS rules) hardenBrainRepo/unhardenBrainRepo: local untracked post-commit hook + committed brain-commit-push.sh (one shared push-retry template), repo-scoped credential with existing-helper reuse, push-probe verify, active-resolver-file rules with taxonomy from _brain-filing-rules.json, minimal DB-free pull cron. PAT redaction via redactSecretsInText. * feat(sources): harden/pull/unharden commands + auto-harden on add --url sources harden/pull/unharden subcommands; --pat-file/--no-harden on add; auto-harden managed clones on add; unharden-before-remove. cli.ts pre-connect early-exit for DB-free 'sources pull --path' (the cron entry, never opens PGLite). * test(durability): unit + integration coverage for brain-repo durability git helpers, core harden/unharden, hook+helper E2E (real background push), cron generators. 41 tests across 4 files. * chore: bump version and changelog (v0.42.48.0) Brain-repo git durability: auto-harden a brain's working tree (local auto-push hook, committed commit-push helper, always-on agent rules, DB-free pull cron, repo-scoped credential, push-probe verify) the moment gbrain gets a PAT + URL. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sources): route harden exit code through setCliExitVerdict A raw process.exitCode write is zeroed by the owned-verdict flush-exit (#2084 PGLite-Emscripten pollution defense); cli-exit-verdict-pin guard caught it. Use setCliExitVerdict(3) so 'sources harden' actually reports needs-attention to cron/automation. * docs: document brain-repo durability (KEY_FILES + multi-source guide) KEY_FILES: extend git-remote.ts entry (divergenceSafePull, pushProbe, detectDefaultBranch, GIT_ENV_AUTH, GBRAIN_GIT_ALLOW_FILE_TRANSPORT) + add brain-repo-durability.ts/sources-harden.ts entry. multi-source-brains.md: add a Durability (auto-harden) how-to covering sources harden/pull/unharden, --pat-file, the guarantees, and the security posture. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
9d88680a51 |
v0.42.47.0 feat(skillpack,advisor): brain-resident skillpacks + proactive gbrain advisor (#2180) (#2231)
* feat(skillpack): brain_resident manifest fields + init-brain-pack scaffolder + tools version-skew lint Add optional brain_resident/schema_pack to the v1 manifest (additive, forward-compatible). New runInitBrainPack scaffolds a brain-resident pack (brain_resident:true, exact gbrain_min_version, 5-section machine-parseable README) beside brain content; applyWritePlan factored out of init-scaffold. brain-pack-lint validates each skill's declared tools: against the serving op set (E6 version-skew). Wires gbrain skillpack init-brain-pack. * feat(skillpack): Topology A brain-pack discovery on sources add + bounded nag After 'gbrain sources add', if the source ships a brain_resident pack, print an agent-readable advisory (ask the user before scaffolding). nag-state.ts tracks declines per (source-repo brain_id, source, pack) with escalate-then-suppress; declines count only on CLI-interactive displays, never cron/MCP. Fail-open: a malformed/absent pack never breaks sources add. * feat(advisor,skillpack): list_brain_skillpack MCP tool + gbrain advisor Topology B: dedicated source-scoped list_brain_skillpack op + get_skill source_id disambiguation (brain-resident-locate.ts); git scaffold-spec never a server FS path; source-aware schema match. LEARN_INSTRUCTION + serve-http banner. gbrain advisor: read-only ranked actions from brain state (8 resilient collectors, shared renderer, JSONL history, --json severity exit codes, local-only argv --apply dispatcher). Exposed over MCP behind mcp.publish_advisor (default off, read-only on remote; workspace collectors no-op remotely). Generalizes post-install-advisory to a single current-state recommended set (install→scaffold). * feat(skills): bundle gbrain-advisor skill + weekly cron recipe + ranking eval skills/gbrain-advisor teaches a harness to run gbrain advisor on a cadence and ping the user (read-only; ask before fixing). Registered in manifest.json, RESOLVER.md, openclaw.plugin.json. E4 ranking-precision eval on seeded-defect fixtures (100%). * chore: bump version and changelog (v0.42.47.0) Brain-resident skillpacks + gbrain advisor (#2180). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: sync CLAUDE.md + KEY_FILES for brain-resident skillpacks + advisor (#2180) Skill count 29->30, Skills section gains the brain-resident skillpacks + advisor capability, KEY_FILES gets current-state entries for the new modules. Regenerate llms bundles. * test,fix: align stale assertions with generalized advisory + advisor resolver triggers (#2180) - post-install-advisory.test.ts: install→scaffold wording (the install verb was removed); restore two-column in book-mirror copy; drop the removed skillpack-list line. - RESOLVER.md: gbrain-advisor trigger now fuzzy-matches a declared frontmatter trigger (resolver round-trip D5/C). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: regenerate llms bundle for updated advisor resolver row (#2180) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
c023a6041d |
v0.42.46.0 fix(engine): federated read scope reaches by-slug reads (#2200) (#2239)
* fix(engine): federated sourceIds[] scope on by-slug secondary reads (#2200) getTags/getLinks/getBacklinks/getTimeline (both engines) + TimelineOpts now accept a federated `sourceIds[]` read grant, precedence over scalar sourceId, filtering `source_id = ANY($::text[])` — mirroring getPage from v0.42.37.0. - getTags: `page_id = (subquery)` -> `IN (subquery)` + DISTINCT so a slug present in >1 granted source unions tags instead of throwing on a multi-row subquery. - getLinks/getBacklinks: federated branch scopes ALL THREE page endpoints (from, to, AND the authoring origin) so a cross-source link can't disclose a foreign slug. Scalar/unscoped branches unchanged (trusted internal callers keep the cross-source view). - getTimeline: Postgres 8-branch cartesian tree collapsed to one fragment-composed query; PGLite adds the sourceIds branch to its dynamic WHERE. * fix(ops): route by-slug reads through the federated source scope (#2200) get_page resolves tags against the concrete page's source; get_tags/get_links/ get_backlinks/get_timeline route through sourceScopeOpts(ctx) (replacing the copy-pasted scalar `ctx.sourceId ? {sourceId} : {}`). New linkReadScopeOpts promotes an UNTRUSTED remote scalar scope to sourceIds[] so legacy/pre-federated tokens also get all-endpoint link scoping; trusted local CLI keeps cross-source. * test: federated read scope on by-slug reads + engine parity (#2200) Per-op federated reads, isolation (out-of-grant -> empty), cross-source decoy guard, far-endpoint + origin leak guards (F1), same-slug union (D3A), empty-array contract, scalar-remote promotion (D1), getTimeline date-window after the Postgres fragment refactor (D5A), and engine-parity arms for all four methods. * chore: bump version and changelog (v0.42.46.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(key-files): document federated by-slug read scope + linkReadScopeOpts (#2200) The #2200 fix routed get_page tags + get_tags/get_links/get_backlinks/ get_timeline through the federated source scope and added sourceIds[] to the engine read methods + TimelineOpts. Bring KEY_FILES.md to current state: the operations.ts entry's sourceScopeOpts read-op list now includes the by-slug reads and documents the linkReadScopeOpts helper (three-endpoint link scoping + untrusted-remote scalar promotion); the engine.ts entry notes the by-slug read methods + TimelineOpts carry the same sourceIds[] federated axis. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
5c49225e4b |
v0.42.45.0 feat(sync): delta-aware cost estimator — stop wedging the daily cron (#2139) (#2224)
* feat(core): shared computeSyncDelta + spend-posture module (#2139) sync-delta.ts: ONE implementation of "what changed since last_commit", consumed by both the sync executor and the inline cost estimator so the gate's dollar figure can't drift from what the sync imports. spend-posture.ts: spend.posture config + parseUsdLimit/formatUsdLimit off-switch parsing (off/unlimited/none → Infinity; undefined at the budget boundary so ledger rows never serialize null). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(sync): delta-aware cost estimator + non-TTY auto-defer + per-source failure acks (#2139) The inline-embed cost gate was a ~400x phantom: it priced the entire tree whenever the working tree was dirty (always, on an active brain), then blocked the daily cron with exit 2. Now: - performSyncInner + the estimator both route through computeSyncDelta, so the estimate mirrors execution (fetch-first delta; dirty-but-caught-up tree → $0). - shouldBlockSync is posture-aware; non-TTY above floor AUTO-DEFERS embeds to capped backfill jobs (exit 0) instead of wedging — single shared runInlineCostGate on both --all and single-source paths. - --full prices delta + stale backlog (full sync sweeps it inline). - off/unlimited on the cost knobs; tokenmax bypasses the backfill cap (still ledgered) but never the cooldown. - --skip-failed/--retry-failed scoped per source; the D15 parallel refusal is lifted (the #1939 ledger is per-source + lock-serialized). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(config): register spend-control keys + validate spend.posture (#2139) Adds spend.posture + the five previously --force-only spend knobs to KNOWN_CONFIG_KEYS so `config set` accepts them directly (removes the archaeology the issue complained about), and rejects invalid spend.posture values at set time with a paste-ready hint. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reindex,enrich,onboard): spend.posture across the remaining cost gates (#2139) reindex-code: tokenmax makes the cost gate informational; --max-cost accepts off/unlimited. enrich + onboard --auto: tokenmax lifts the refuse-without-cap guardrail and runs UNCAPPED (spend still ledgered by BudgetTracker). Explicit --max-usd always wins over posture. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: cost-gate, delta estimator, spend-posture, off-switch coverage (#2139) New sync-delta + sync-cost-estimate unit suites; rewritten cost-gate serial tests (auto-defer instead of exit 2, posture, off-switch, format split, single-source); parseUsdLimit/posture-aware shouldBlockSync; backfill cap-off + tokenmax-bypass + cooldown-still-refuses; config known-key acceptance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(spend-controls): single spend-control surface + ref-map + follow-up TODOs (#2139) New docs/operations/spend-controls.md (every gate, key, default, off switch, posture interaction); CLAUDE.md reference-map row; two P3 follow-up TODOs (measured chunk-count gating, per-source defer granularity). llms bundles regenerated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(spend): SSRF-harden estimator fetch + complete off/uncapped across reindex/enrich/onboard (#2139) Ship-stage codex pre-landing review caught four P1s in the secondary cost gates: - The delta estimator's fetch-first ran `git fetch` through the plain git() helper, bypassing the GIT_SSRF_FLAGS + GIT_TERMINAL_PROMPT=0 hardening that real sync uses. Added `fetchRemote()` to git-remote.ts (same flags as pullRepo) and route the estimator through it — a cost preview / dry-run can no longer hit a remote through a less-protected path. - `reindex --max-cost off`, `enrich --max-usd off`, `onboard --auto --max-usd off` were parsed but didn't actually proceed/uncap. Now: explicit off (and spend.posture=tokenmax) proceed past the confirmation/missing-cap refusal AND run uncapped. enrich threads an Infinity sentinel mapped to "no BudgetTracker ceiling" (never raw Infinity → no null in audit rows); reindex/onboard use their native undefined=uncapped path. Spend still ledgered. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.42.45.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(KEY_FILES): update sync/embedding/git-remote/reindex entries to post-#2139 truth document-release pass: the cost-gate entries described the pre-#2139 behavior (full-tree-ceiling estimator, --skip-failed-rejects-under-parallel, exit-2 confirmation gate). Updated to current truth — delta-aware estimator via the shared computeSyncDelta, per-source failure acks under parallel, non-TTY auto-defer (no exit 2), posture-aware shouldBlockSync. Added entries for the two new core modules (sync-delta.ts, spend-posture.ts) + fetchRemote on git-remote.ts + reindex --max-cost off. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
090bb53203 |
v0.42.44.0 docs(tutorial): point AlphaClaw deploy link at the official site (#2165) (#2171)
* docs(tutorial): point AlphaClaw deploy link at the official site (#2165) Step 4 of the personal-brain tutorial linked to the wrong top-level domain for AlphaClaw. Corrected so the deploy step works as written. * chore: bump version and changelog (v0.42.44.0) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a81f7e05e8 |
v0.42.43.0 feat(context): push-based context (#2095) + teardown-exit hardening (#2084) (#2175)
* fix(cli): exit deliberately after bounded teardown instead of riding the 10s backstop (#2084)
Root cause: bounded teardown (endPoolBounded, #2015) RESOLVES, but lingering
sockets — embedding-provider fetch keep-alive, PgBouncer txn-mode sockets the
bound raced past — keep Bun's event loop alive, so every `gbrain query` paid
a flat 10s tax exiting via the hard-deadline force-exit banner.
Three changes, one contract:
- flushStdoutThenExit (cli-force-exit.ts): when main() resolves and the
command is not a daemon, exit deliberately — after stdout AND stderr drain
(writableLength===0, 'drain'-event + poll loop, 2s unref'd guard for a
blocked pipe). Incident #1959 (force-exit truncating piped stdout) is the
regression class; pinned by a 256KB real-pipe subprocess test.
- drainThenDisconnect (cli.ts): ONE owner-disconnect helper at all 8 sites
(op-dispatch, CLI_ONLY fall-through, search dashboard, doctor remediation
x3, ze-switch, dream, read-only timeout path). Drains the background-work
registry, then disconnect (best-effort), bounded by the 10s unref'd
hard-deadline — which is now armed around the TEARDOWN window only, not
before the op handler (the old placement would have force-killed any op
slower than 10s). Closes the filed TODOS P3 drain-hoist: six sites
previously skipped the drain entirely and had no hang timer at all.
- Inner process.exit sweep: mid-handler exits in engine-owning/output-bearing
paths (status, friction, claw-test, smoke-test, eval cross-modal /
takes-quality replay / conversation-parser / whoknows-thin, status-thin)
become process.exitCode + return so they flow through the drains and the
flush-exit. Pre-engine usage/parse/refusal exits stay as-is.
BrainRegistry.disconnectAll deliberately unchanged: zero production callers
in src/, per-engine disconnects already bounded, and the kernel reclaims
sockets on exit (src/core/timeout.ts doctrine).
DAEMON_COMMANDS gains 'watch' ahead of the #2095 push transport.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(e2e): PgBouncer transaction-mode pooler in CI + teardown e2e (#2084)
Three consecutive waves (#1972 → #2015 → #2084) fixed pooler-teardown bugs
verified only against one production deployment — CI had no transaction-mode
pooler and could never see the class. Now it can:
- docker-compose.ci.yml: `pgbouncer` service (transaction pooling) fronting
postgres-1, mirroring the production split-pool topology (direct :5432 +
pooled :6543). AUTH_TYPE=plain (pg16 SCRAM verifiers need the plaintext
password in the userlist) + IGNORE_STARTUP_PARAMETERS for the
statement_timeout/idle_in_transaction_session_timeout startup params
gbrain's client sets (the Supabase pooler whitelists the same).
- test/e2e/pgbouncer-teardown.test.ts: schema + fixture via the DIRECT url
into a dedicated `gbrain_pgbouncer` database (never races shard TRUNCATEs),
then spawns the real CLI against the POOLED url and asserts: exit 0,
stdout intact (the #1959 truncation class), and NO
"did not return within 10000ms — force-exiting" banner (pre-#2084 it
printed on 100% of query-shaped ops on this topology). Class bound, not
exact timing. Skips gracefully without GBRAIN_PGBOUNCER_URL.
- scripts/ci-local.sh: threads GBRAIN_PGBOUNCER_URL +
GBRAIN_PGBOUNCER_DIRECT_URL into all three e2e phases.
Verified live: both tests green against pgbouncer 1.25.2 in transaction mode.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(schema): context_volunteer_events table (v116) — push-context feedback log (#2095)
One row per page the brain volunteers (op / reflex / watch channels).
"Used" is DERIVED, never written: pages.last_retrieved_at > volunteered_at
(the existing bumpLastRetrievedAt write-back is the open/cite signal), so
there is no second tracking path. session_id/turn are nullable
caller-supplied attribution; rationale is a deterministic template string,
never raw conversation text.
- Migration v116 (idempotent) + mirrors in src/schema.sql +
src/core/pglite-schema.ts + regenerated schema-embedded.ts (regen also
folds in pre-existing comment-only drift from the v114 links edits).
- src/core/context/volunteer-events.ts: insertVolunteerEvents (ONE
multi-row parameterized INSERT — never per-row awaited round-trips) +
purgeStaleVolunteerEvents (90-day GC, returns 0 on pre-v116 brains).
- Dream cycle purge phase prunes stale events alongside op_checkpoints /
brainstorm checkpoints / batch-retry audit files.
- RLS on Postgres comes from the v35 auto_rls_on_create_table event
trigger (the same mechanism that covered v110 page_aliases and v115
op_checkpoint_paths); the volunteer Postgres e2e pins it.
- No ::jsonb anywhere; no bootstrap probe needed (nothing references the
table pre-creation; writers guard with try/catch).
Tests: v116 shape + columns + indexes + live insert/purge round-trip on
PGLite (test/migrate.test.ts, 161 pass); schema-bootstrap-coverage green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(context): multi-turn window extraction + confidence-scored volunteer core (#2095)
- entity-salience.ts: extractCandidatesFromWindow(turns) — runs the existing
per-turn extractor across the last N turns (oldest→newest), merges by the
normalizeAlias form with occurrence/newest-turn/user-mention metadata, and
orders by salience (recency > frequency > user-role) so the MAX_CANDIDATES
cap drops stale assistant chatter, not the entity the user just named.
Closes the filed assistant-introduced-entities recall TODO; true pronoun
coreference (never-named antecedents) stays out of scope.
- retrieval-reflex.ts: ReflexPointer gains source_id + arm + confidence +
matchedNorm. ARM_CONFIDENCE (alias 0.9 / title 0.8 / slug-suffix 0.6)
lives next to the arm definitions so identity and score can't drift.
Arm-2 provenance is classified in JS (codex D8 — the combined OR can't
report which predicate matched). Federated sourceIds[] scope (alias arm
loops per source; arm 2 uses source_id = ANY — no engine-interface
change). Suppression gains 'slug-only' mode (codex D7, REQUIRED for
windowing): the legacy title-whole-word rule would suppress every entity
merely MENTIONED in a prior window turn, breaking the feature by
construction — slugs only enter context when a pointer/page was actually
surfaced. Default stays 'slug-and-title' for the window=1 legacy path.
- volunteer.ts (new): parseWindow (lenient user:/assistant: prefixes, CRLF,
unprefixed → one user turn), volunteerContext (zero-LLM: extract →
resolve → +0.05 multi-turn/newest-turn boost → min_confidence 0.7 gate →
cap 3/5; deterministic rationale strings, never raw conversation text),
and volunteerUsageStats (per-arm/channel precision from the
last_retrieved_at join, labeled approximate — 5-min throttle false
negatives, unrelated-read false positives; codex D9).
Tests: 35 green across volunteer-context (window parsing, pronoun follow-up
via assistant-introduced entity, confidence gating, slug-only suppression,
takes-fence privacy, multi-source scope, caps, stats join math) +
retrieval-reflex back-compat + resolve-ipc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ops): volunteer_context op — CLI (stdin) + MCP, drained event sink (#2095)
New read-scope op on the contract surface (CLI `gbrain volunteer-context`
with stdin → window, MCP tool for free): takes a rolling conversation
window, returns confidence-gated page pointers with rationales + synopses.
`window` is optional-unless-stats (validated in the handler, codex D9);
`stats: true` returns the volunteered-vs-used precision summary, labeled
APPROXIMATE (the 5-min last-retrieved throttle and unrelated reads both
bias the join). Source scope threads through sourceScopeOpts — federated
grants narrow the volunteer to the granted sources.
Event logging is fire-and-forget through a new `volunteer-events`
background-work sink (volunteer-events.ts, mirrors last-retrieved: tracked
dangling promise set + bounded drain + snapshot-drop on timeout so a
long-lived process never accumulates ghosts). ONE batched INSERT per call,
drained on every exit path by the commit-1 drain hoist; failure never
fails the op (pinned by an injected failing-engine test).
cli formatResult renders both shapes (pointer lines with confidence/arm/
rationale; the stats summary with per-arm precision).
Tests: op contract surface, window-required validation, sink round-trip
with session_id/turn attribution, failing-engine fail-open, federated
grant scoping, stats mode (26 green on PGLite) + a real-Postgres e2e
proving the op + sink + stats join AND that context_volunteer_events has
RLS enabled (keeps the auto-RLS event-trigger mechanism honest for v116).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(context): reflex consumes the rolling window + ambient-channel logging (#2095)
The default-on retrieval reflex now extracts entities from the last N turns
(retrieval_reflex_window_turns, default 4; env
GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS; window=1 reproduces the legacy
current-turn-only behavior exactly). assemble() passes the recent
user/assistant turns (hard cap 12); the reflex slices to the configured
window. Assistant-introduced entities and "what did she invest in?"
follow-ups whose antecedent was NAMED in the window now surface pointers —
the issue's "zero agent-initiated queries" success criterion on the
ambient path.
Under windowing, suppression switches to slug-only (codex D7): the legacy
title-whole-word rule would suppress every entity merely MENTIONED in a
prior window turn, breaking the feature by construction. Slugs only enter
prior context when a pointer/page was actually surfaced, so
already-surfaced pages still suppress. The suppression mode flows through
all three resolver rungs (host opts, serve IPC request, direct Postgres).
Ambient-channel feedback (codex D11): the server-side resolver paths
(serve IPC + direct Postgres) log volunteered pointers with
channel: 'reflex' through the drained volunteer-events sink, so
`gbrain volunteer-context --stats` measures the default-on path where most
volunteering happens. Host-injected resolvers (no gbrain engine) can't
log — documented gap. Precision gates, 1.5s ceiling, fail-open, and the
pointer cap are unchanged.
Tests: prev-assistant-turn entity fires; window=1 legacy parity; slug-only
vs already-surfaced suppression; throwing resolver stays fail-open
(16 green).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(cli): gbrain watch — push transport over stdin (#2095)
The issue's headline: the brain volunteers pages as the conversation flows,
instead of waiting to be asked. `some-transcript-feed | gbrain watch` reads
turns line-by-line ('user:'/'assistant:' prefixes set the role; unprefixed
lines are user turns), keeps a rolling window (--window-turns, default 4),
and streams confidence-gated pointers with rationales to stdout (--json for
JSONL). Session dedupe rides the core's slug-only suppression — a slug is
volunteered at most once per session. Events log on channel 'watch' with
session_id + turn through the drained sink.
Lifecycle: watch BLOCKS in the stdin iteration (like `jobs work`) — an
interactive TTY stays alive until Ctrl-C/Ctrl-D, piped input ends at EOF —
so it is deliberately NOT in DAEMON_COMMANDS (reverts the commit-1
placeholder): when main() resolves the work is over, the CLI_ONLY finally
drains volunteer events via drainThenDisconnect, and the entrypoint
flush-exit ends the process. Keeping it in the daemon set would have made
the piped EOF path hang on lingering sockets — the exact #2084 class.
SIGINT closes the stream and flows through the same drain path instead of
killing mid-write. Per-turn resolution failures are fail-open (the stream
never dies on a transient DB error).
Full wiring (eng-review D12): CLI_ONLY + CLI_ONLY_SELF_HELP (WATCH_HELP) +
THIN_CLIENT_REFUSED_COMMANDS (thin clients use the volunteer_context MCP
op) + main --help entry.
Tests: 18 green — help, per-turn volunteering + clean EOF return, rolling
window via assistant-introduced entity, session dedupe, --json shape with
turn attribution, channel-watch event rows, --min-confidence gate, CRLF/
blank tolerance, daemon-gate semantics. Live smoke: piped `gbrain watch`
on a fresh PGLite brain exits 0 at EOF with no force-exit banner.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: KEY_FILES + push-context guide + TODOS for the #2084/#2095 wave
- docs/architecture/KEY_FILES.md (current-state): context entries gain the
window extractor, arm provenance/confidence, suppression modes, volunteer
+ volunteer-events modules; background-work entry now lists FIVE sinks and
the drainThenDisconnect owner-disconnect contract; new entries for
src/core/cli-force-exit.ts (the exit contract) and src/commands/watch.ts.
- docs/guides/push-context.md (new): the three channels (reflex/op/watch),
the confidence model, CLI usage, config keys, and the approximate-stats
caveat. Linked from CLAUDE.md's reference map.
- CLAUDE.md: ops line mentions volunteer_context + the guide link;
bun run build:llms regenerated in the same commit (freshness test green).
- TODOS.md: #2095 deferrals filed (SSE push channel, policy skill + doctor
check, structured messages[] param); the #1981 entity-detection TODO
narrowed (window extraction covered assistant-introduced entities +
named-antecedent follow-ups); the drain-hoist P3 marked DONE by the
#2084 wave.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(e2e): truncate context_volunteer_events in setupDB (#2095)
The new feedback-log table wasn't in ALL_TABLES, so volunteered-event rows
persisted across e2e runs on a reused database and poisoned count/stats
assertions in volunteer-context-postgres on the second run. No FK to pages
(slug join), so position before pages is for hygiene only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cli): own the exit verdict — never trust ambient process.exitCode (#2084)
Caught by the full unit suite: `gbrain apply-migrations` on PGLite started
exiting 99. Root cause: PGLite's Emscripten runtime writes the WASM
backend's proc_exit status into process.exitCode (initdb at create-time,
the postmaster at close-time — `exitCode=status` in pglite's dist), and
the writes land ASYNCHRONOUSLY, outside any snapshot/restore window around
create/close (a guarded attempt verified this). The pre-#2084 success path
never read process.exitCode, so the pollution was invisible; the new
deliberate flush-exit propagated it faithfully.
Fix: gbrain records its own verdict. setCliExitCode(n)/getCliExitCode() in
cli-force-exit.ts — every gbrain-owned exit-code assignment routes through
the setter (still mirrored to process.exitCode for outside readers), and
both exit paths (entrypoint flushStdoutThenExit + the drainThenDisconnect
hard-deadline backstop) read the getter. Swept all assignment sites:
cli.ts (op error, friction, claw-test, smoke-test, eval runners, status,
import errors) + reindex/transcripts/brainstorm/frontmatter/autopilot.
Also updates the v0.42.20 structural pins to the drainThenDisconnect shape
(ordering invariant asserted INSIDE the helper + >=8 helper call sites,
superseding the two-inline-pairs assertion).
Verified: apply-migrations spawn test green; `init --migrate-only` exits 0;
an errored op still exits 1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: re-pin the teardown-arming invariant at its post-#2084 home
Master's v0.42.41.0 triage wave and the #2084 wave fixed the same
pre-armed-timer bug independently; the merge keeps #2084's shape (arming
inside the shared drainThenDisconnect helper, covering all 8 exit paths).
The structural pin now asserts the same invariant — no pre-try arming;
gated, unref'd, before-drain, cleared — at the helper.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: coverage for ambient reflex-channel logging + watch window/cap flags
Ship coverage audit (85%, gate PASS) named five gaps; the two substantive
cheap ones close here: the codex-D11 logChannel='reflex' path now has a
behavioral pin (events land on channel 'reflex' through the drained sink;
no logChannel → no events), and gbrain watch's --window-turns / --max-pages
flags are exercised (turn-1 attribution under window=1; cap to one page).
Remaining flagged-not-blocking: the wallclock-timeout branch (untestable
without >10s real-clock flake — same rationale as the arming pin),
formatResult's volunteer case (module-private), and the cycle purge wiring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: close the remaining plan-audit gaps — formatResult rendering + watch SIGINT
formatResult exported for tests (same import-safety contract as cliAliases);
test/cli-format-volunteer.test.ts pins the pointer lines, empty-gate message,
and approximate stats summary. test/watch-command.test.ts gains a real
subprocess SIGINT test: piped stdin that never reaches EOF, SIGINT mid-stream,
assert exit 0 with no force-exit banner — the drain-then-exit lifecycle under
the actual signal, not just the shared exit path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: doctor's FAIL verdict was zeroed by the owned exit — sweep stragglers + class pin
The merged-state suite caught it: doctor --fast --json reported FAIL but
exited 0. Master's v0.42.41.0 brought raw `process.exitCode =` writes
(doctor.ts hasFail ternary, extract.ts) that the #2084 verdict-owning exit
silently zeroes — getCliExitCode() deliberately never reads ambient
process.exitCode (the PGLite-Emscripten pollution defense), so any setter
that bypasses setCliExitCode reports success on failure.
Swept both sites and added the structural class pin: a test greps src/ for
raw `process.exitCode =` outside cli-force-exit.ts, so the next merge that
introduces one fails loudly instead of lying about exit codes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.43.0)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: quarantine the watch SIGINT subprocess test to the serial lane
The parallel unit shards flake on concurrent CLI subprocess spawns (failed
at 7ms in-suite, green solo) — same isolation rationale as
apply-migrations-pglite-spawn.serial.test.ts and #2141's R3 quarantine.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: update project documentation for v0.42.43.0
Post-ship doc verification against the release diff (#2095 push-based
context + #2084 superset hardening), with a cross-model doc review:
- push-context.md: version tag corrected to v0.42.43.0; per-call knobs
now cover prior_context/days and watch's flag surface accurately;
feedback-log writes described as best-effort; synopsis fence-strip
described as unconditional.
- CLAUDE.md: stale operation count (~47 -> ~90); volunteer_context
release reference corrected to v0.42.43.0.
- KEY_FILES.md: ci-local entry rewritten to current topology (4-shard
parallel default, four Postgres services, transaction-mode PgBouncer
+ GBRAIN_PGBOUNCER_URL/_DIRECT_URL exports); stale E2E file counts
dropped from the selector entry.
- TESTING.md: inventory entries for the new #2084 structural pins
(cli-exit-verdict-pin, cli-pipe-truncation), the push-context test
suite (volunteer-context, watch-command, watch-sigint.serial,
cli-format-volunteer), migrate v117 coverage, and the two new E2E
files (pgbouncer-teardown env gating, volunteer-context-postgres RLS
pin); check:all row corrected (not a superset of verify).
- AGENTS.md + RELEASING.md: ci:local descriptions updated to the
sharded + pooler topology.
- CHANGELOG (wording only, entry preserved): "retrieved" instead of
"opened" for the used-signal, pooler scoped to the local CI gate,
feedback log labeled best-effort.
- llms-config.ts: index the new push-context guide; bundles
regenerated (build:llms) and freshness test green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(test): correct the v116 reference — the table shipped as migration v117
* fix: pre-landing review hardening — federated alias parallelism, trust-boundary clamps, shared protocol helpers (#2095)
Five specialist reviewers (testing/maintainability/security/performance/
data-migration) on the reconciled diff; every finding applied:
Performance: the alias arm now resolves all granted sources CONCURRENTLY
(a federated caller paid M sequential RTTs per turn — ~355ms at 5 sources
cross-region, inside the reflex's 1.5s budget); watch's session dedupe is
O(1) Set membership instead of a monotonically growing priorContext string
(O(T²) over a long-lived session); getWindowTurns iterates from the tail
(per-turn cost no longer grows with session length); the resolver's
provenance maps fold into the existing candidate pass.
Security: volunteer_context clamps caller-supplied attribution at the trust
boundary — session_id capped at 256 chars (a read-scoped token could bank
~1MiB TEXT per request, retained 90 days), turn logged only when a safe
integer (a non-integer threw inside the batched INSERT and silently dropped
the whole batch). The privacy comments now state precisely what rationale
may contain (the matched entity's surface form — which by construction
resolved to an existing alias/title/slug — never free conversation text).
Maintainability: TURN_PREFIX_RE + formatVolunteeredPage exported from
volunteer.ts and shared by watch/cli (the two surfaces can no longer
drift); volunteerEventRowsFrom is the single VolunteerEventRow assembly
site for all three channels; watch's window default now honors the same
retrieval_reflex_window_turns config knob the reflex reads; the stale
pre-v116 comments swept to pre-v117.
Testing: the two flake-class CRITICALs fixed (pipe test asserts the
backstop banner instead of a cold-CI-hostile 9s wall bound; the SIGINT test
waits on watch's new machine-readable ready line instead of a fixed 15s
sleep — 2.5s and deterministic now); new coverage for the sink's timeout
branch + ghost-reference drop, watch per-turn fail-open, untrusted knob
clamps (min_confidence/max_pages/days), window-cap ordering (newest user
mention survives), serve-IPC suppression passthrough + channel=reflex
logging, windowTurnCount edge semantics, and structural pins for the sink
registration + cycle purge wiring. The exit-verdict pin's grep is now
operator/whitespace-tolerant.
Deferred with TODOs: resolver index shapes for the per-turn query;
batched first-prune after a long dream-cycle gap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(context): red-team hardening — pre-cap dedupe, delivery-side reflex logging, window clamp
Four red-team findings on the #2095 push-context surface:
- RT1 starvation: watch's session-dedupe Set filtered AFTER volunteerContext's
cap, so a recurring already-pushed entity burned cap slots every turn and
starved fresh pages behind it. VolunteerOpts.excludeSlugs now skips inside
the pointer loop BEFORE the confidence gate and the cap.
- RT3 honest stats: reflex-channel event logging moved from inside the
resolver to the DELIVERY point — serve's resolve-IPC onDelivered hook fires
only after the response write succeeds, and buildReflexAddition logs only
after the per-turn timeout admits the block. A block the client's 250ms
budget abandoned was never injected and no longer counts as volunteered.
(logChannel resolver opt removed; logDeliveredReflexPointers is the seam.)
- RT5 unbounded window: --window-turns is clamped to [1, 64] so a config typo
can't reintroduce the re-scan-everything-per-turn cost class.
- RT2/RT4 documented + filed: PGLite watch connection monopoly (WATCH_HELP,
push-context guide, TODO to route watch via serve IPC); host-resolver
suppression contract at ResolveEntitiesFn (TODO for a capability gate).
Tests: starvation guard (watch + volunteerContext unit), window clamp floor +
ceiling, delivery-side logging (helper writes channel=reflex through the
drained sink; bare resolver writes nothing; empty list no-op), IPC wiring test
rewired to onDelivered. KEY_FILES.md + push-context.md updated; build:llms run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(context): env-plane window knob works config-less; harden two gateway-state-leak victims
Three CI-only check failures, two root causes:
1. windowTurnCount ignored GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS when
loadConfig() returned null (no config file AND no DATABASE_URL — a clean
CI shard with no brain). loadConfig drops its env→config mapping in that
case, so the documented escape hatch silently died and the window fell
back to 4 → windowed extraction widened when the test set window=1 →
prior-turn entity leaked. Fixed: read the env var directly in
windowTurnCount, mirroring reflexEnabled's direct process.env read. This
is a real product bug, not just a test artifact — any config-less host
using the env hatch was affected. Regression test pins it.
2. sync-cost-preview + doctor-federation-health failed only IN-SHARD: a
sibling test configured a non-legacy (ZeroEntropy 1280-d / $0.05) gateway
and never reset it. The legacy-embedding preload only restores the
OpenAI/1536 default when the gateway slot is EMPTY, so a non-empty foreign
config survives into the next file — and a file's beforeAll runs BEFORE
the preload's restoring beforeEach, so federation-health built a
vector(1280) column and its 1536-d fixture hit CheckExpectedDim. My new
test files reshuffled the deterministic file→shard assignment, exposing
this latent ordering bug. Hardened both victims to establish the gateway
state they assert (sync-cost-preview resets to the unconfigured fallback;
federation-health pins legacy 1536 before initSchema) so they're
order-independent. Verified against a simulated leaker run before them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(context): use withEnv() in the window env-hatch test (test-isolation guard)
The regression test added in
|
||
|
|
4ee530f3c5 |
v0.42.42.0 fix(cli): bounded teardown + explicit exit — kill the 10s force-exit tax on txn-mode poolers (#2084) (#2141)
* feat(core): finishCliTeardown + flushThenExit — bounded teardown, owned exit verdict (#2084) cli-force-exit.ts becomes the single owner of one-shot CLI exit + teardown: - finishCliTeardown: bounded sink drain -> bounded disconnect under a backstop whose deadline is COMPUTED from the bounds it guards (floor 10s; GBRAIN_TEARDOWN_DEADLINE_MS env override). Arms at teardown start, never before the op handler. - flushThenExit: stdio write-fence (unref'd guard, EPIPE-safe) + REF'D aliveness grace for non-TTY stdio — Bun only delivers queued pipe writes while the process is alive (no flush API reaches the native queue). - setCliExitVerdict/currentExitCode: the exit verdict lives in a gbrain-owned channel, never read back from process.exitCode (PGLite's Emscripten runtime scribbles its own status there mid-run). - background-work.ts exports backgroundWorkSinkCount() for the deadline formula. Unit tests + a spawned-Bun harness proving byte-complete piped output. * fix(cli): route all nine disconnect sites through finishCliTeardown; one exit seam (#2084) Deletes the pre-handler 10s force-exit timer (it measured handler + teardown combined: PgBouncer txn-mode deployments paid a flat 10s banner tax on every query, and any >10s op was killed mid-run with exit 0 and truncated output). Sweeps op-dispatch, CLI_ONLY fall-through, search dashboard, read-only timeout path, dream, doctor x3 (fixing a pre-existing pool leak when DB checks throw), and ze-switch. The ONE process exit lives in main().then/catch via flushThenExit(currentExitCode()), gated by shouldForceExitAfterMain(). Exit-code writers (op-dispatch catch, reindex, transcripts, brainstorm, autopilot, frontmatter) now set the verdict through setCliExitVerdict. * fix(pglite): contain Emscripten's process.exitCode writes at PGlite.create (#2084) PGLite's WASM runtime writes its own status into process.exitCode (99 at create; in-memory brains run initdb whose status lands on a later tick; the exit status at close) — on PGLite every error exit was silently clobbered. preservingProcessExitCode wraps create() to keep the global tidy; db.close() stays unwrapped (its 0-write is baseline behavior test runners depend on). The CLI verdict itself is immune: it lives in the owned channel. * test: e2e + structural pins for the #2084 teardown contract E2E: failed op exits 1; every swept command spawned (brain-copy isolation for mutators, no-network); slow-handler regression via the deadline env knob; piped --json parses complete; teardown banner absent on every happy path; daemon survival untouched. Structural: no bare awaited engine disconnects in cli.ts; DISCONNECT_HARD_DEADLINE_MS gone; >=9 helper call sites; verdict channel + create-wrap pins. * test: fix R1 env-isolation violations in retrieval-reflex tests Pre-existing on master: both files mutated GBRAIN_RETRIEVAL_REFLEX directly, failing scripts/check-test-isolation.sh (bun run verify). Converted to the canonical withEnv() pattern; the reflex describe's beforeEach also never restored the flag, leaking it across the shard. * docs: KEY_FILES entries for the teardown contract; close + file TODOS (#2084) KEY_FILES.md: current-state entry for cli-force-exit.ts (helper + central exit seam pair, verdict channel, cli.ts-scoped claim); background-work.ts and pglite-engine.ts entries updated. TODOS.md: the drain-before-owner-disconnect P3 (filed from #1972) is done by this wave; files the trigger-gated GBRAIN_COMMAND_DEADLINE_MS follow-up (eng-review D2/D14). * fix: pre-landing review fixes (#2084) Review army (testing/maintainability/security/performance, 0 critical): - drain defense-in-depth: a throwing drain warns and still disconnects (cannot escape a caller's finally or skip the engine teardown) - behavioral tests for preservingProcessExitCode (connect pins 0; create-throw restores the pre-call verdict) - D9 widening test (live-registry sink count feeds the deadline formula), env 0/negative boundary cases, verdict mirror-write assertion - stale comments: header diagram backstop line, structural-test 'both lifecycle calls' contradiction, KEY_FILES 10s-force-exit clauses, e2e D11 falsification story corrected - named the formula's pool-end literals * fix: adversarial-review hardening — daemon-safe command resolution, flush knob, ref'd backstop (#2084) Cross-model adversarial review (Claude subagent + Codex, both P1'd it): - shouldForceExitAfterMain now resolves the command through parseGlobalFlags — the old first-non-dash heuristic read `gbrain --timeout 30s serve` as command "30s" and the new exit seam would have killed the daemon ~250ms after boot with exit 0 (unit-pinned) - GBRAIN_FLUSH_GRACE_MS env override for the non-TTY aliveness grace (batch consumers piping large payloads to slow readers can raise it; agent loops can lower it) - backstop timer is now REF'D: a hung teardown on an otherwise-empty event loop previously exited naturally — skipping the flush and surfacing PGLite's scribbled process.exitCode - flushThenExit: real process.exit latched once per process - doctor-site comment corrected; in-command process.exit teardown-bypass class (pre-existing) filed as a P2 TODO * chore: bump version and changelog (v0.42.42.0) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: move #2084 exitCode-containment lifecycle tests to the serial quarantine (R3) * docs: update project documentation for v0.42.42.0 - docs/TESTING.md: replace the stale 4-file serial-quarantine enumeration with a current-state description (the quarantine is glob-discovered, now several dozen files incl. the #2084 exitCode-containment suite); add unit inventory entries for test/cli-finish-teardown.test.ts and test/flush-then-exit-harness.test.ts. - docs/architecture/KEY_FILES.md: rephrase the pglite-engine exitCode containment note to current-state wording (clears the check-key-files-current-state prose-history warning). llms bundles regenerated (byte-identical: both docs are link-only). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: apply cross-model doc-review findings for v0.42.42.0 Codex review of docs-vs-shipped-code found 9 gaps; all verified against the code before fixing: - CHANGELOG.md (0.42.42.0 entry, precision narrowing only — no entries touched): "every CLI exit path" -> "every cli.ts disconnect site"; "on every path" -> "on every routed exit path"; dream/doctor/ze-switch claim scoped to dispatcher teardown (command-internal process.exit sites are tracked in TODOS as the open P2). - docs/architecture/KEY_FILES.md: the teardown backstop is REF'D, not unref'd (matches the F3 adversarial-review decision in the code). - src/core/cli-force-exit.ts: header diagram comment had the same stale unref'd claim + `process.exitCode ?? 0`; now matches the implementation (ref'd timer, `currentExitCode()`). Comment-only change. - docs/TESTING.md: verify is the 30-check parallel battery via run-verify-parallel.sh (was described as 4 checks); CI is 10 weighted LPT shards + dedicated verify/serial/slow jobs (was "4-way FNV on shard 1"); test:serial runs one bun process per file (not --max-concurrency=1); dead "cap: 10" line rewritten as debt guidance; inventory entries added for test/cli-should-force-exit.test.ts and test/e2e/pglite-cli-exit.serial.test.ts. bun run verify green (30/30); #2084 test files green; llms bundles regenerated (byte-identical — reference docs are link-only). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: route v0.42.41.0's raw exitCode writers through the verdict channel; reconcile merged structural pins (#2084) CI fallout from merging the v0.42.41.0 triage wave into the #2084 exit-seam design — both waves fixed the same timer-placement bug independently: - doctor.ts + extract.ts set failure exit codes via raw `process.exitCode =` writes (v0.42.41.0's process.exit -> exitCode conversion); the #2084 exit seam reads only the gbrain-owned verdict channel, so doctor FAILs exited 0 (Tier 1 RLS e2e + half-migrated-Minions tests). Converted to setCliExitVerdict, same as the wallclock-124 site in the merge commit. - cli-force-exit-teardown-arming.test.ts pinned v0.42.41.0's inline finally-armed timer, which the merge replaced with finishCliTeardown; rewritten to pin the merged invariant (no pre-try arming in cli.ts; the backstop arms inside the helper before the drain). - eval-capture drain timing bound 1s -> 2s: flaked at 1023ms under CI shard load after the new test files shifted LPT shard packing (13x budget slack still proves bounded-not-hung). --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
+1
|
7c27fa129b |
v0.42.41.0 fix: triage wave — 6 data-loss/availability fixes + 9 community PRs (#2128)
* fix(oauth): default omitted authorize scope to client's full grant When a client omits `scope` on /authorize, the authorize() grant computed `(params.scopes || []).filter(...)` → the empty set. That empty grant was written to oauth_codes and propagated into the access AND refresh tokens, so every request failed `insufficient_scope` even though the client was registered with e.g. `read write`. Because refresh inherits the stored grant, it never self-healed — reconnecting just minted another empty-scoped token. Some MCP connectors (observed with Claude Desktop) omit `scope` on /authorize, so they hit this on every connection. Fix: when no scope is requested, default to the client's full registered scope (RFC 6749 §3.3 permits a server default). This mirrors exchangeClientCredentials, which already does `requestedScope ? ... : allowedScopes`. The result is still clamped to the allowed set, so an explicit over-broad request cannot escalate. Adds test/oauth-authorize-scope-default.test.ts covering: omitted/empty → inherits full grant; explicit subset honored; clamp preserved (over-broad and disallowed-only requests cannot escalate or trigger inheritance). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sync): skip Python venv/ in the code walker collectSyncableFiles (first-sync walker) and the incremental PRUNE_DIR_NAMES set skipped node_modules but not Python venv/. On a Python repo the walker descended into venv/ (thousands of files); the resulting slug collisions crashed putPage's INSERT ... ON CONFLICT ... RETURNING with "undefined is not an object (evaluating 'row.deleted_at')". Add `venv` alongside node_modules in both the import.ts inline skip and PRUNE_DIR_NAMES. venv is the Python equivalent of node_modules. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gateway): carry asymmetric input_type across the AI SDK to the wire body (#1400) dimsProviderOptions() threads input_type ('query' | 'document') into providerOptions.openaiCompatible for asymmetric models (ZE zembed-1, Voyage v3+), but the AI SDK's openai-compatible adapter validates providerOptions against a fixed schema and silently drops the field before building the HTTP body. Every embedQuery() was therefore encoded document-side: the ZE shim's hard default fired ('document'), Voyage and local openai-compat servers got no input_type at all, and asymmetric retrieval silently collapsed toward surface-token overlap — while the providerOptions-level contract test stayed green. Fix: an AsyncLocalStorage (same pattern as __budgetStore) populated in embedSubBatch() only when providerOptions actually threads an input_type, read at body-rewrite time by the fetch shims: - zeroEntropyCompatFetch: recovers the threaded value; document default preserved for ingest paths. - voyageCompatFetch: opt-in like the dims.ts Voyage branch — inject only when threaded; the field stays off the wire otherwise. - NEW openAICompatAsymmetricFetch: fallthrough default for every other openai-compatible recipe (llama-server, litellm, ollama, ...) — the canonical local/proxy paths for asymmetric models. Strict pass-through when nothing was threaded, so symmetric deployments see zero wire change; recipes with their own compat fetch (azure) keep it via the compat.fetch ?? precedence. KNOBS_HASH_VERSION bumped 10→11: cached query_cache rows were keyed on document-side query vectors; pre-fix rows must not be served to post-fix lookups (same convention as the v=3 embedding-provider bump). One-time global cold-miss on upgrade; refills within cache.ttl_seconds. Tests: test/embed-input-type-wire.test.ts runs the REAL SDK transport with a mocked global fetch and asserts on the outbound body — the only layer where this regression is observable. Covers ZE hosted, llama-server, litellm, ollama (query + document sides) and pins the pass-through for non-asymmetric models and Voyage's opt-in shape. 4 of the original 7 assertions fail on master, proving the pin. One structural pin in test/ai/zeroentropy-compat-fetch.test.ts updated to the new line shape (same semantic); KEY_FILES.md gateway.ts entry updated to the new truth. Supersedes #1400 (closed unmerged) — same ALS mechanism, extended to Voyage + all openai-compatible recipes. Credit to @billy-armstrong for the original diagnosis. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sync): honor .gitignore in code walk; prune vendor/dist/build collectSyncableFiles (the full-sync / dry-run enumerator) reimplemented its own directory skip list inline (node_modules || ops), bypassing the canonical pruneDir gate and ignoring .gitignore entirely. On a Laravel/PHP repo this descended into vendor/ (~50k Composer files), storage/, and public/build/, trying to import 52k dependency/build files and flooding the index with library internals (a 35-min sync that never finished, killed by the watchdog at 3%). - collectSyncableFiles now enumerates via `git ls-files --cached --others --exclude-standard` when dir is a git work tree, so the walk honors .gitignore (tracked + untracked-not-ignored). Falls back to the FS walk for non-git dirs. EroLab: 52164 -> 1028 files. - The FS fallback now prunes through the canonical pruneDir() instead of a drifted inline list, so the two skip lists can't diverge again. - PRUNE_DIR_NAMES gains vendor/dist/build (dependency + build-output trees). Addresses #1483 (.gbrainignore), #1159 (--respect-gitignore), and the maintainer's #1942 vendor/dist/build prune. Walker regression suites (sync-walker-symlink, brain-writer-walk-prune, sync, sync-walker-submodule) green: 90 pass. * fix(config): ignore DATABASE_URL auto-loaded from cwd .env (#427) Bun merges .env files from the process cwd into process.env before any user code runs. loadConfig() prefers env DATABASE_URL over ~/.gbrain/config.json, so any gbrain invocation from inside a web-app checkout silently retargets the brain at that app's database — reads go to the wrong DB and apply-migrations can write gbrain's schema into a production app database (#427). effectiveEnvDatabaseUrl() re-parses the .env files Bun auto-loads from cwd and treats a DATABASE_URL whose value matches one of them as file-origin: ignored, with a one-time stderr notice. GBRAIN_DATABASE_URL and genuinely exported DATABASE_URLs are honored unchanged, so the operator escape hatch and the e2e suite's env-provided URL keep working. Applied at loadConfig, getDbUrlSource (doctor parity), init --non-interactive, and migrate --to. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): arm the disconnect hard-deadline at teardown entry, not before the op body The 10s force-exit timer in the shared-op dispatch was armed BEFORE the try block, so any op whose handler ran past 10s wall-clock was killed mid-flight with process.exit(0) and zero stdout. On a slow Postgres pooler (6-10s per fresh connection) a healthy `gbrain search` was force-exited every time — an empty 'success' indistinguishable from no results. The v0.42.20.0 exitCode honor can't help: a mid-op kill fires before any error path sets exitCode. Move the arming into the finally (teardown entry), matching the fall-through owner-disconnect site later in main(): the timer still bounds a hung drain/disconnect (the C13 contract) but can no longer kill a slow-but-progressing op. Verified on a transaction-pooler Supabase brain: search went from 0 bytes/exit 0 at 10s to real results at ~21s. * fix(import): stamp source_id on extracted call-graph edges importCodeFile built CodeEdgeInput rows without source_id, so every edge landed NULL. getCallersOf/getCalleesOf filter `AND source_id = <scoped>` whenever a worktree pin or --source is in play — NULL never matches, so scoped call-graph queries silently returned 0 rows on multi-source brains even though the edges existed (2,122 edges, 26 targeting the probed symbol, count 0 returned). One-line fix: carry the sourceId already in scope into the edge input. Existing NULL rows backfill with: UPDATE code_edges_symbol e SET source_id = p.source_id FROM content_chunks c JOIN pages p ON p.id = c.page_id WHERE c.id = e.from_chunk_id AND e.source_id IS NULL; (same for code_edges_chunk). Verified: code-callers returns 21 callers where it returned 0. * docs(migrations): NULL embeddings BEFORE the column-type alter The Postgres recipe ordered ALTER COLUMN TYPE vector(N) before the UPDATE that clears stale embeddings. pgvector refuses to cast existing vectors across dimensions ('expected 1024 dimensions, not 1536'), so the recipe as written aborts the transaction on any brain that has embeddings — which is every brain doing this migration. Swap the steps: NULLs cast fine. * fix: honor legacy token source grants in oauth * fix(cli): bound read-scope op handlers at 180s wallclock (pre-landing review) With the hard-deadline timer correctly scoped to teardown, a genuinely wedged read handler (hung pooler connection mid-query) would hang the CLI forever — the #1633 zombie class the old pre-try timer accidentally bounded at 10s. Reads now get a generous withTimeout (180s default, far above any healthy slow-pooler run; --timeout=Ns overrides; exit 124 with the teardown finally still draining + disconnecting). Writes/admin stay unbounded: a long import/embed must never be killed by a default. * fix(import): stamp unscoped edges 'default', matching the pages-table default Review catch: 'sourceId ?? null' fixed the scoped path but left the unscoped one (reindex --code without --source, importCodeFile callers without opts.sourceId) stranding edges at NULL while their pages land under the schema default (pages.source_id DEFAULT 'default') — so getCallersOf(sym, { sourceId: 'default' }) missed them. Same bug, other door. Fallback is now 'default'. * fix(core): runtime dim-migration recipe NULLs embeddings before the alter Review catch: the doc fix corrected docs/embedding-migrations.md, but embeddingMismatchMessage still PRINTED the broken order — ALTER before UPDATE ... SET embedding = NULL — and linked to the now-contradicting doc. pgvector refuses to cast existing vectors across dimensions, so the printed recipe aborted on any brain that has embeddings. Swap the steps and say why inline. * feat(migrate): v116 — backfill NULL edge source_id + index from_symbol_qualified 1. Backfill: edges written before the stamping fix sit at source_id=NULL and stay invisible to scoped call-graph queries until repaired. Derive each edge's source from its own from_chunk's page (pages.source_id is NOT NULL DEFAULT 'default'). Same SQL verified live on a 2,122-edge production brain. 2. Indexes: getCalleesOf filters both edge tables on from_symbol_qualified, which had no index — every callee lookup was a seq scan, amplified per-BFS-node by the recursive code walk. With NULL edges repaired, scoped walks actually expand, so the latent cost becomes real. Mirrored into src/schema.sql; schema-embedded.ts regenerated. * docs(migrations): align the rationale list with the corrected recipe order The 'Why we don't do this automatically' list still said alter-then-wipe; reorder to wipe-then-alter and replace the fragile 'step 3' numeric cross-reference with a name-based one. * test: regression coverage for edge source_id stamping, timer placement, recipe order - import-code-edges-source-id: scoped import stamps edges + scoped getCallersOf/getCalleesOf match (verified failing pre-fix), plus the unscoped-import case asserting 'default' stamping. - cli-force-exit-teardown-arming: structural pin — the hard-deadline timer arms inside the finally (teardown entry), never before the op body; daemon guard, unref, clearTimeout intact. - embedding-dim-check: recipe order pinned — UPDATE precedes ALTER so the printed SQL can't drift from docs/embedding-migrations.md again. * fix(cli): hard-exit after teardown on wallclock timeout; bound makeContext too Adversarial review, two findings on the new timeout path: 1. On timeout the finally drained, disconnected, then CLEARED the hard-deadline timer — removing the only backstop while the abandoned handler (withTimeout races, it does not cancel) can hold ref'd sockets/SDK timers that keep Bun's loop alive: 'timed out' printed, process immortal — the zombie class this branch exists to kill, resurrected through its own fix. The finally now exits explicitly after teardown completes on the timeout path. 2. makeContext does DB I/O (resolveSourceId) for EVERY op and sat outside any bound — a pooler wedge at context build hung reads, writes, and admin alike. It now shares the same wallclock bound. * fix(import): normalize edge source once — closes the '' door and the unscoped chunk fan-out Adversarial review: txOpts used truthiness while the edge stamp used nullish — sourceId:'' put pages under 'default' but stamped edges '', FK-violating against sources(id) and silently dropping the file's whole call graph in the best-effort catch. The unscoped getChunks could also fan out to same-slug chunks from another source. One normalized edgeSourceId (sourceId || 'default') now drives both the chunk lookup and the stamp. * fix(engine): default edge source_id to 'default' at the insert layer (both engines) Adversarial review: addCodeEdges still wrote e.source_id ?? null, so any future caller that forgets the field reintroduces invisible NULL edges the day after the v116 backfill runs. A NULL source_id is invisible to every scoped call-graph query; default to the schema-default source the way the pages table does. Applied to both engines (parity). * fix(core): facts alter recipe NULLs embeddings before cross-dimension alters Adversarial review: buildFactsAlterRecipe shipped the same defect class this branch fixes for content_chunks 350 lines up — a cross-dimension ALTER ... USING cast that pgvector refuses while rows hold old-width vectors. Dimension changes now wipe first (the facts pipeline re-embeds on next write); same-dim type swaps (halfvec <-> vector) keep the lossless cast and PRESERVE data. Both behaviors pinned by tests. * v0.42.39.0 chore: version bump + CHANGELOG + TODOS Marks the v0.42.20.0 'decouple the op-dispatch force-exit timer' follow-up complete — this branch ships exactly that decoupling. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(postgres-engine): atomic JSONB merge in updateSourceConfig — eliminate lost-update race ## Problem `updateSourceConfig` used a read-then-write pattern: read the current `config` row, normalize it in JavaScript, then write the merged result back with `SET config = <normalized> || <patch>`. Under concurrent callers (two background autopilot/cycle paths patching different keys simultaneously), both callers can read the same stale row. The later `SET config = ...` then clobbers the earlier patch, silently dropping whatever keys the first caller wrote. Reproduced at 21/25 lost-update events under real Postgres with parallel callers. ## Fix Fold the normalization and merge into a single atomic `UPDATE … SET config = CASE … END || patch` statement. Because the `SET` expression evaluates against the row-locked latest version of `config`, there is no snapshot window between the read and the write. Concurrent callers now converge correctly (50/50 clean in reproduction test). The `CASE` also normalizes historical bad JSONB shapes inline: - `object` — used as-is - `string` — double-encoded config; inner text parsed with the SQL `IS JSON` guard (Postgres 16+) so unparseable strings fall back to `{}` instead of raising `invalid input syntax for type json` - `array` — array of patch objects aggregated into a flat object via `jsonb_object_agg` - anything else — falls back to `{}` `pglite-engine.updateSourceConfig` already used an atomic `||` merge; this change brings postgres-engine to parity. ## Test Added two assertions to `test/list-all-sources.test.ts`: 1. JSONB string holding non-JSON text normalizes to `{}` (no cast throw) 2. JSONB string holding double-encoded valid JSON is parsed then merged * fix(doctor): five correctness fixes — stale locks, content sanity, graph coverage, exit code, gateway guard ## 1. Stale lock break hints cover gbrain-cycle: keys The doctor stale-lock report only recognized `gbrain-sync:` lock prefixes; everything else fell back to `gbrain sync --break-lock`, which is wrong for dream/autopilot cycle locks. A `gbrain-cycle:<source>` or `gbrain-cycle` lock now suggests `gbrain dream --break-lock [--source <name>]`, and unknown lock shapes fall back to `gbrain doctor` instead of a misleading sync command. ## 2. content_sanity_audit_recent counts reject and quarantine as hard failures v0.42 renamed the hard disposition path: rejected pages emit a `reject` event and quarantined junk pages emit `quarantine`; `hard_block` is now only the pre-v0.42 legacy alias. The status check only counted `hard_block`, so fresh `reject` / `quarantine` events from the new path cleared as `ok` whenever fewer than 10 events existed. The check now sums all three for the hard count, and `soft_block + flag` for the soft count. ## 3. graph_coverage excludes test fixture entity pages from the denominator Brains seeded with code sources (e.g. a sync of the gbrain repo itself) could accumulate test fixture pages typed as `entity` / `person`. Including these in the entity-count denominator diluted coverage and produced spurious warnings ("Entity link coverage 0%, timeline 0%") on knowledge-only brains with no real entity pages. The check now queries a per-entity stats CTE that excludes `tools/gbrain/test/*` slugs and the `templates/new-person` stub, with an additional guard for the all-fixture case (`eligibleEntityCount = 0`). ## 4. process.exitCode instead of process.exit at doctor main exit point `process.exit(hasFail ? 1 : 0)` was a hard kill that prevented cleanup handlers (Bun unload events, open DB connections) from running. Using `process.exitCode = hasFail ? 1 : 0` defers the actual termination until the end of the event loop, allowing cleanup to complete. ## 5. checkSubagentCapability exported for test seams + gateway loop guard The function was private, making it untestable in isolation. It is now exported. Additionally, users running gbrain with a non-Anthropic chat model via `agent.use_gateway_loop=true` no longer receive a spurious warning that `ANTHROPIC_API_KEY` is missing — subagents route via the gateway loop in that configuration and do not need the key directly. ## Tests Doctor test suite: 77 pass, 0 fail (no regressions). * fix(engine): deleteFactsForPage excludeSourcePrefixes (#1928) + reconnect() parity (#2034) Engine-layer API for two cycle/availability fixes that share these files: - deleteFactsForPage gains optional excludeSourcePrefixes so the fence reconcile can protect non-fence facts (e.g. cli: conversation facts). - reconnect(ctx?) is now a first-class BrainEngine method on both engines (PostgresEngine already had it; PGLite gains config capture + reconnect) so callers stop using disconnect()+bare connect(). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cycle): stop extract_facts from wiping conversation facts (#1928) The fence reconcile delete-then-reinsert wiped cli:-origin facts (no fence to recreate them); a failed-sync full walk turned it brain-wide (1829 rows, 0 reinserted, status ok). Now: exclude cli: rows from the wipe, do NOT inherit the failed-sync->full-walk fallback for this destructive phase, and warn on net-negative reconcile. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(autopilot,supervisor): reconnect() instead of disconnect()+bare connect() (#2034) The autopilot health-probe recovery called connect() with no args after disconnect(), losing the startup config (database_url undefined -> FATAL restart-loop on every DB blip) and opening a null-pool window. Both call sites now use engine.reconnect(), which restores the captured config. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(write-through): mirror to the assigned source's local_path, never the global repo (#2018) put_page write-through resolved the disk target from the global sync.repo_path, so a default-source page (local_path NULL) got written into an unrelated federated source's working tree. Now it uses the assigned source's own local_path; NULL local_path skips (no leak); the global path is used only as a sole-source fallback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pglite-lock): heartbeat + steal-grace so live holders are never stolen (#2058) A live holder's lock was force-removed after 5min age alone, letting a second process share the single-writer data dir -> WAL corruption. The lock now heartbeats while held; a holder is reaped only when its PID is dead OR its heartbeat went stale past the steal grace. Pairs PID liveness with heartbeat age to also defeat PID reuse. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(migrate,doctor): self-heal idx_timeline_dedup drift (#2038) A migration renumbered during a merge (v102) could be recorded-as-applied without its DDL running, leaving the 3-column index so every timeline write failed the 4-column ON CONFLICT. runMigrations now always runs a shape-keyed drift repair (dedupe-then-rebuild) even when no migration is pending, and doctor surfaces the drift. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(timeline): un-silence the swallowed batch catch; pin Date-batch round-trip (#2057) The meetings extractor's bare catch {} hid a brain-wide timeline-write failure (0 entries, no error). It now counts + surfaces batch errors. Adds a Date-bearing batch regression test proving the #1861 jsonb_to_recordset refactor already fixed the original ::text[] cast failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: bump version and changelog (v0.42.41.0) Triage fix wave: 6 authored critical fixes (#1928 facts wipe, #2018 write-through leak, #2034 reconnect loop, #2058 WAL lock, #2038 timeline migration drift, #2057 timeline silent-empty) + community PRs #2064 #2052 #2020 #2033 #2074 #2075 #2009 #2072 #2073. TODOS: deferred #1994 #1963 #2050. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address adversarial review findings (#1928, #2058, #2038, #2057) Codex as-built review of the authored fixes surfaced 4 real issues: - #2058: add a pid+acquired_at ownership token. A stale holder reaped + replaced past the grace must NOT let its resumed heartbeat refresh, nor releaseLock remove, the NEW owner's lock (re-opened the concurrent-writer hole). Heartbeat and release now verify the on-disk lock is still ours. + regression test. - #1928: the destructive-full-walk guard keyed off phases.includes('sync'), which wrongly suppressed a legitimate full reconcile when sync was SKIPPED (no engine / no brainDir). Key off a syncAttempted flag set only when sync actually ran. - #2038: dedupe keeps MIN(id) not MIN(ctid) — deterministic and consistent with the existing v-migration lower-id rule. - #2057: the extract CLI caller now surfaces batch_errors (stderr + exit 1) instead of printing a clean success over failed inserts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(key-files): sync reference to v0.42.41.0 triage-wave behavior Update KEY_FILES.md to current-state truth for the shipped fixes (no release-history clauses, per the reference-doc discipline): - write-through.ts (#2018): resolves the disk target from the assigned source's own local_path; sole-source falls back to sync.repo_path, multi-source skips with source_has_no_local_path rather than leak. - engine.ts (#2034): reconnect() is now a REQUIRED lifecycle method on both engines; config-restoring, never disconnect()+bare connect(). - migrate.ts (#2073): document v116 edge source_id backfill + callee index, and the always-run (version-counter-blind) timeline dedup self-heal. - new entry for timeline-dedup-repair.ts (#2038) + the timeline_dedup_index doctor check. - new entry for pglite-lock.ts (#2058): heartbeat + steal-grace (GBRAIN_PGLITE_LOCK_STEAL_GRACE_SECONDS) so a live holder is never stolen. - extract-facts.ts (#1928): cli:-fact protection, no failed-sync full-walk inheritance, net_fact_deletion warn floor. bun run build:llms re-run (KEY_FILES is link-only so bundles unchanged); freshness + current-state guards green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(write-through): preserve nested multi-source layout; narrow #2018 leak guard The first #2018 fix skipped any no-local_path source on a multi-source brain, which broke the legitimate nested layout (a source without its own tree nests under the host repo at .sources/<id>/ — pinned by put-page-write-through.test). Narrow the guard: a no-local_path source nests under sync.repo_path as before; only SKIP when sync.repo_path is literally another source's own local_path (the actual leak — writing there pollutes that sibling's repo). Caught by the sharded suite. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: satisfy test-isolation guard for the new lock/reconnect tests CI `verify` flagged 3 intra-process isolation violations in the tests added this wave (the parallel runner shares one process per shard): - pglite-lock.test.ts: the GBRAIN_PGLITE_LOCK_STEAL_GRACE_SECONDS mutation now goes through withEnv() instead of a raw process.env write (R1). - pglite-reconnect: renamed to *.serial.test.ts — it creates per-test engines to exercise the connect/reconnect lifecycle, which doesn't fit the shared beforeAll-engine model (R3/R4). verify is now 30/30; both files green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pglite): reconnect() is a no-op for in-memory engines (#2034) CI serial-tests + test(5) caught two in-branch regressions from the #2034 PGLite reconnect(): - worker/queue claim-error recovery + their renewLock e2e test assume PGLite reconnect is absent/no-op (queue.ts documents it). Making it a real disconnect+reopen wiped an in-memory engine's state mid-job. reconnect() now no-ops for in-memory (no database_path) — file-backed still re-opens the dir (state persists on disk). Restores the documented worker assumption. - connection-resilience 'Supervisor still has the 3-strikes-then-reconnect path' pinned the removed unsafe-cast text; updated to assert the direct this.engine.reconnect() call. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: quarantine embed-input-type-wire to serial lane (CI test(5) leak) #2033's embed-input-type-wire.test.ts configures a 1280-dim embedding gateway; the active dimension survived into engine-find-trajectory when CI's 10-way hash-disjoint sharding co-located them (this branch's added files reshuffled the assignment), failing 7 trajectory tests with 'expected 1280 dimensions, not 1536'. resetGateway() in afterEach clears the gateway but the dimension still leaked. It mutates global gateway/embedding state, so it belongs in the serial lane (own bun process, true isolation) by the repo's own definition. Root-caused by reproducing the exact failing pair locally. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Austin Arnett <austin@sdsconsultinggroup.org> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Dave MacDonald <djmacdonald@ucdavis.edu> Co-authored-by: pabloglzg <186649799+pabloglzg@users.noreply.github.com> Co-authored-by: Alex P. <12667893+aphaiboon@users.noreply.github.com> Co-authored-by: Garry Tan <bo.m.liu@gmail.com> Co-authored-by: jbarol <barol.j@gmail.com> Co-authored-by: maxpetrusenkoagent <max.petrusenko.agent@gmail.com> Co-authored-by: PAI <pai@scaffolde.ai> |
||
|
|
ecd6ae8772 |
v0.42.40.0 fix(extract,ingest): well-form lone UTF-16 surrogates before jsonb (#2011) (#2031)
* fix(extract,ingest): well-form lone UTF-16 surrogates before jsonb (#2011) excerpt() in link-extraction.ts sliced the link-context window by raw UTF-16 index, so a boundary landing inside a non-BMP char (emoji, math, CJK) left an unpaired surrogate half in `context`. Serialized to JSONB for the jsonb_to_recordset batch insert, Postgres rejects it at the ::jsonb cast and aborts the whole batch — wedging `extract --stale` because the staleness bookmark only advances on a clean finish. - text-safe.ts: new ensureWellFormed() (Bun isWellFormed/toWellFormed) — one shared surrogate-cleaning primitive. - link-extraction.ts: excerpt() well-forms the slice (root-cause fix). - batch-rows.ts: new sanitizeForJsonb() = ensureWellFormed(stripNul(s)) applied to every free-text body field (link context; timeline summary/detail/source; take claim/source). Identity/security fields stay un-sanitized and fail closed. - postgres-engine.ts + pglite-engine.ts: scalar addLink + addTimelineEntry use sanitizeForJsonb too, matching the batch path on both engines. - brainstorm/orchestrator.ts: consolidate hand-rolled sanitizeUnicode onto ensureWellFormed (also fixes consecutive-lone-surrogate mishandling). Tests: ensureWellFormed unit cases (incl. consecutive lone surrogates), an excerpt window-split regression, PGLite + Postgres-e2e surrogate cases across all free-text fields and scalar paths, and fail-closed identity-field tests proving sanitization was NOT extended to slugs/holders. * v0.42.39.0 chore: bump version and changelog (#2011) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: update project documentation for v0.42.39.0 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * v0.42.40.0 chore: re-slot release version (was 0.42.39.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: fix env-mutation isolation violations in retrieval-reflex tests check:test-isolation (rule R1) flags direct process.env mutation in non-serial test files — bun's parallel runner loads multiple files into one process, so a leaked GBRAIN_RETRIEVAL_REFLEX flips reflex behavior in unrelated tests. Both files landed via the #2019 merge; convert the beforeEach/afterEach env juggling to the canonical withEnv() wrapper, which restores the prior value via try/finally even on throw. Fixes the failing `verify` CI check on #2031 (and the `test-status` aggregate that inherits it). All 30 verify checks green locally. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
8f45624e55 |
v0.42.39.0 feat(context): Retrieval Reflex — teach the agent when/what to retrieve (#1981) (#2019)
* fix(integrations): parameterize resolver-row fence by recipe id The install fence was hardcoded gbrain:agent-voice:resolver-rows, so any second copy-into-host-repo recipe wrote a mislabeled block (and refresh/ uninstall keyed on recipe id would miss it). Derive it from manifest.recipe. * feat(context): Retrieval Reflex — teach the agent when/what to retrieve (#1981) Deterministic per-turn pointer layer in the context engine: a zero-LLM, precision-biased scan resolves salient entities (names, @handles) to existing brain pages and injects compact pointers (name → slug → safe synopsis). Detect + point, never auto-dump. Fail-open, capped, suppression on prior context only. Engine-aware resolver ladder (no second DB connection): host ctx.brainQuery → PGLite serve resolve IPC (unix socket) → Postgres cached direct → disabled. Synopsis runs through get_page's privacy strip. Plus the retrieval-reflex recipe + policy skill, the retrieval_reflex_health doctor check, config gate, and the init next-step hint. * chore: bump version and changelog (v0.42.38.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(architecture): KEY_FILES entries for Retrieval Reflex surface (#1981) Document the new src/core/context/ modules, the context-engine resolver ladder, the serve resolve IPC, the retrieval_reflex_health doctor check, and the recipe-id-keyed install fence. Current-state only. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
03ffc6ebdb |
v0.42.37.0 fix(jobs): reap stale locks, bound disconnect, complete cooperative-abort (#1972) (#2015)
* fix(jobs): reap stale dead-holder cycle/sync locks (#1972) A crashed sync (OOM, recycle, SIGKILL) stranded its gbrain_cycle_locks row until something contended for it — reclaim was on-contention only. Add a host-scoped background reaper: reapDeadHolderLocks deletes locks whose holder PID is provably dead on this host, scoped to the gbrain-sync:*/gbrain-cycle* namespaces only (never elections/supervisor/reindex), with a snapshot-matched delete (date_trunc on acquired_at) that is TOCTOU-safe against PID reuse. Reuses isHolderDeadLocally (same-host + ESRCH + 60s grace). doctor --fix now auto-reaps for no-autopilot brains. DRY: selectLockRows + shared mapper now back inspectLock + listStaleLocks (killed the triplication). * fix(db): bound pool disconnect so teardown can't eat CLI output (#1972) pool.end() against PgBouncer transaction-mode never drained, so disconnect blocked until the CLI's 10s force-exit fired and process.exit()'d mid-write, truncating stdout (e.g. #1959's relational query returned empty). Add a gbrain-owned endPoolBounded(pool): Promise.race of pool.end({timeout}) against a hard timer, so teardown is bounded regardless of what postgres.js does and is testable. connection-manager ends its direct + read pools concurrently so the per-pool bounds don't stack. PGLite disconnect is unaffected. * fix(cycle): complete cooperative-abort coverage + wire lock reaper (#1972) v0.42.29 made only the embed phase honor the abort signal; a 24h pull still showed force-evicts from a long non-embed phase ignoring it. Thread the signal into every cycle-reachable long loop: extract (extractForSlugs + the full-walk extractLinksFromDir/extractTimelineFromDir), extract_facts (per-page loop + embed signal + the phantom-redirect 30s lock-retry), and consolidate's bucket loop. Add a terminal abort check so an aborted cycle never stamps last_full_cycle_at as a completed run (Codex #9). lint now yields + checks abort every 200 pages (it's synchronous; the yield is what lets the signal land). New phase-duration force-evict attribution log names any phase that crosses the 30s deadline. Wire reapDeadHolderLocks at cycle start. * chore: bump version and changelog (v0.42.38.0) #1972 — stale-lock reaper, bounded pool disconnect, and complete cooperative-abort coverage across cycle phases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(key-files): current-state for the #1972 reaper, bounded disconnect, abort coverage document-release: update db-lock.ts (reapDeadHolderLocks + selectLockRows DRY), db.ts (endPoolBounded), and abort-check.ts (coverage now spans extract/ extract_facts/consolidate/lint + terminal guard) entries to current truth. * test(isolation): fix shard-order flakes exposed by #1972's new test files Adding 3 new test files reshuffled the hash-based shards, exposing two pre-existing test-isolation bugs: - cycle-consolidate.test.ts assumed the global legacy-embedding preload's 1536-d gateway config still held at initSchema, but a co-sharded test that calls resetGateway() in teardown nulls it, so initSchema fell back to the 1280-d default and built a halfvec(1280) facts column its 1536-d fixtures can't fill. Re-pin the legacy OpenAI/1536 config in beforeAll (the pattern legacy-embedding-preload.ts documents for 1536-d fixture tests). - db-lock-heartbeat-takeover.test.ts (merged from master's #1794) mutated process.env.GBRAIN_LOCK_STEAL_GRACE_SECONDS raw, tripping check:test-isolation rule R1. Convert to withEnv(). --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
1eb430a2df |
v0.42.37.0 fix(security,ingest): source-isolation grant enforcement + non-string frontmatter guard + papercuts (#1999)
* fix(security): scope cross-source reads to the caller grant; close get_page exact-path leak One shared resolveRequestedScope() routes every source-scoped read op (query, code_callers/callees, search_by_image, code_blast/flow, get_page) through a single fail-closed trust+grant ladder: a remote caller's __all__ collapses to its granted sources (never the whole brain) and an explicit out-of-grant source_id is rejected. get_page's exact-match path now honors a federated grant via getPage(sourceIds[]) in both engines. Legacy bearer tokens carry their stored permissions.source_id grant (bounded, never widened). Also retries getConfig on transient connection loss. Closes #1924, #1371, #1393, #1336, #1603. * fix(ingest): non-string frontmatter no longer aborts lint/sync; embed/hook/catalog papercuts Parser coerces a non-string title to a string and falls back to inference for slug/type (never fabricating a "123" slug), with a lint NON_STRING_FIELD finding surfacing the malformed frontmatter; a defensive guard in content-sanity stops a non-string title from crashing the whole lint/sync run brain-wide. Plus: embed --catch-up no longer arms the overflowed 32-bit budget timer (and surfaces unembeddable chunks); the frontmatter pre-commit hook ships a correct .md/.mdx regex; and the skill catalog parses YAML block-scalar descriptions. Closes #1883, #1658, #1556, #1948, #1946, #1840, #1711. * v0.42.37.0 fix(security,ingest): source-isolation grant enforcement + non-string frontmatter guard + papercuts Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: add NON_STRING_FIELD frontmatter validation class to docs for v0.42.37.0 The v0.42.37.0 non-string-frontmatter fix added an eighth validation class (NON_STRING_FIELD / lint code frontmatter-non-string-field). Update the two current-state docs that enumerate the validation classes: - skills/frontmatter-guard/SKILL.md (seven->eight + table row) - docs/integrations/pre-commit.md (seven->eight + table row) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
959af1068d |
v0.42.36.0 fix(sync): resumable, durable, single-flight sync — converges under pool exhaustion + repeated kills (#1794) (#1980)
* fix(retry): match EMAXCONNSESSION + SQLSTATE 53300 as retryable conn errors (#1794) * feat(schema): add op_checkpoint_paths append-only delta table (migration v115) (#1794) * refactor(op-checkpoint): append-only deltas via executeRawDirect + withRetry (#1794) * fix(sync): resumable-checkpoint durability + lock-thrash fix (#1794) Durable append-only checkpoint writes (executeRawDirect + retry), fail-loud consecutive-failure abort, first-file/10s flush cadence, race-safe pending-delta under parallel workers, guaranteed final flush on every exit path incl. SIGTERM (no-retry one-shot via registerCleanup), bankedFiles/reason observability, event-loop yield to keep the lock heartbeat alive, and routing the bare (no-source) sync through withRefreshingLock. * fix(db-lock): heartbeat-aware takeover + direct-pool refresh (#1794) * fix(cycle): treat SyncLockBusyError as skip, not a phase failure (#1794) * docs(sync): document the 5 checkpoint/lock env knobs (#1794) * v0.42.36.0 fix(sync): resumable, durable, single-flight sync — converges under pool exhaustion + repeated kills (#1794) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(key-files): update sync.ts + op-checkpoint.ts entries to resumable-checkpoint current state (#1794) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
612753f318 |
v0.42.35.0 fix(sync): recover from unreachable last_commit instead of full-walking forever (#1970) (#1975)
* v0.42.35.0 fix(sync): recover from unreachable last_commit instead of full-walking forever (#1970) When a source's history is rewritten (force-push, master→main consolidation, squash), the recorded last_commit can fall outside HEAD's history. The old guard sent both "object missing" and "not an ancestor" to performFullSync — a full repo re-walk that never advances the bookmark under a cron timeout on a large cross-region brain, so the source goes silently stale. Fix: only a truly-absent object forces a full reconcile. A present-but-non- ancestor bookmark is diffed tree-to-tree directly (git diff A..B needs no ancestry), importing only the real delta. Adds: oversized-diff fallback to full reconcile (F-B); performFullSync now purges deleted files, gated to file-backed pages by source_path so manual/put_page and metafile pages are spared (F-A); rename-to-unsyncable deletes the stale old page (F-C). 7 new PGLite e2e tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(architecture): record #1970 sync bookmark recovery + full-sync delete reconcile in KEY_FILES Update the sync.ts entry to current truth: entry-time bookmark-reachability guard (gc'd anchor → full reconcile; non-ancestor-but-present → direct tree-to-tree diff), oversized-diff fallback, performFullSync now authoritative for deletes (file-backed pages by source_path), and rename-to-unsyncable delete. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
099d9a8f55 |
v0.42.34.0 feat(search): typed-edge relational retrieval — relationship questions get relationship answers (#1959)
* feat(search): deterministic relational-query parser
Pure, ReDoS-bounded parser that detects relationship queries ("who invested
in X", "who at X works on Y", "who introduced me to X", "what connects A and
B") and maps them to typed edges. Schema-pack-extensible vocab with subset
validation against the link types ingest produces, so query-side and
ingest-side relation vocabularies can't drift. No-match / pronoun-seed /
adjacency guards keep it precision-first (a candidate only; the arm fires
only when a real seed also resolves).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(engine): relationalFanout typed-edge fan-out (both engines)
Generalizes traversePaths to a SEED ARRAY, aggregating to ranked NODES
(shortest hop, edge-richness count, via-link-types, shortest connecting
path, canonical chunk id) instead of edges. Within-source traversal (never
crosses a source boundary even across a federated scope), link_source=
'mentions' excluded by default, deleted_at filtered at seed + every neighbor
+ every node, bounded depth (<=3) + candidate cap. Adds RelationalFanoutRow
/ RelationalFanoutOpts + the relational SearchResult/SearchOpts fields to
types.
Lands in lockstep in postgres + pglite engines, pinned by a DATABASE_URL-
gated parity block in engine-parity.test.ts; a PGLite unit test exercises
the SQL (typed-edge filter, mentions exclusion, deleted_at, canonical chunk,
multi-seed connects, determinism) in default CI.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(search): relational recall arm + federation key hardening
Wires edge-derived candidates into bare hybridSearch as a FOURTH RRF arm
(relational-recall.ts): parse the original query -> scope-aware,
confidence-gated seed resolution (drops fallback_slugify; never traverses
from a guess) -> relationalFanout -> batch-hydrate, reinforcing each page's
REAL canonical chunk (page-level key for chunkless entity pages) ->
--explain attribution + fail-open audit row. Text-only (no-op in image
mode); pure no-op for non-relational queries; rides every downstream stage
(cosine, post-fusion boosts, dedup, reranker, autocut, token budget).
Mode wiring: relationalRetrieval + relational_retrieval_depth knobs
(conservative off; balanced/tokenmax on; depth 2), per-call thread-through
in both bare + cached paths, KNOBS_HASH_VERSION 9->10 (rel=/reld=), config
keys, modes-dashboard descriptions, and a `relational` param on the query op.
Federation hardening (structural, engine-wide): the RRF/dedup key now
carries source_id via a shared rrfKey() (fixes a latent cross-source
collapse where same-slug pages in different sources merged), and the query
cache scopes by a canonical source-set key (cacheScopeKey) so a federated
read can't be served a single-source row.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(eval): relational benchmark + recall@k harness metrics
NamedThingBench harness gains recall@k / recall@10 (the relational headline
metric) on QuestionResult + FamilyReport, plus typed seed/linkTypes/kind on
NamedThingQuestion so the graph-relationship family is machine-checkable.
Adds the relational benchmark corpus (test/fixtures/retrieval-quality/
relational/): a small entity graph whose answers are LEXICALLY UNRECOVERABLE
— every page body is generic and never names the entity it relates to, so
only the typed edge connects query to answer. corpus.ts is the canonical
source for both the seed loader and the 38-question gold set; relational.jsonl
is generated from it (a drift test pins them equal).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(eval): relational A/B proof + arm fires on all retrieval paths
Fixes the integration bug the eval caught: the relational arm was only
injected on the main RRF path, so it silently did nothing whenever vector
was unavailable — no embedding provider configured (the default in many
deployments) OR embed failure. The arm is now built once and fused via RRF
on ALL THREE hybridSearch return paths (no-embedding-provider, embed-failed
keyword fallback, main path). Without this it would have been dead in
exactly the setups that most need it.
Adds `gbrain eval retrieval-quality --ab-relational`: runs the gold set
twice (arm off vs on) in a fixed mode and reports the graph-relationship
recall@10 lift + Hit@3 + latency add. The CI A/B test pins the headline
result — recall@10 jumps from <25% (lexically unrecoverable) to >75% with
the arm on — and a non-relational query returns identical results arm-on vs
off (the no-op / no-regression gate).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.34.0)
Relational retrieval feature: typed-edge recall arm + federation key
hardening. Also updates the KNOBS_HASH_VERSION 9→10 assertions across the
remaining search test files (the bump invalidates relational-off cache rows).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: document typed-edge relational retrieval (v0.42.34.0)
CLAUDE.md Search Mode: add relationalRetrieval to the knob table, the
knobs_hash v=9→10 note, and a relational-retrieval summary. RETRIEVAL.md:
add the relational recall arm to the pipeline diagram. Regenerate llms
bundles (build:llms).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(test): size relational fixtures to the actual embedding-column dim
CI shard runs with the ZeroEntropy gateway default (1280-d), but the
relational test fixtures hardcoded 1536-d embeddings, so chunk inserts were
rejected with "expected 1280 dimensions, not 1536" (CheckExpectedDim) — the
`test (6)` shard + `test-status` failures. The column width tracks the
configured gateway default and can shift with shard order, so fixtures now
probe `content_chunks.embedding`'s actual `atttypmod` after initSchema and
size embeddings to it (the pglite-engine.test.ts pattern), via a shared
`probeEmbeddingDim` helper. Verified passing at a forced 1280-d column.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
|
7cf46230e5 |
docs(designs): add COMMUNITY_IDEAS ledger from open-PR backlog triage (#1969)
Curated, high-bar diary of the valuable ideas surfaced by the community-PR wave, grouped into 10 themes with contributor credit and OPEN/CLOSED/HELD status, so good thinking survives PR closure. Captured during a full triage + hygiene pass over the open-PR backlog. Pure docs; no code impact. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
b31de6613e |
v0.42.33.0 fix(sources): confine sync re-clone to gbrain-owned clones; never delete a user working tree (#1881) (#1960)
* fix(sources): confine sync re-clone to gbrain-owned clones; never delete a user working tree (#1881) recloneIfMissing deleted local_path whenever a source had a remote_url and a non-healthy on-disk state, with no check that gbrain actually created the clone. A source whose local_path was a user's live working tree (remote_url set, no gbrain-created clone) could have its directory removed and re-cloned over. - isOwnedClone(): ownership, not path-containment. True only for a config .managed_clone marker (written by addSource --url) or exact normalized-path equality with defaultCloneDir(id) (back-compat for pre-marker default clones). - recloneIfMissing: ownership guard aborts before ANY filesystem op; EXDEV-safe sibling-temp clone + atomic swap (old aside -> new in -> drop old) with best-effort restore + a message naming where the original is preserved; symlink-leaf reject before the destructive rename. - sync.ts validate_repo_state guards reclone on isOwnedClone (no per-sync warn). - sources restore degrades to a warning for an unowned source instead of the misleading "missing clone, try sync" hint. Tests: #1881 regression (tree survives), isOwnedClone matrix, symlink reject, EXDEV swap residue-free, --clone-dir owned-via-marker, restore CV3, unownedHint healthy/degraded, sync-level refusal. * chore: bump version and changelog (v0.42.33.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: document sources-ops reclone-ownership invariant for v0.42.33.0 (#1881) Add the missing src/core/sources-ops.ts entry to KEY_FILES.md capturing the must-never-violate reclone-ownership guarantee: gbrain only deletes/re-clones a clone it created (isOwnedClone), never a user working tree. Covers managed_clone marker, defaultCloneDir back-compat, EXDEV-safe swap, TOCTOU + symlink-leaf guards, unmanaged_path SourceOpError, and the read-only sources restore path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
5a06af5a57 |
v0.42.32.0 fix(sync): coerce non-string frontmatter titles + bounded auto-skip failure ledger (#1939) (#1956)
* fix(import): coerce non-string frontmatter title/slug/type (#1939) YAML `title: 2024-06-01` parses to a Date and `title: 1458` to a number; the old `(frontmatter.X as string)` cast was a compile-time lie, so downstream `.toLowerCase()` threw and (via the importer failure gate) could wedge sync indefinitely. parseMarkdown now coerces via coerceFrontmatterString (Date -> UTC ISO date, deterministic), and the pure assessContentSanity self-protects against a non-string title. * feat(sync): bounded auto-skip failure ledger; poison file can't wedge indexing (#1939) New src/core/sync-failure-ledger.ts owns the failure store + a crash-safe, multi-source, concurrent bounded auto-skip valve. A file that fails N consecutive syncs (GBRAIN_SYNC_AUTOSKIP_AFTER, default 3) auto-skips so it can't freeze all indexing forever, while fresh failures still fail-closed and a `<head>` history-rewrite sentinel hard-blocks even with --skip-failed. - (source_id, path) keying — failures never merge across sources - success clears a path so attempts are truly consecutive - advance-before-ack ordering (a crash can't mark a file skipped while wedged) - shared applySyncFailureGate used by BOTH the incremental and full-sync gates - legacy-row normalization + duplicate collapse on load - cross-process lock + atomic temp-rename, age-based stale-lock break sync.ts re-exports the ledger for existing callers; import.ts records source-scoped and defers the bookmark to the gate under managedBookmark. * fix(doctor): sync_failures severity via one shared decision on both surfaces (#1939) Local buildChecks and remote doctorReportRemote now both route through decideSyncFailureSeverity, so a stuck bookmark escalates WARN -> FAIL consistently (oldest-open age > fail cadence, or large unresolved count), auto-skipped pages stay visible (WARN, not hidden), and the acknowledged/acknowledged_at field-split that caused drift is gone. The remote surface stays subprocess-free (file read + Date.parse only). * chore(test): add trailing newline to e5-lease-cap-ab baseline fixture * fix(sync): address adversarial review findings on the failure ledger (#1939) - #1: a parse-failed file that is later deleted/renamed-away no longer leaves a permanent open ledger row. Removed paths (filtered.deleted, renamed-from, and the "gone from disk" forward-delete skip branch) are treated as resolved so the ledger self-heals instead of aging doctor to a stuck FAIL. - #3: decideSyncFailureSeverity escalates to FAIL on OPEN (blocking) failures only — auto_skipped rows already advanced the bookmark, so they stay WARN-visible regardless of count, matching the state-machine contract. * chore: bump version and changelog (v0.42.30.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: document sync-failure ledger + auto-skip valve for v0.42.30.0 KEY_FILES.md: new src/core/sync-failure-ledger.ts entry (bounded auto-skip state machine, decideGateAction/decideSyncFailureSeverity/applySyncFailureGate, GBRAIN_SYNC_AUTOSKIP_AFTER); update sync.ts (failure store moved to ledger, re-exported), doctor.ts (sync_failures severity via shared rule on both surfaces), markdown.ts (coerceFrontmatterString), import.ts (managedBookmark). live-sync.md: poison-file auto-skip tricky-spot. Regenerated llms-full.txt. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: re-bump to v0.42.31.0 (queue collision on 0.42.30.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: re-bump to v0.42.32.0 (queue collision) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
f401d7407e |
v0.42.31.0 feat(links): open link_source provenance + link-add/link-rm/link-sources (#1941) (#1957)
* feat(links): relax link_source CHECK to kebab-case provenance + migration v114 Open link_source from a closed allowlist to a kebab-case format gate (^[a-z][a-z0-9]*(-[a-z0-9]+)*$, char_length<=64) so external derivers stamp their own provenance (e.g. citation-graph) without a per-deriver migration. Migration v114: Postgres NOT VALID + VALIDATE (lock-friendly, transaction:false); PGLite plain DROP+ADD. Updates the schema.sql + engine provenance contract comments. (#1941) * feat(links): expose link provenance on link ops + link-add/link-rm/link-sources add_link/remove_link now accept --link-source/--link-type; add_link guards the reconciliation-managed built-ins (markdown/frontmatter/mentions/ wikilink-resolved) and defaults omitted provenance to 'manual' (was the misleading engine default 'markdown'). New cliHints.aliases mechanism with a startup collision guard registers link-add/link-rm; printOpHelp shows the invoked alias name. New list_link_sources read op + listLinkSources engine method (both engines, {sourceId?,sourceIds?}, deterministic order) powers `gbrain link-sources`, added to the minion read allowlist. (#1941) * test(links): kebab provenance, op guard, link-sources, aliases + parity Covers the v114 regex/length boundaries, upgrade-path constraint swap on existing data, the managed-built-in op guard + manual default, remove_link type/source filters, list_link_sources scoping (scalar + federated) and PG/PGLite parity, and alias resolution/collision/help. Fixes the prior 'inferred'-rejection assertion (now valid kebab) in the mentions test. (#1941) * chore: bump version and changelog (v0.42.31.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: update KEY_FILES for v0.42.31.0 link provenance surface KEY_FILES.md current-state updates for #1941: link_source now an open kebab-case provenance (migration v114), the add_link/remove_link guard + defaults, list_link_sources + listLinkSources, and cliHints.aliases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(minions): supervisor queue-singleton keying + pidfile cleanup (follow-up #1849) Two correctness bugs in the v0.42.29.0 supervisor-singleton work, caught by adversarial review: - supervisorLockId mixed a config-derived DB identity into the key, but the lock row already lives inside the target database. Two supervisors on the same physical DB via different-but-equivalent URLs (pooler vs direct port, host alias, trailing params) computed different ids and BOTH acquired the "singleton" lock. Key on the queue alone; the database half of the mutex is physical. Removes the now-dead currentDbIdentity() from worker-registry. - The pidfile-cleanup process.on('exit') listener was installed AFTER the DB-lock acquire, so the LOCK_HELD early-exit stranded the pidfile this process had just created. Install the listener first. Regression test pins the listener-before-lock ordering; updates the lockId test to the queue-only invariant; KEY_FILES updated to current state. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
David BreslauerandGitHub
|
f8d4ce6fc4 | feat(skills): add idea-lineage (#1830) | ||
|
|
613da94093 |
v0.42.29.0 fix(minions): long-job abort-honoring + attempt accounting + supervisor singleton; topic-aware voice (#1737, #1849, #1851) (#1943)
* fix(minions): honest attempt accounting + cooperative abort-honoring + per-handler timeouts (#1737) - Wall-clock and stall dead-letter paths now increment attempts_made (terminal, no retry — wall-clock fires at 2x cumulative timeout; retrying non-idempotent embed/subagent work would duplicate side effects). Surface stalled_counter in jobs get so 'started 3 / stalled 2 / attempts 0' reads true instead of looking like broken accounting. - Thread AbortSignal through embed-backfill/autopilot-cycle -> runPhaseEmbed -> runEmbedCore -> embedAll(Stale)/embedPage, checking it on BOTH --stale and --all paths and between embed batches. A timed-out embed phase now bails within a batch, so the cycle finally releases gbrain_cycle_locks instead of running the full 10-15 min after the job was killed (the daily cycle-wedge). New shared src/core/abort-check.ts (isAborted/throwIfAborted/anySignal). - Per-handler default wall-clock budget (handler-timeouts.ts) stamped at submit for long handlers without an explicit timeout_ms. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(minions): queue-scoped DB supervisor singleton + canonical pidfile + doctor max-rss check (#1849) - Acquire a queue-scoped DB lock (tryAcquireDbLock, keyed on the raw DB identity + queue) on supervisor.start(): a second supervisor on the same (db, queue) fails fast with exit 2 regardless of $HOME/--pid-file. Refresh on a dedicated timer; on refresh failure past the threshold, fail SAFE (exit non-zero) before the TTL could lapse and let a second supervisor take over. Release on shutdown. - Canonical default pidfile keyed on brain id (currentBrainId, config-only, no DB connect) so two brains under one HOME no longer share supervisor.pid. - doctor: new supervisor_singleton check surfaces the effective --max-rss (from the started audit event) and warns when the lock holder's (host,pid) differs from the local pidfile — comparing host+pid, not bare pid. Registered in doctor-categories. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(agent-voice): topic-aware persona context via server-resolved topicId (#1851) Summon Mars/Venus into a specific conversation topic so they boot already knowing the recent thread. A per-topic call link carries only topicId (+ optional display topicName); the server resolves the recent-conversation context from the brain (topics/<topicId>.md) — topic CONTENT is never accepted over the wire (that would be prompt injection + a leak into URLs/referrers/logs). topicId is a strict slug with a path-traversal guard. New '# Topic Context' prompt slot injected after the persona body so identity-first ordering still wins; persona identity unchanged. No topicId -> generic behavior. Contract doc + persona skill docs updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: file #1737 slot-reservation fair-scheduling follow-up TODO (F7) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.42.29.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: sync KEY_FILES.md for v0.42.29.0 minions + abort wave (#1737, #1849) Fold the #1737/#1849 behavior into the existing per-file entries and add the two new core files, keeping the doc at current-state truth: - queue.ts: honest attempt accounting on wall-clock + stall dead-letter paths; defaultTimeoutMsFor stamping at submit. - supervisor.ts: queue-scoped DB singleton lock (supervisorLockId, classifySupervisorSingleton, LOCK_LOST, refresh-fail-safe, brain-id pidfile, max_rss_mb audit). - worker-registry.ts: currentDbIdentity(). - New entries: src/core/abort-check.ts, src/core/minions/handler-timeouts.ts. - New doctor.ts extension: supervisor_singleton check. - cycle.ts / embed.ts extensions: AbortSignal threading note. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
f7f8512b14 |
v0.42.28.0 fix(engine): batch inserts use jsonb_to_recordset, not text[] array literals (#1861) (#1927)
* fix(engine): batch inserts use jsonb_to_recordset, not text[] array literals (#1861) addLinksBatch/addTimelineEntriesBatch/addTakesBatch passed free text through unnest(${arr}::text[]); postgres.js serialized it to a Postgres text[] literal that array_in rejected ("malformed array literal") on calendar/Zoom context, aborting the whole `extract links --stale` sweep. Bind the batch as one JSONB doc via jsonb_to_recordset(($1::jsonb)->'rows') through the audited executeRawJsonb contract instead. Shared row builders (src/core/batch-rows.ts) keep both engines byte-identical; NUL is stripped only from free-text body fields (context/summary/detail/claim), while identity/security fields (slugs/source_ids/holder/kind/dates) still reject NUL. addTakesBatch is now batchRetry-wrapped ('addTakesBatch' audit site) and its BrainEngine signature takes BatchOpts. Scalar addLink context is NUL-stripped too. Regression tests on both engines: PGLite always-on poison/NUL/parity suite + DATABASE_URL-gated Postgres lane (the engine that actually crashed). * test: make "no Anthropic key" tests hermetic via withoutAnthropicKey hasAnthropicKey() reads both ANTHROPIC_API_KEY and ~/.gbrain config; tests that only deleted the env var fired a real LLM call on configured machines (warning flipped NO_ANTHROPIC_API_KEY -> LLM_OUTPUT_NOT_JSON). New test/helpers/no-anthropic-key.ts neutralizes both sources (env + GBRAIN_HOME temp dir) for the duration of the call. Refactors the five no-key tests in think-pipeline + takes-mcp-allowlist to use it, including two that previously passed only by luck of the live LLM output. * chore: docs + version bump (v0.42.28.0) KEY_FILES.md/RETRIEVAL.md describe the jsonb_to_recordset batch path; TODOS.md files the #1861 follow-ups (element-isolation, remaining ::text[] sites, shared SQL-string hoist, batch-insert edge-case tests). CHANGELOG + VERSION + package.json to 0.42.28.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: sync TESTING.md batch-insert references for v0.42.28.0 The #1861 fix migrated links/timeline/takes batch inserts from unnest(::text[]) to jsonb_to_recordset. Update the stale "postgres-js unnest() binding" note and add the two new poison-regression test files (test/links-timeline-jsonb-poison.test.ts PGLite half, test/e2e/jsonb-batch-poison-postgres.test.ts Postgres lane) to the inventory. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sql-query): reject top-level array jsonb params in executeRawJsonb (#1861 P2a) The "no top-level array" rule was only a comment. A bare JS array bound to a $N::jsonb position can serialize as a Postgres array literal (not jsonb) through postgres.js, silently re-entering the "malformed array literal" class #1861 just escaped. executeRawJsonb now throws a clear error steering callers to the { rows: [...] } object wrapper. Verified breaks zero call sites (all pass objects or null). Codex adversarial P2a; batch-size enforcement (P2b) filed as a TODO. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
805814451e |
v0.42.26.0 docs(supabase): update connection-string setup to new UI + Transaction pooler (#1848) (#1875)
* docs(supabase): update connection-string setup to new UI + Transaction pooler Supabase moved the connection string under "Connect" in the top nav and now shows three options (Direct, Transaction pooler, Session pooler). Update the tutorial, gbrain init prompts, the setup skill, the verify runbook, and the live-sync guide to recommend the Transaction pooler (port 6543) — which gbrain is tuned for (prepared statements disabled, DDL/locks routed to a derived direct connection). Document the IPv4 footgun: the derived direct connection is IPv6-only, so on IPv4-only hosts reads work but sync silently skips pages. Tutorial 7c now leads with the free fix (GBRAIN_DIRECT_DATABASE_URL -> Session pooler, port 5432) and keeps the IPv4 add-on as the paid alternative. Removes stale "transaction mode breaks sync (.begin() is not a function)" warnings and the port-6543 "Session pooler" mislabels. Extends PR #1848 by @FilipHarald. * chore: bump version and changelog (v0.42.26.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
9a0bae8d62 |
v0.42.25.0 fix(pricing): unify chat-model pricing into one canonical source; add Opus 4.8 (#1819) (#1827)
* fix(pricing): unify chat-model pricing into one canonical source; add Opus 4.8 (#1819) Single canonical CANONICAL_PRICING table (src/core/model-pricing.ts) with canonicalLookup; ANTHROPIC_PRICING and takes-quality MODEL_PRICING become derived views. cost-tracker, cross-modal runner, skillopt preflight, brainstorm orchestrator, and brain-score all source from it. Adds Opus 4.8 ($5/$25) so --max-cost-usd and the dream-cycle budget meter enforce on 4.8 runs; fixes a stale Opus 4.7 $15/$75 in the takes-quality gate and reconciles Gemini 2.0 Flash to $0.10/$0.40. Because every table derives from canonical, cross-table price drift is structurally impossible. * chore: bump version and changelog (v0.42.25.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: document canonical chat-pricing table for v0.42.25.0 Add KEY_FILES.md entries for src/core/model-pricing.ts (canonical CANONICAL_PRICING + canonicalLookup) and refresh the now-derived anthropic-pricing.ts + takes-quality-eval/pricing.ts entries to current-state. Add the "one canonical chat-pricing table" cross-cutting invariant to CLAUDE.md. Fix the stale model-price snapshot pointer in SEARCH_MODE_METHODOLOGY.md (anthropic-pricing.ts -> model-pricing.ts). Regenerate llms-full.txt. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
f868257405 |
v0.42.24.0 fix(minions): route lock claim/renewLock through direct session pool (#1822)
* fix(minions): route lock claim/renewLock through direct session pool The Minion lock heartbeat (claim + renewLock) ran every UPDATE through engine.executeRaw(), which is hardcoded to the read pool. On Supabase that is the transaction-mode pooler (6543), which recycles connections per transaction. A lock is held for minutes, so the pooler periodically reaps the socket mid-heartbeat -> CONNECTION_ENDED -> the lock looks expired -> the worker force-evicts its own job and the claim loop wedges silently. Add BrainEngine.executeRawDirect(): same contract as executeRaw, but routes to the direct session-mode pool (5432, GBRAIN_DIRECT_DATABASE_URL) when dual-pool is active. No-op delegation on PGLite / non-Supabase / kill-switch. claim/renewLock now use it. Single-statement UPDATEs only, so the double-claim guard and the renewLock no-inline-retry contract are preserved. Statements inside an open transaction keep their tx connection (in-transaction guard keys on peekReadPool() !== _sql); the lock hot-path never runs inside transaction(). The Postgres impl shares its cancellation plumbing with executeRaw via a private runUnsafe helper. New test/postgres-execute-raw-direct.test.ts covers the routing decision (dual-pool on/off x in-tx/not + abort short-circuit) without a live Postgres; queue-lock-retry.test.ts gains a guard that claim can never fall back to executeRaw. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * v0.42.24.0 chore: bump version and changelog Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: update project documentation for v0.42.24.0 Document executeRawDirect on the BrainEngine contract and the claim/renewLock direct-session-pool routing in KEY_FILES.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: make D3 executeRaw-no-retry guard refactor-aware The DRY refactor in this PR extracted executeRaw/executeRawDirect's shared cancellation plumbing into a private runUnsafe(conn, ...) helper, so the single conn.unsafe() call moved out of executeRaw's body. The D3 guard read executeRaw's source and asserted conn.unsafe( appeared exactly once there, which now fails (it's zero — executeRaw delegates). The D3 invariant (no per-call retry wrapper) is unchanged; it just spans the delegate now. Update the guard to check both public methods delegate to runUnsafe without reconnect/retry, and assert the exactly-once conn.unsafe + cancel-only catch in runUnsafe. Also extends coverage to executeRawDirect so the lock hot-path can't reintroduce a retry wrapper either. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |