mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-28 14:59:47 +00:00
ee45653a0220dec2cc32a4881dad351db650cbbf
352
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
ee45653a02 |
fix: initialize foreground chat gateway (#2590) (#3003)
Co-authored-by: Eddie Ash <119880+cazador481@users.noreply.github.com> |
||
|
|
706d3cea3d |
Scope maxWaiting backpressure by data.sourceId (#2970)
The submission-time backpressure cap counts all waiting (name, queue) rows regardless of which source a job targets. On a multi-source brain this makes per-source submissions with maxWaiting: 1 mutually exclusive: while one source's sync sits waiting, every other source's freshness sync coalesces into that row and never runs. The dispatch log shows the starved source 'dispatched' each interval (queue.add returns the other source's waiting row), so the starvation is invisible unless you notice sources.last_sync_at falling behind — we found a secondary source 29 hours stale on a 5-minute freshness interval. Fix: when the submitted data carries a string sourceId, key the advisory lock, the waiting count, and the coalesce target on it. Submissions without sourceId keep the existing single-scope behavior, so single-source brains and non-sync jobs are unchanged. The new test asserts same-source submissions still coalesce while a different source gets its own row and its own cap; it fails on master. Co-authored-by: Forge (Ron) <forge@zsimovan.dev> |
||
|
|
2934c53c1d |
fix(sources): validate --path is a git repo with committed content at registration (#2707) (#2975)
* fix(sources): validate --path is a git repo at registration time (#2707) `sources add --path <dir>` accepted any existing non-git directory with zero validation, deferring the failure to the first `gbrain sync` ("Not inside a git repository: ..."). By the time that surfaces, the source has been silently stale for however long nobody read the sync logs. Add a registration-time check (git-remote.ts:isInsideGitRepo, mirroring sync.ts's discoverGitRoot walk-up so subdir-of-git-repo sources still pass) that rejects an existing-but-non-git --path directory with an actionable error pointing at `git init && git add -A && git commit`. Non-existent paths are unaffected (out of scope — different, pre-existing failure mode) and `--force` opts out for callers who want to register before git-init exists. This is registration-time validation ONLY — it never auto-`git init`s the directory, preserving the consent boundary #2967 established for sync-time self-heal (a --path source is the user's own external directory; gbrain must not mutate it without explicit ask). Also documents the git requirement (docs/guides/multi-source-brains.md), including the "files must be committed, not just present" gotcha and that a stale/unreachable sync anchor already self-heals on plain `gbrain sync` (verified manually against HEAD — no reset-anchor command needed). * fix(sources): require a committed HEAD + shell-quote remediation cmd (codex round 1) Codex review round 1 on #2707 found two real gaps: 1. isInsideGitRepo alone accepts a `git init`ed-but-never-committed directory (rev-parse --show-toplevel succeeds with no HEAD), so registration would still pass a source that fails sync's own "No commits in repo ... Make at least one commit before syncing." Add hasGitCommits (git rev-parse HEAD) as a second required check. 2. The remediation command in the error message interpolated the raw path unquoted — spaces, $(), backticks, etc. would break or, worse, execute unintended shell syntax if pasted. POSIX single-quote it (mirrors src/commands/connect.ts:shellQuote; duplicated locally rather than imported, since commands/ depends on core/ not the reverse). * fix(sources): require tracked content in HEAD, not just a resolvable HEAD (codex round 2) Codex review round 2 P1: hasGitCommits (rev-parse HEAD) accepted a repo with an empty commit (git commit --allow-empty) followed by untracked files — HEAD resolves fine (to git's well-known empty-tree object), so registration passed, but the first sync would "succeed" importing nothing and then silently never notice the untracked files change. The exact same gap applied to an untracked subdirectory of an otherwise- real git repo (monorepo case). Replace hasGitCommits with hasTrackedContent (`git ls-tree HEAD -- .`, non-recursive — one entry is enough, no need to walk the whole subtree). `-C path` + pathspec `.` scopes correctly to both a repo toplevel and a subdirectory-of-a-repo source, and an empty tree lists zero entries where a bare `rev-parse HEAD` would still succeed. Also subsumes the "no commits at all" case hasGitCommits covered (ls-tree on an unborn repo fails the same way), so this is one check instead of two. Updated the error copy and docs/guides/multi-source-brains.md to match what's actually verified now. * fix(sources): O(1)-output tree-emptiness probe, avoid maxBuffer overflow (codex round 3) Codex review round 3 found the round-2 `git ls-tree HEAD -- .` listing buffers the whole (non-recursive) tree — a real repo with ~17-20K directly-tracked entries exceeds execFileSync's default 1 MiB maxBuffer, throws ENOBUFS, and the catch-all incorrectly rejects a perfectly valid registration. Replace the listing with `git rev-parse --verify HEAD:./` (resolves the tree object for `path` specifically, correct for both toplevel and subdirectory sources same as before) compared against git's canonical empty-tree SHA-1 (4b825dc6...) — a fixed ~40-byte read regardless of how many entries the tree has, structurally immune to this class of bug rather than just raising the threshold. Added a 300-file regression test locking this in. Declined a second round-3 finding (P1: reject a tree if ANY untracked file exists anywhere under the path, not just when the tree is entirely empty) — untracked files never being synced is standard, existing git-source behavior throughout this codebase (identical for --url managed clones), not a bug specific to this validation. Enforcing zero-untracked-files at registration would reject ordinary repos with gitignored build output, .DS_Store, editor swapfiles, etc. Out of scope relative to what #2707 actually asks for (a directory with real, committed content that will sync) and how every other git source in this system already behaves. * fix(sources): derive empty-tree OID per repo instead of hardcoding SHA-1 (codex round 4) Codex review round 4 P2, confirmed by directly testing against a `git init --object-format=sha256` repo: the hardcoded SHA-1 empty-tree constant only matches SHA-1 repositories. An empty SHA-256 repo's real empty-tree OID is a different (64-char) hash, so the SHA-1 comparison silently mismatched and let an empty/untracked SHA-256 source through — exactly the case this validation exists to catch. Replace the constant with `emptyTreeOid()`: `git hash-object -t tree --stdin < /dev/null` computed in the target repo's own context, so it returns the correct empty-tree OID for whichever object format that repo actually uses, without gbrain needing to know or care which one. Added gated regression tests (git 2.29+ / --object-format=sha256, test.skipIf on older git) for both the empty-repo-rejected and real-content-registers-fine cases. Converging here (4 review rounds; this is the last outstanding finding from round 4, and round 4 raised only this one issue). * fix(test): --force the incidental non-git second source in #1434 routing test (#2707) CI caught a real regression from this PR's registration-time git validation: test/sync-sole-non-default-routing.test.ts's "2+ non-default sources" case registers a bare mkdtempSync temp dir (no git init) as a second source purely to have 2 sources present — the directory's content was never meant to be exercised, only its existence as a distinct local_path. #2707's new validation correctly rejects that dir at registration time, since nothing else in the test suite told it otherwise. --force is the right fix, not adding unnecessary git-init/commit boilerplate to secondRepo: it documents that this specific registration intentionally doesn't care about git-validity, matching what a real caller opting into the legacy lenient behavior would do. Verified: the specific test (3/3 pass), plus every other test file in the repo using `sources add --path` (sources.test.ts, sources-ops.test.ts already covered by the PR's own commits; repos-alias.test.ts, sync-cost-gate.serial.test.ts — 11/11 pass, no similar fixture gap). typecheck clean, verify 31/31. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
|
d698b44438 |
fix(search): drop compiled_truth from pages.search_vector — was overflowing tsvector on large pages (#2704) (#2977)
A single markdown page whose compiled_truth exceeds Postgres's hard 1,048,575-byte tsvector cap made update_page_search_vector() throw "string is too long for tsvector" INSIDE the pages UPSERT transaction. Not a per-file ledger entry — a transaction abort. The whole source's sync checkpoint stayed pinned (Sync BLOCKED) until the oversized file was fixed or manually skipped, even though every other file in the run imported fine. --retry-failed re-failed the same files every run; the 3-consecutive-failure auto-skip eventually moved past them, but for a scheduled collector that meant hours of blocked cycles per oversized file, per source. Root cause: pages.search_vector indexed compiled_truth — the unbounded whole-page body — even though it's write-only dead weight for actual search. searchKeyword() (postgres-engine.ts / pglite-engine.ts) ranks and queries content_chunks.search_vector exclusively (Cathedral II Layer 3, chunk-grain, already populated separately from compiled_truth via chunking at import time, and well under the tsvector cap since chunkText() targets embedding-sized pieces). Verified directly: no pages.search_vector / bare search_vector read appears anywhere outside this trigger's own definition and the reindex/backfill machinery that maintains it. Fix: v124 migration recreates update_page_search_vector() without compiled_truth — title + timeline stay (both naturally small), so the column keeps carrying some signal rather than going fully inert. Updated in lockstep (documented contract, see reindex-search-vector.ts's own comment): migrate.ts's new v124, reindex-search-vector.ts's recreatePagesFn, and the fresh-install baselines in pglite-schema.ts + src/schema.sql (regenerates schema-embedded.ts via `bun run build:schema`). No backfill: existing rows keep whatever search_vector they already computed until their next UPDATE — harmless, since nothing reads this column, and the brains that actually hit this bug never successfully wrote a value for the oversized page in the first place. Considered (from the issue) and rejected: truncating compiled_truth to fit under the cap. Silent, position-dependent recall loss, and the byte cap doesn't line up cleanly with any natural character/token boundary for UTF-8 content. content_chunks.search_vector already gives full, untruncated chunk-grain coverage for large pages — truncating a now-redundant whole-page vector would trade a real bug for a subtler one. ## Test plan - New test/page-search-vector-overflow.test.ts: a >1MB page (genuinely diverse tokens — a repetitive lorem-ipsum-style fixture does NOT reproduce this bug, since to_tsvector's cap is on its DEDUPLICATED output size, not raw input length) now imports successfully instead of throwing; remains keyword-searchable via the chunk-grain path; a normal page's search_vector still carries title signal (not fully inert). Verified the test is meaningful both directions: fails with the exact reported error on the pre-fix trigger (git-stashed the fix, reran, confirmed byte-for-byte match: "string is too long for tsvector (2684620 bytes, max 1048575 bytes)"), passes with the fix restored. - Updated fts-language-migration.serial.test.ts: removed an assertion that configurable_fts_language (v123) is LATEST_VERSION — that was only ever true until the next migration landed; the codebase's own pattern elsewhere for this (migrate.test.ts) uses toBeGreaterThanOrEqual, not exact-match, for exactly this reason. - bun run typecheck clean, bun run verify 31/31 green. - test/page-search-vector-overflow.test.ts (3/3), test/reindex-search-vector.serial.test.ts, test/fts-language-migration.serial.test.ts, test/migration-v120.test.ts, test/sync.test.ts, test/bootstrap.test.ts, test/migrate.test.ts — 254 total, 0 fail, no regressions. ## Design consultation Investigated jointly with masa-codex (async design review) before implementing — their read of the codebase (content_chunks.search_vector already covers keyword search; pages.search_vector's compiled_truth feed is the only overflow-prone, effectively-dead write) matched independent verification and shaped the "remove from trigger" fix over the issue's alternative options (chunk-grain rebuild — largely already exists; input truncation — rejected above; ledger-entry-only — insufficient alone, since the page upsert failing in the same transaction also loses the chunk write, so checkpoint advancing without this fix would make the content permanently unsearchable, not just delayed). Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
|
4c71a76c0a |
fix(jobs): retry resets started_at/attempts/stalled_counter (#2783) (#2974)
* fix(jobs): retry resets started_at/attempts_made/attempts_started (#2783) `gbrain jobs retry` re-queued a dead job by resetting status/error_text/ locks/delay/finished_at, but left started_at, attempts_made, and attempts_started untouched. On re-claim, claim()'s `started_at = COALESCE(started_at, now())` preserved the ORIGINAL first-claim timestamp instead of re-stamping it. handleWallClockTimeouts() anchors on `now() - started_at`: a retry issued more than timeout_ms * 2 after the original claim was immediately dead-lettered again in under a second, with attempts_made already past max_attempts — making retry useless for exactly the case it exists for (recovering work after an outage that outlasted the job's timeout). An explicit `jobs retry` is an operator asserting "run this fresh", so retryJob now also clears started_at (NULL, re-stamped on next claim) and resets attempts_made/attempts_started to 0. Two new tests: direct assertion that retry resets all three columns, and a full repro of the reported bug (wall-clock-killed job retried long after the original claim now survives re-claim instead of being immediately re-killed). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS * fix(jobs): also reset stalled_counter on retry (#2783) Codex review round 1 found the fix was incomplete: a job dead-lettered by stall exhaustion (handleStalled() at stalled_counter + 1 >= max_stalled) retained its exhausted stalled_counter across retry. The retried job's very first lock expiry after re-claim would immediately re-satisfy the dead-letter threshold, contradicting the same "run this fresh" intent the started_at/attempts reset already established. New test mirrors the existing wall-clock repro: exhaust the stall budget via two real handleStalled() calls, retry, confirm one more stall now requeues instead of dead-lettering again. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
MasaandGitHub
|
354c8c36a9 |
fix(takes): fail closed when takes-write source resolution errors (#2698 follow-up) (#2973)
resolveTakesSourceId() caught every error from resolveSourceId() and fell back to undefined, which restores the pre-#2698 unscoped (cross-source) slug lookup for takes add/update/supersede/resolve. resolveSourceId() only ever throws when a source was explicitly in play (an invalid or unregistered GBRAIN_SOURCE, a .gbrain-source dotfile pointing at a source that doesn't exist, or a genuine DB error) — it never throws for "nothing configured," which resolves cleanly to the seeded 'default' source. So swallowing the error had no legitimate case to protect and only reintroduced the cross-source write bug on any resolution failure. Let it propagate so the write is blocked instead. Adds regression coverage for both the unchanged happy path (no source configured resolves cleanly) and the newly fail-closed path (an unregistered GBRAIN_SOURCE blocks the write instead of falling back to an unscoped lookup). |
||
|
|
dbf2b3f562 |
fix(takes): default takes extraction to the configured chat_model instead of hardcoded cloud Haiku (#2997) (#3021)
extractTakesFromPages hardcoded anthropic:claude-haiku-4-5 as the classifier
model. On an OAuth/local-only install (no ANTHROPIC_API_KEY; chat routed
through a gateway model) every takes extraction died with llm_unavailable —
the takes layer silently never populated and the takes_count health check
stayed red despite a working configured chat_model.
Resolution is now `opts.model || getChatModel()` — the same file-plane
gateway-config idiom enrich.ts uses — NOT engine.getConfig('chat_model')
(the DB config plane), keeping model routing on the single config plane the
rest of the codebase reads. Explicit opts.model still wins; unconfigured
installs fall through to the gateway's DEFAULT_CHAT_MODEL.
Adds a regression test that pins the file-plane read: a conflicting DB-plane
config.chat_model row is ignored, the gateway-configured chat_model is used
when opts.model is unset, and explicit opts.model wins. Verified the
file-plane test fails against the pre-fix code.
Takeover of #2997 by @Nazim22 with the model read moved from the DB config
plane to the file-plane gateway idiom.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Nazz <nazim.mj@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
9ed53e4e1c |
fix(code-edges): per-row $n::text::jsonb binds in addCodeEdges — Bun SQL mis-encodes jsonb[] arrays (#2968) (#3020)
Bun SQL double-encodes ::jsonb[] array binds on the Postgres engine: every edge_metadata element landed as a jsonb string scalar instead of an object, so the resolver's `edge_metadata || jsonb_build_object(...)` UPDATE produced a jsonb array and resolved_chunk_id was never readable — code_callers / code_callees / code_blast returned nothing on Postgres-engine brains while the resolver logged edges_resolved > 0. PGLite was unaffected (per-row placeholders already). Rewrites both inserts (code_edges_chunk, code_edges_symbol) to per-row $n::text::jsonb placeholders via sql.unsafe — the same shape executeRawJsonb and the PGLite engine use. Adds the DATABASE_URL-gated Postgres regression test this class requires (PGLite cannot reproduce it): asserts jsonb_typeof(edge_metadata) = 'object' for resolved + unresolved inserts and that the resolver-style || UPDATE keeps object shape. Verified the test fails 3/3 against the pre-fix code and passes with the fix. Takeover of #2968 by @zsimovanforgeops with the missing regression test added. Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Forge (Ron) <forge@zsimovan.dev> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
MasaandGitHub
|
9f7244a77f |
fix(cli): remove sync --install-cron help text — no handler ever existed (#2795) (#2972)
The top-level `gbrain --help` advertised `sync --install-cron` since the line was first added, but `src/commands/sync.ts` never parsed or handled the flag — `gbrain sync --install-cron` silently ran an ordinary one-off sync instead of installing anything, manufacturing false confidence in the exact durability layer operators reach for it to secure. git blame shows the line was introduced once (v0.42.29.0 help-text scaffold) and never touched again — no design intent to recover. Implementing it would also compete with autopilot, which already owns this job: `gbrain autopilot --install` runs a self-maintaining daemon (sync+extract+embed) on a schedule, including a per-source freshness check that submits `sync` jobs on its own interval. A second, separate sync-only cron would be a competing scheduler outside the D10 cycle-lock invariant that already keeps autopilot's own targeted-submit and full-cycle paths from double-processing. Removed the misleading line and pointed sync's --watch entry at `autopilot --install`, mirroring the existing `dream` command's "See also: autopilot --install (continuous daemon)." pattern one section below. Added regression coverage to test/cli-help-discoverability.test.ts asserting the help text no longer promises install-cron and does point at autopilot. |
||
|
|
6ec3dd410e |
fix(gateway): land Anthropic cache_control breakpoints on the system block, not just the call-level auto marker (#2490) (#2981)
gateway.chat() requested cacheSystem:true but never got a system-prompt cache hit on single-turn callers (page-summary, skillopt, enrich): the call-level providerOptions.anthropic.cacheControl is real (it becomes Anthropic's documented top-level "auto-cache the last cacheable block" shorthand via @ai-sdk/anthropic 3.0.47+), but for a stable system prompt paired with a different user message every call, "the last cacheable block" is that ever-varying tail -- every call writes a fresh cache entry there and never reads a prior one. Fix: pass system as a SystemModelMessage object (ai's documented shape for attaching provider options to the system block) carrying its own providerOptions.anthropic.cacheControl when cacheSystem is requested, and mirror the same marker onto the last tool def (Anthropic caches everything up to and including the last cache_control block it sees). The call-level marker is kept, not removed -- it still gives toolLoop()'s growing multi-turn conversation a rolling cache breakpoint on each turn's tail. All three markers now derive from one canonical cacheControlValue computed after provider_chat_options config merging, so a configured TTL override (e.g. ttl: '1h') applies consistently instead of only reaching the call-level marker. Verified red-before-fix by stashing the gateway.ts diff and confirming the new assertions fail on unfixed code, then restoring. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
|
bcf3b73dcf |
fix(sync): self-heal a never-git-initialized default brain dir (#2964) (#2967)
* fix(sync): self-heal a never-git-initialized default brain dir (#2964) The dream cycle's sync phase throws unconditionally on a legacy sync.repo_path-anchored default brain dir that was never git init-ed (predates git-backed sync, or was rsync'd without its .git), failing every nightly run with no recovery. doctor's sync_freshness/ sync_consolidation checks report "ok" for this exact brain, but only because they query the sources table (0 rows for a legacy default brain) — a coincidental false-negative, not a real diagnosis. Self-heal by git-initializing the dir and capturing the current on-disk state as the sync baseline, scoped to !opts.sourceId only — gbrain owns this directory outright, unlike a registered local source (sources add --path, no --url) which is the user's own external directory and should keep failing loudly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS * fix(sync): dry-run no-write contract, unborn-HEAD recovery, no-gpg-sign (#2964) Codex review on the initial self-heal patch ( |
||
|
|
23e0541d9b |
fix(cycle): budget the patterns subagent from remaining job time — one phase's fixed 35-min worst case defeats any interval-derived cycle budget (#2781) (#2959)
* fix(cycle): budget the patterns subagent from remaining job time, not a fixed constant (#2781) The autopilot-cycle job gets an interval-derived timeout stamped at submit, but the patterns phase submits its subagent with a fixed 30-min job timeout and waits up to 35 min — one phase's worst case exceeds ANY interval-derived budget <= 35 min, so the parent job dead-letters mid-patterns and the tail phases (consolidate -> schema-suggest) starve for days (#2781, the deeper half left open by the #2852 dispatch-floor fix). - MinionJobContext.deadlineAtMs: absolute deadline from the claim-time timeout_at stamp (the DB ground truth handleTimeouts() sweeps against; re-stamped on every claim so retries get a fresh budget). Null when the job has no per-job timeout. - worker: the per-job abort timer now derives its delay from timeout_at when present, so the in-process timer, the DB sweeper, and the handler-visible deadline agree on ONE absolute instant. - autopilot-cycle + autopilot-global-maintenance handlers thread deadlineAtMs into runCycle; CycleOpts carries it to the patterns phase. - patterns: clampSubagentBudgets() derives BOTH the child job timeout and the wait timeout from the same child deadline (parent deadline minus a 60s stop-margin reserve — enough for the wait poll + force-evict grace + cleanup, deliberately NOT a promise that tail phases complete). Under a 2-min minimum the phase skips honestly (insufficient_cycle_budget) instead of submitting a guaranteed-kill LLM call; the next cycle retries with a fresh budget. - Direct callers (gbrain dream) pass no deadline and keep the configured timeouts unchanged. Follow-up (separate PR): synthesize has the same shape plus sequential per-child waits that accumulate N x subagent_wait_timeout_ms past any parent budget; it needs per-wait remaining-time recomputation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RaNDjecPPvhC7LRqkxkhGF * fix(cycle): address review — cancel timed-out patterns child; thread deadline through phase-wrapper handlers - P1: the child's timeout_ms clock starts at ITS claim, so a queued child could outlive the parent deadline the wait was clamped to. On wait timeout, cancelJob strips it (waiting -> cancelled; active -> lock stripped, worker abort fires next renew tick). - P2: makePhaseHandler (standalone patterns/synthesize/... minion jobs) now threads job.deadlineAtMs into runCycle like the autopilot handlers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RaNDjecPPvhC7LRqkxkhGF --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
c873ce3014 |
feat(ai): add Mistral provider recipe (#3001)
Adds an EU-hosted provider covering embedding, expansion and chat on one OpenAI-compatible endpoint (https://api.mistral.ai/v1), so a brain that must stay inside EU jurisdiction does not need a US hop for any AI touchpoint. Every field is measured against the live API, not copied from docs: - mistral-embed is fixed 1024 dims and accepts no dimension parameter. Both spellings are rejected: {"dimensions": N} returns 400 extra_forbidden, {"output_dimension": N} returns 400 "does not support output_dimension". The generic openai-compatible branch of dimsProviderOptions() already falls through to `return undefined` for these model ids, so nothing is emitted. Same contract as voyage-4-nano, pinned by a negative assertion in the test. - max_batch_tokens 65536: a 65,286-token batch is accepted, 66,960 returns 400 code 3210 "Too many tokens overall, split into more batches." - chars_per_token 2: the value is a DIVISOR in splitByTokenBudget() (estTokens = text.length / charsPerToken), so lower is the conservative direction. The module default of 4 assumes English prose; a German-language corpus measured 3.58 chars/token, which the default overshoots toward overflow. codestral-embed is deliberately left out: it returns 1536 dims, and a touchpoint carries a single default_dims. Listing it under a 1024 declaration is the mixed-dim case embedding-dim-check.ts exists to catch. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
4528bfa79c |
feat(cli): GBRAIN_DRAIN_TIMEOUT_MS env override for the per-sink teardown drain budget (#2996)
The one-shot CLI teardown drains fire-and-forget background sinks with a hardcoded 2s per-sink budget (DEFAULT_DRAIN_TIMEOUT_MS). That budget assumes a sub-second cloud chat provider; on a self-hosted provider (e.g. an ollama model at 10-20s per completion) a facts:absorb extraction can never finish inside it, so every one-shot CLI exit — sync timers especially — aborts the in-flight chat with 'pipeline_error: The operation was aborted', and the same touched pages retry-and-abort on every subsequent sync. Facts from those pages silently never land, and doctor's facts_extraction_health warns permanently. Fix: resolveDrainTimeoutMs() — GBRAIN_DRAIN_TIMEOUT_MS env override (same env-only escape-hatch pattern as GBRAIN_TEARDOWN_DEADLINE_MS and GBRAIN_FLUSH_GRACE_MS) over the 2000ms default. Explicit drainTimeoutMs from a call site still wins; computeTeardownDeadlineMs already computes the backstop from the resolved value, so the deadline scales with it. Garbage/zero/negative env values fall back to the default. Tests: default, env override, garbage/zero/negative fallback, finishCliTeardown drains with the env-resolved budget, explicit opts still win over env. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Anton SenkovskiyandGitHub
|
6498b872ea |
fix(extract): --dry-run must not write extract_rollup_7d (#2994)
`extract-conversation-facts --dry-run` promises "no DB writes, no
checkpoint advance" (help text) and correctly skips the fact INSERTs,
orphan delete, checkpoint advance, and receipt page. But
writeRunReceiptAndRollup was called unconditionally at both exit paths,
and its upsertExtractRollup always UPSERTs a row into extract_rollup_7d
("ALWAYS fire so doctor's extract_health sees the cycle ran") — so a
dry run mutates the DB.
Gate both writeRunReceiptAndRollup call sites on !dryRun. The writer
returns void and its only non-rollup action (the receipt page) is
already suppressed in dry-run via facts_inserted > 0, so gating at the
call site skips nothing else. Mirrors the existing !dryRun guards on
the fact-insert / checkpoint / audit paths.
Regression test: a dry run leaves extract_rollup_7d empty. Fails before
(row count 1), passes after. Hermetic PGLite + stubbed transports, no
live LLM.
Note: --dry-run still calls the extractor (LLM) by design — facts_extracted
is the reported "would extract N" preview count; left unchanged.
|
||
|
|
324c355318 |
test(e2e): production guard — setupDB refuses non-test databases (#2957)
setupDB() TRUNCATEs every data table on whatever DATABASE_URL points at, and run-e2e.sh deliberately preserves an exported DATABASE_URL — one stray environment variable away from wiping a production brain, with no guard of any kind (found during an independent review, 2026-07-18). assertSafeE2eDatabaseUrl (pure, unit-tested) now runs before any connection: allowed when the database name carries "test" as a word segment (the gbrain_test convention used by CI and .env.testing.example), or when GBRAIN_E2E_ALLOW_DB names the exact database intentionally. Refusal is loud and actionable. 7 unit tests, no DB required. Co-authored-by: Aleksei Razsadin <razsoc.01@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
184b6cb8a1 |
fix search: title candidate arm + gated OR fallback for lexical recall (#2956)
Pages were unreachable by their own exact titles: FTS indexed only chunk body text while the title-weighted pages.search_vector (GIN-indexed since its introduction) was never queried by any search path, and websearch_to_tsquery AND-at-chunk-grain semantics meant one non-matching token zeroed keyword recall with no fallback — long or acronym-bearing titles (e.g. "IAWG ... AAR-LL deck") fell through to the vector arm alone and missed. - searchTitles (both engines): page-grain candidate arm over pages.search_vector (title 'A' + compiled_truth 'B' + timeline 'C'), ts_rank_cd ranked, representative-chunk LATERAL join, full filter parity with searchKeyword (visibility, soft-delete, source grants, hard-excludes, dates, types); fused as a weighted RRF list at the keyword arm's intent-effective k on all three hybrid return paths; fail-open with warnOncePerProcess. No schema changes — the index already existed, dark. - AND->OR one-retry fallback for the keyword arm, gated behind SearchOpts.orFallback (only hybridSearch opts in; countMentions, link resolution, eval, and keyword-only MCP callers keep the strict-AND contract). Refused for queries carrying websearch operators (negation, quoted phrases). searchTitles carries its own page-grain fallback. - Lexical arms parallelized (Promise.all) on the main path. Verified: typecheck clean; 18 hermetic PGLite tests + 2 engine-parity e2e cases (CI Postgres); consumer regression enrichment 18/0 + link-extraction 127/0; independent live QA on a 10,664-page brain — exact-title target miss -> rank 1 (exact_title_match), controls held, negation/quoted guards proven, strict-consumer contract pinned. Diagnosed from a 3-lane read-only diagnostic; adversarial review round closed findings on fallback scope, Postgres test coverage, and operator handling before this commit. Co-authored-by: Aleksei Razsadin <razsoc.01@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
912407bef1 |
fix(search): per-call token-budget meta masked the real cut on both cache paths; restore vacuous search-lite coverage (#2954)
* fix(search): per-call token-budget meta no longer masks the real cut; restore vacuous search-lite coverage
The search-lite integration tests were structurally vacuous: putPage never
creates chunks and searchKeyword joins content_chunks, so every fixture
query returned zero rows on every machine. The tight-budget cut test's
defensive skip ('keyword search may dedupe by page') silently returned
before its assertions had ever executed anywhere, and the two budget-meta
tests ran against empty result sets.
Restoring the fixture (upsertChunks per page) and hardening the
assertions immediately exposed a real meta bug: with a per-call
tokenBudget, the inner hybridSearch enforces the same resolved budget
(per-call wins in resolveSearchMode) and its meta carries the true
dropped count — but hybridSearchCached re-applies the budget to the
already-cut set and published THAT pass's meta, which always reads
dropped=0. onMeta consumers saw a budget record claiming nothing was
dropped while rows were; telemetry (recorded from the inner meta)
disagreed with the caller-visible meta.
- hybrid.ts finalMeta: prefer innerMeta.token_budget when a per-call
budget is set (outer budgetMeta stays as the fallback and remains the
enforcement for the cache-HIT path, where no inner run exists)
- test fixture: chunk each page (pattern: chunk-grain-fts.test.ts)
- cut test: defensive skip replaced with a hard >=2 precondition;
results non-empty + strictly-fewer-than-unbounded + dropped>0 now
actually execute (revert-checked red on the unfixed meta)
- budget-meta tests: assert non-empty result sets so kept=results.length
can no longer pass vacuously at 0=0
The unbounded 'builder' query returns 2 of 3 fixture pages by design —
dedup Layer 3 caps any single page type at 60% of results and the
fixture is all-person — which the >=2 precondition accommodates.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT
* fix(search): cache-HIT budget meta prefers the stored cut record per review; exact-count assertions, mixed-type fixture
- hit path had the symmetric masking (codex P2): the re-application runs
on the already-trimmed stored set and read dropped=0 while the miss
that produced the same result set reported the real cut. Prefer
hit.meta.token_budget unconditionally — tokenBudget is folded into
knobsHash ('tb='), so a hit only ever serves a lookup with the
identical resolved budget as the write and the outer pass can never
cut further (verified against mode.ts; this is why the reviewer's
'outer wins when it drops' branch is unreachable). budgetMeta remains
the fallback for legacy rows stored without a budget record
- new serial test drives a real store-then-hit roundtrip (mocked
embedQuery, real PGLite cache) and pins hit token_budget == miss
token_budget; revert-checked red (dropped=0) on the unfixed path
- lite test: dropped asserted as the exact unbounded-minus-kept count
and used>0 (dropped>0 alone accepts any wrong positive; used<=250
alone accepts a bogus zero), fixture types mixed (person/company/note)
so dedup Layer-3 type-diversity policy no longer shapes the test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
89f226eb38 |
fix(search): classify cache hit/miss in telemetry — hits were invisible, misses unclassified (#2953)
* fix(search): classify cache hit/miss in telemetry — hits were invisible, misses unclassified (#2952) search stats reported 0 hit / 0 miss forever: recordSearchTelemetry fired only from bare hybridSearch, whose meta never carries a cache field, and a cache HIT returned from hybridSearchCached before any record at all — so hit searches also vanished from count/results/tokens/rank-1. - HybridSearchOpts: internal _telemetryCacheStatus ('miss' | 'disabled') threaded from hybridSearchCached into the inner hybridSearch (same pattern as _queryEmbedDeadline), folded into the RECORDED meta only — onMeta payloads unchanged, count/sum_tokens/budget_dropped/rank-1 behavior byte-identical for the miss/disabled paths - hit path: record once from hybridSearchCached with the already-built cachedMeta (cache.status='hit'), post-slice/budget result count, tokens from the budget pass, and the same rank-1 rule as the inner paths - bare hybridSearch direct callers (think/gather, brainstorm, enrich, evals, ...) keep recording exactly as before, with no cache field - test: serial wiring test drives a real store-then-hit roundtrip through hybridSearchCached (mocked embedQuery, real PGLite SemanticQueryCache) and pins the decision matrix (miss / hit / consult-skipped / bare); revert-checked red on pre-fix source at the miss classification Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT * fix(search): harden cache-hit telemetry per review — mode-gated tokens, embed-failure coverage - hit-path tokens_estimate now gated on the MODE-resolved budget, mirroring the inner paths' resolvedMode.tokenBudget > 0 meta condition (a tokenmax budget-off brain would otherwise record real tokens on hits but 0 on misses, inflating avg-tokens as the hit rate rises) - test: exact token-delta parity assertion (hit contributes the same tokens as the miss that stored the served set) — catches the class of hit/miss accounting asymmetry the increase-only check accepted - test: embed-provider-failure flavor of the disabled path (consult degrades via catch, keyword fallback serves, neither counter bumps) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
f1031d5a0b |
fix(cli): stop thin-client jobs/config from fabricating a scratch PGLite (#2951)
`jobs list|get` have had remote MCP routing since v0.32, but the CLI shell still ran connectEngine() before dispatch — on a thin-client install that fabricates an empty scratch PGLite in the thin-client GBRAIN_HOME and replays the entire migration chain on every invocation, before the remote call even runs. Host-only jobs subcommands (work, supervisor, submit, ...) and `config` did the same instead of refusing. - cli.ts: dispatch thin-client `jobs list|get` engine-free (runJobs(null, ...)); refuse the other jobs subcommands with a pinpoint hint; add `config` to THIN_CLIENT_REFUSED_COMMANDS with a hint (it reads/writes the host brain's config plane). - jobs.ts: widen runJobs to accept a null engine, guarded so null can only reach the MCP-routed list/get branches. - tests: behavioral (no scratch store created, no migration replay, refusals carry hints) + source-audit pins in the existing idioms. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
3a5c4c194c |
fix(remote): poll MinionJob.status, not .state — ping now sees completion (#2950)
submit_job/get_job return the MinionJob row verbatim; its lifecycle field is `status` (src/core/minions/types.ts), not `state`. remote ping typed and read `state`, so every poll saw undefined, the terminal check never matched, and ping always burned its full --timeout and exited 1 even when the autopilot-cycle had completed — printing "Job #N is still undefined." on the way out. Reads fixed to `status`; the ping's own JSON output keys (`state`, `last_state`) are unchanged for consumers. Source-audit regression test pins the field reads. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
d165e99f0b |
fix(brain-writer): deadline race verdict from the sentinel, not the clock the timer raced (#2947)
Part of #2946. The hung-COUNT deadline race derived its verdict from a post-await wall-clock re-check, which races the timer's own drift: on loaded CI runners setTimeout callbacks fire measurably EARLY relative to Date.now(), so each padding/boundary adjustment (>=, +1ms) only moved which wrong status the partial-scan test received ('scanned', then 'partial'). The race's timeout arm now resolves a module-private sentinel; the sentinel winning IS the deadline verdict (deadlineHit), consulted by the post-await check without re-reading the clock. A COUNT that resolves null (failed/absent count) stays distinguishable and does not skip the scan; a COUNT that resolves slowly without the timer winning is still caught by the retained wall-clock re-check. The +1ms pad is gone — the sentinel makes timer drift irrelevant for the hung path. Verified: partial-scan suite green 8 consecutive runs incl. the new null-vs-sentinel distinction test. Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
8b325041ee |
v0.42.63.0 fix: preserve configured PGLite schema database path (#3016)
* fix(schema): preserve configured PGLite database path * chore: bump version and changelog (v0.42.63.0) Co-Authored-By: OpenAI Codex <noreply@openai.com> --------- Co-authored-by: OpenAI Codex <noreply@openai.com> |
||
|
|
a46f28a63e |
fix(cli): keep doctor --json stdout clean — v123 migration handler printed to stdout (#3019)
The v123 configurable-FTS migration (#2941) logged its completion notice via console.log. Migrations run lazily inside any command's first DB connect, so on the nightly heavy run (fresh Postgres service DB) the line landed as the first line of `gbrain doctor --json` stdout and broke the fm_wallclock jq parse ("Invalid numeric literal at line 1, column 7", run 29731426470). runMigrations' contract routes all migration noise to stderr; move the v123 prints (and the pre-existing v2 slug-rename print) there. Also un-vacuous the fm_wallclock harness: its register-source step used `bun run -e` (bun dumps usage with exit 0 instead of running the code) and `connect({})` (in-memory), so the source was never registered and doctor scanned nothing. It now resolves the engine the way the CLI does and registers the source in the DB doctor actually reads. Regression test: test/migrate-stdout-clean.test.ts re-runs migrations from v122 asserting zero stdout writes, plus a source-level guard that migrate.ts contains no console.log. Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
f72de97943 |
feat(sync): --src-subpath + --exclude for monorepo subdir-source support (#753, supersedes #774) (#2942)
* feat(sync): --src-subpath + --exclude for monorepo subdir-source support (#753) Rebased port of PR #774 onto current master. A single git repo can hold N logical sources at subdirectories: git operations run at the discovered repo root (git rev-parse --show-toplevel — worktrees and submodules resolve natively) while file walking, imports, deletes and renames are scoped to the subpath. Passing the subdirectory directly as the repo path (auto-discovery) works through the same code path. Path-containment guards (the point of the feature): - NAV-1/NAV-2: the realpath-resolved scope must live inside the realpath-resolved git root — ../-traversal and symlinked subdirs pointing outside the repo are rejected before any git op. - NAV-1 TOCTOU: per-file realpath re-validation during the incremental import drain and rename reimport; symlink-escape files are recorded as failures (fail-closed — the bookmark cannot advance past an escape). - NAV-4: an --exclude set that filters out every candidate warns loudly. Scoped syncs use git-root-relative slugs + source_path in BOTH the full and incremental paths (runImport gains slugRoot), fixing the original PR's full/incremental slug divergence in the auto-discovery flow. --exclude matches scope-relative paths in both paths; exclusion never deletes previously-imported pages. The full-sync delete reconcile is scope-restricted and relativizes against the slug base so a healthy scoped source can't trip the #2828 mass-delete valve. .gitignore management resolves to the git root at every call site. Preserves all master-side sync work since the original branch: the #2828 mass-delete safety valve, #1794 resumable checkpoints + pinned targets, #1950 stall watchdog, #2335 heartbeat bump, #1970 bookmark reachability, and the git-ls-files walker (#2315/#2462/#2678, whose symlink/cycle hardening is untouched). Co-authored-by: Jeremy Knows <jeremy@veefriends.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: listEverCommittedPaths uses gitContextRoot after #753 root-triple refactor Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Jeremy Knows <jeremy@veefriends.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
f8d11f67a3 |
feat(search): configurable FTS language + reindex command (lands #580/#581/#582) (#2941)
Squashed superset takeover of the FTS-language trilogy by @rafaelreis-r (#580 env-var language for query+write side, #581 migration backfill, #582 gbrain reindex-search-vector), rebased onto current master with the security review's required fixes applied: - Migration renumbered v116 -> v123 (master's v116 was already claimed by code_edges_source_backfill; master is at v122). - Restored the v120/#1647 search_path hardening: all four CREATE OR REPLACE trigger-function bodies (migration handler + reindex command) now carry SET search_path = pg_catalog, public, since CREATE OR REPLACE resets proconfig and would otherwise strip the hardening on upgrade. - reindex-search-vector: shared progress reporter (stderr phases reindex_search_vector.pages/.chunks), id-keyset batched backfill (5000 rows/UPDATE) instead of single whole-table statements, and --json no longer bypasses the --yes/TTY confirmation gate. - Allowlist validation regex + injection tests kept exactly as authored. - Stale v33/v116 comments swept; docs/guides/multi-language-fts.md written (README referenced it but no PR added it); llms bundles regenerated via bun run build:llms. - Trilogy tests quarantined as *.serial.test.ts (env mutation, per check-test-isolation). Verified live on PGLite: fresh init with GBRAIN_FTS_LANGUAGE=portuguese produces portuguese-stemmed vectors with search_path pinned; the reindex command retokenizes an english brain to portuguese in place. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Rafael Reis <rafael.reis@contabilizei.com.br> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
9fe4628d02 |
feat(engine): opt-in Postgres RLS source-scope binding (lands #2387) (#2940)
Takeover of community PR #2387 with the security review's required fixes applied. Original work by @harrisali0101. With GBRAIN_RLS_SCOPE_BINDING=1, source-scoped Postgres read methods wrap their queries in a transaction that binds set_config('app.scopes', $1, true) (federated sourceIds CSV > scalar sourceId > '*') so operator-managed RLS policies can filter rows at the SQL layer — defense-in-depth layer 2 under the mandatory app-layer source filters. Review fixes on top of the original PR: - Flag-off is now a TRUE pass-through: no new per-read transaction wrap (the #1794 PgBouncer pool-exhaustion class). Only the three search methods keep a transaction when off — exactly the sql.begin() + SET LOCAL statement_timeout wrap they already had on master — via the helper's alwaysTransaction option. - Preserved the PR's latent setseed fix: listCorpusSample pins setseed() + SELECT to one connection when seeded (alwaysTransaction gated on opts.seed), so the deterministic path can't split across pooled connections. - Updated the two postgres-engine shape tests to pin the new invariant (search methods route through withScopedReadTransaction with alwaysTransaction; helper owns the sql.begin(); flag-off path is callback(this.sql)). - New behavioral tests (test/postgres-engine-rls-scope.test.ts): flag-off pass-through, flag-off alwaysTransaction, flag-on set_config emission, federated > scalar > '*' precedence, and the CSV as a bound parameter (never interpolated). - Fixed the helper header comment to match the actual branching behavior. - Operator docs in docs/ENGINES.md: env var, policy SQL, the ALTER ROLE ... SET app.scopes='*' default requirement, FORCE ROW LEVEL SECURITY for owner roles, and the honest caveat about unwrapped paths. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Harris <79081645+harrisali0101@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
1833d95896 |
fix(dream,chronicle): synthesize/concepts output family — source scope, output root, retrieval reach, durable provenance, honest judge failures (#1586 #2415 #2163 #2569 #2606) (#2939)
Five verified-open fixes to the dream/synthesize output family: - #1586: thread the cycle's resolved sourceId (cycleSourceId) through runPhaseSynthesize -> SubagentHandlerData.source_id -> subagent tool OperationContext, and stamp collected refs + summary page with the same source, so synthesized pages stop landing in 'default'. - #2415: new config knob dream.synthesize.output_root (default 'wiki', zero behavior change unless set) drives the synthesize prompt slug templates, the patterns reflection lookup + prompt, and remaps the filing-rule allow-list globs. Registered in KNOWN_CONFIG_KEYS. - #2163: synthesize_concepts writes concept pages through importFromContent (put_page's parse->chunk->embed pipeline) instead of bare engine.putPage, so concepts/ pages are chunked + embedded and reachable by retrieval. - #2569: stampDreamProvenance persists dream_generated + dream_cycle_date into pages.frontmatter (JSONB merge via executeRawJsonb) at write time, so generated pages are DB-queryable and put_page write-through can't erase the marker. - #2606: chronicle judge detects stopReason 'length' truncation and no-JSON-array parse failures as distinct skipped reasons (judge_truncated / judge_parse_failed) instead of a false terminal no_events; output cap raised to 4000 and configurable via chronicle.judge_max_tokens. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Garry Tan <garrytan@gmail.com> |
||
|
|
42375bded5 |
fix(sync): stop the sync data-loss family — ops/ prune, DB-only write-through, full-sync gate drift (#2404, #2426, #2607) (#2938)
Three verified-open defects, one family: sync silently destroying or diverging on content it should preserve. #2404 (P0) — 'ops' was hardcoded in PRUNE_DIR_NAMES (a v0.2.0-era carve-out), so any path with an ops segment was 'pruned-dir': committed ops/*.md never imported, and modified ops/* files hit the unsyncableModified delete loop (whose #1433 guard only spared 'metafile'), silently deleting put-created pages like the bundled daily-task-manager's canonical ops/tasks on every sync. Fix: remove 'ops' from the prune list (ordinary user content; the vendor/generated entries stay), and harden the delete loop to also skip 'pruned-dir' — a page under a pruned dir can only exist via a deliberate put_page. #2426 (P0) — write-through content stayed DB-only and was deleted by sync --full. All three compounding bugs fixed: 1. writePageThrough now best-effort commits the artifact (path-limited git commit) on durability-hardened repos, so the post-commit hook can push it; result carries committed?: boolean. 2. scripts/brain-commit-push.sh stages+commits BEFORE any pull — the old fetch+pull-rebase-first order aborted on any dirty tree, so the helper could never commit a MODIFIED page; brain_push's rebase-on-reject already handles an advanced remote. 3. The full-sync delete-reconcile partitions stale pages by git history (listEverCommittedPaths): never-committed source_paths are DB-only write-through — pages are KEPT and re-exported to the working tree instead of soft-deleted. Builds on the #2828 mass-delete valve (covers the below-valve cases). #2607 — the sync --full git ls-files fast path bypassed pruneDir, so a full pass imported (and resurrected soft-deleted) pages under dot-dirs and vendored trees that incremental sync excludes. Fix: isCollectibleForWalker applies the same segment-level pruneDir gate as classifySync, so full and incremental enumeration agree. One regression test per defect (all verified failing against master src): test/sync-ops-pages.serial.test.ts, test/write-through-commit.serial.test.ts, the #2426 helper-order test in test/brain-durability-hook.serial.test.ts, test/sync-reconcile-db-only.serial.test.ts, test/import-git-fastpath-prune.test.ts. Fixes #2404 Fixes #2426 Fixes #2607 Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
KonradopenclawandGitHub
|
a8e6b1d177 | feat(ai): add Moonshot Kimi provider recipe (#2378) | ||
|
|
54a8070640 |
fix(facts): make dream extract_facts idempotent so fence rows don't duplicate each cycle (#2932)
Port of #1837 (mvanhorn) onto current master. The extract_facts cycle phase unconditionally wipe-and-reinserted every page's fence-owned DB rows, so re-running a cycle on unchanged content churned rows and — on the Postgres engine reported in #1781 — accumulated duplicates each run. The phase now de-dupes extracted facts by the canonical (claim, source) content key and reconciles the page-scoped DB index: no-op when already in sync, insert-only for new keys, wipe/reinsert only when stale rows need cleanup. Adjustments over the original PR to fit current master: - preserve #1972's abortSignal threading into the batch embed call - preserve #1928's excludeSourcePrefixes: ['cli:'] on every wipe, and exclude cli:-origin conversation facts from the existing-row set so they neither count as stale (which would force a wipe every cycle) nor get compared against the fence Fixes #1781 Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e1cefd0654 |
fix(subagent): orchestration fix-wave G — configurable timeouts/caps, honest child outcomes, fenced timeline writes (#2937)
Four verified-open issues in the subagent-orchestration family, one PR: - #1594: dream synthesize subagent job/wait timeouts promoted from hardcoded 30/35-min constants to config keys dream.synthesize.subagent_timeout_ms / subagent_wait_timeout_ms. Approach ported from stale PR #1596 (credit @ai920wisco). - #2778: add_timeline_entry joins the subagent brain-tool allowlist, fenced fail-closed by the shared enforceSubagentSlugFence (extracted from put_page's inline check — same trusted-workspace allow-list / wiki/agents/<id>/ namespace policy). The per-turn output cap is now resolveMaxOutputTokens (data.max_tokens → agent.max_output_tokens → 8192, was hardcoded 4096); a max_tokens stop surfaces as stop_reason 'max_tokens' instead of a silent end_turn, and a mid-tool-round cap hit injects a truncation note so the model re-issues the dropped call. - #2782: patterns phase status now reflects the child outcome — non-complete outcome with zero writes → fail (PATTERNS_CHILD_<OUTCOME>), partial writes → warn. Patterns timeouts get the same config-key pair (dream.patterns.subagent_timeout_ms / subagent_wait_timeout_ms). - #2113: facts extraction cap is config facts.extraction_max_tokens (default 4000, was hardcoded 1500); stopReason 'length' is checked, retried once at 2x the cap, and surfaced on stderr instead of silently extracting zero facts. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: ai920wisco <ai920wisco@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a0ef951586 |
fix(dream): gate patterns phase on gateway provider reachability, not ANTHROPIC_API_KEY (takeover of #2279) (#2936)
Absorbs PR #2279's intent (drop the hardcoded ANTHROPIC_API_KEY gate) with the end-state the gateway world actually wants: the patterns phase now probes the RESOLVED patterns model through probeChatModel(normalizeModelId) — the same semantics as think/index.ts and synthesize's makeJudgeClient. Fixes two misclassifications of the old env gate: - Non-Anthropic stacks (litellm, deepseek, openrouter, ...) were skipped as "no upstream" even though the subagent routes them through the gateway (agent.use_gateway_loop). They now pass the gate; their auth is checked lazily at dispatch and surfaces in the job outcome. - Anthropic keys set via `gbrain config set anthropic_api_key` (stdio MCP launches without shell env) were treated as missing. hasAnthropicKey inside probeChatModel reads both sources. Skip reason renames no_api_key → no_provider (carrying the probe's detail). Both pinning tests updated: the structural test asserts the probe wiring; the PGLite E2E swaps its env-only helper for the shared hermetic withoutAnthropicKey (env + config file) so it can't flip to a live LLM call on a dev machine with a config-file key. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: brettdavies <brettdavies@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
bd2ba46a61 |
fix(retry): reconnect on null instance pool in ALL non-batch config accessors (takeover of #1891) (#2934)
Ports the still-unmerged remnant of PR #1891 (#1593 follow-up). Master's getConfig gained retry-with-reconnect in #1603, but its siblings — setConfig, unsetConfig, listConfigKeys — still touched `this.sql` bare, so the first config write/list after a mid-cycle instance-pool teardown threw the retryable "No database connection" (issue #1678) unhandled instead of rebuilding the pool. Adds the connRetry helper from #1891 (same retry+reconnect posture as batchRetry, but no batch audit JSONL — a config accessor is not a sized batch), refactors getConfig onto it, and wraps the other three. Writes are safe to retry: withRetry only retries connection-class failures and both writes are idempotent (upsert / delete). Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: jalagrange <jalagrange@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
2df41a84c9 |
feat(ai-gateway): derive OpenAI prompt_cache_key for native-OpenAI chat models (takeover of #2442) (#2933)
Ports the still-unmerged half of PR #2442. OpenAI caches prompt prefixes automatically, but a stable prompt_cache_key keeps requests that share a prefix on the same inference engine, lifting the automatic-cache hit rate. chat() now derives a stable key from the system prompt + sorted tool names for native-openai models and passes it via providerOptions.openai. promptCacheKey. Config provider_chat_options still overrides the derived key; anthropic/google/openai-compatible providers are untouched. The Anthropic half of #2442 (cache_control "silent no-op") is superseded: @ai-sdk/anthropic 3.0.74 forwards call-level providerOptions.anthropic. cacheControl as the Messages API's request-level cache_control (automatic prefix caching), so master's existing marker is live on current deps. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: CoachRyanNguyen <CoachRyanNguyen@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
da1bab532a |
fix(import): make checkpoints staging-first — canonical dir identity + self-describing metadata (#2935)
Port of #1731 (diazMelgarejo) onto current master. gbrain import wrote ~/.gbrain/import-checkpoint.json with the caller's raw dir argument, so a checkpoint left behind by an interrupted run (e.g. SIGTERM) could carry "." or a symlinked spelling — an identity that resolves to whatever CWD the next consumer happens to run from. Downstream tooling that treated the checkpoint dir as an owned staging boundary could then act on the wrong directory. - runImport captures the import target ONCE via resolveImportTargetDir (resolve + realpathSync) and threads that canonical value through collection, checkpoint load/save, and resume filtering - checkpoints are self-describing (schema_version: 1, owner: "gbrain", kind: "import"); loadCheckpoint tolerates absent metadata (legacy path-based files) but rejects present-and-wrong metadata and any relative dir - checkpoint contract documented in docs/guides/live-sync.md (llms bundle regenerated) - test/import-resume.test.ts fixture now realpaths its tmpdir so planted checkpoints match the canonicalized dir (macOS /var -> /private/var) Fixes #1728 Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Lawrence Melgarejo <Lawrence@cyre.me> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
b075a9c8d4 |
test+ci: unbreak master — symlink-walker probe files + OSV caller permissions (#2926)
* test: symlink-walker tests use a non-metafile probe (README now skipped by design, #2315) The import walker deliberately skips README/metafiles since #2315 (closing #345); the symlink-hardening tests used README.md as their probe file and went red on the intersection. Probe with notes.md instead; test intent (cycle hardening + strategy filter) unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci: grant security-events write to the OSV caller job — reusable workflow requires it at startup Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
BrettandGitHub
|
a31f16f471 |
fix(markdown): treat # lines inside closed frontmatter as YAML comments, not headings (#2153)
`parseMarkdown` previously walked the lines after the opening `---` and
recorded the first `^#{1,6}\s`-shaped line as a `headingBeforeClose`,
then flagged MISSING_CLOSE when that index came before the actual closing
fence. YAML allows `#` comment lines anywhere inside the document, so a
template that leads with annotation comments inside the fence (e.g. a
`# Research Template` header before the keys) hit a false-positive
MISSING_CLOSE even though the closing `---` was present.
Fix: only walk for the closing `---`. When it is found, content between
the fences is YAML; `#` lines are comments, not headings. When the close
is genuinely missing, surface the first heading-shaped line as a
where-it-went-off-the-rails hint (this path was already correct; we keep
it for the genuine missing-close case).
Two regression tests added to `test/markdown-validation.test.ts`:
- `#` comment lines at the top of a closed frontmatter
- `#` comment lines interleaved with keys
All 68 tests across the four markdown/frontmatter test files stay green.
|
||
|
|
7ffac65c62 |
fix(extract,ingest,cycle): source-provenance wave — thread source identity through ingest_capture, fs-walk links, and cycle extract (#1522 #1747 #1503) (#2920)
* fix(extract,ingest,cycle): source-provenance wave — thread source identity through ingest_capture, fs-walk links, and cycle extract (#1522 #1747 #1503) Three fixes in the same invariant class (source identity silently dropped on the write path, collapsing to the 'default' source): - #1522: the ingest_capture Minion handler validated IngestionEvent provenance (source_id/source_kind/source_uri) then dropped it on the importFromContent call. Now threads source_kind/source_uri + ingested_via='ingest_capture' into the page write, and routes the write under event.source_id when it names a registered source AND the event is trusted (fail-closed: untrusted webhook payloads carry a caller-controlled x-gbrain-source-id header and must not choose their write source; unregistered emitter ids keep default routing so the webhook path can't FK-fail). - #1747: the fs-walk extractors (extractLinksFromDir / extractTimelineFromDir / extractForSlugs) built batch rows with no source_id, so addLinksBatch/addTimelineEntriesBatch mapped missing → 'default' and the pages JOIN dropped every row on a non-default source ("Links: created 0 from N pages", no error). ExtractOpts gains sourceId; the CLI fs path resolves it via resolveSourceId (--source-id > env > dotfile > registered path > sole-non-default) and rows are stamped from/to/origin_source_id + timeline source_id. - #1503: the cycle's extract phase (runPhaseExtract) never passed a sourceId, so federated-brain dream/autopilot cycles persisted nothing every night. It now threads cycleSourceId (explicit --source or resolveSourceForDir(brainDir) — the same seam runPhaseSync uses) into runExtractCore for both the incremental and full-walk paths. Regression tests: handler provenance write-through (registered/ unregistered/untrusted routing), fs-walk + CLI resolution + incremental cycle route landing edges/timeline in the right source (all red on unfixed master), and a negative control pinning the pre-fix JOIN-drop shape. Reuses the ExtractOpts.sourceId threading approach from PR #1719, rebased onto the current signal-aware signatures and extended to the cycle + timeline paths. Fixes #1522 Fixes #1747 Fixes #1503 Co-authored-by: seungsu <kss530c@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: update cycle source pin for cycleSourceId threading The #1972 source-pin test asserts the literal runPhaseExtract call site in cycle.ts. The #1503 fix appended cycleSourceId after opts.signal; signal threading is unchanged (still the 5th arg, forwarded to runExtractCore). Pin updated to the new literal — invariant intact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: seungsu <kss530c@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
maxpetrusenkoagentandGitHub
|
b263d9bc20 |
fix(minions): reconnect worker after promote connection loss (#2025)
Recover the worker-owned Postgres pool when promoteDelayed escapes a retryable connection error, preventing the repeated Promotion error: No database connection loop from issue #1491.\n\nAdds a regression test proving reconnect happens before the worker continues to claim work. |
||
|
kubiandGitHub
|
73bbbde01d | fix frontmatter scans to respect git excludes (#2462) | ||
|
|
ff8ce4d764 |
fix(import): walker skips SYNC_SKIP_FILES metafiles so import and sync agree (#2315)
Closes #345. The bulk-import walker isCollectibleForWalker filtered admitted files by extension only, while incremental sync excludes README/index/log/schema via isSyncable -> SYNC_SKIP_FILES. A directory import therefore ingested every directory README as a folder-titled ghost page that trigram-corrupts fuzzy entity resolution and inflates orphan count. Apply SYNC_SKIP_FILES (basename guard) at the top of isCollectibleForWalker so both the FS-walk and the git-fast-path collection routes agree with sync. Also add RESOLVER.md to SYNC_SKIP_FILES: a structural routing metafile (docs-aligned with schema.md/index.md/log.md/README.md), not indexable content. Co-authored-by: ElliotDrel <ElliotDrel@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
9eac872136 |
fix(postgres-engine): build-then-swap reconnect() so a failed rebuild can't brick the engine (#1593) (#1906)
The instance-pool reconnect() did disconnect() (nulling _sql) BEFORE connect(), so a connect() failure during a transient Postgres blip left _sql null for the rest of the process — every subsequent non-retry-wrapped call then fell through to the never-connected module singleton and threw 'No database connection', crashing the autopilot worker into a respawn loop. Build-then-swap: snapshot the live pool, build a fresh one, end the old only once the new validates, restore on failure. Keeps upstream's reap-detection + pool-recovery audit. Confirmed by jalagrange on closed PR #1593; this is the Layer-1 fix he left to us. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
xd-NejiandGitHub
|
cfc120fcb3 | fix(stats): exclude soft-deleted pages from visible counts (#2235) | ||
|
Mersad AjanovicandGitHub
|
0a021f6f6b | fix: send admin SSE cookies through reverse proxies (#1560) | ||
|
mzkaramiandGitHub
|
cd9bd3f731 | fix(auth): add register-client agent binding flags (#1976) | ||
|
|
3a2033e8e2 |
v0.42.52.0 fix(sync): bump last_sync_at heartbeat on 0-changes sync (#2335)
D4 invariant ("never advance last_commit on partial", sync.ts comment)
preserved. last_sync_at is a monitoring signal read by doctor
sync_freshness (warn 24h / fail 72h), separate from the import-converged
bookmark. Without this heartbeat write, a cron-driven */15 sync over a
quiet vault pins last_sync_at to the last real commit, so doctor
falsely flags the source as stale for as long as the vault is quiet.
Reproduction (5 lines, no gbrain install required):
1. Setup a fresh obsidian source + commit a single .md file
2. gbrain sync --source obsidian # first_sync, last_sync_at = NOW
3. (do nothing) gbrain sync --source obsidian # up_to_date
4. SELECT last_sync_at FROM sources WHERE id = 'obsidian';
5. Observed: still pinned to step 2. Expected: bumped to step 3.
Fix: in the up_to_date early-return (sync.ts line ~1786), execute a
single `UPDATE sources SET last_sync_at = now() WHERE id = $1` before
returning. The D4-protected writeSyncAnchor path is untouched.
Test: test/sync.test.ts adds a describe block that runs two
consecutive syncs against a quiet vault and asserts last_sync_at
advances while last_commit is unchanged. PGLite + executeRaw pattern
matches the existing #1970 test scaffold.
Workaround: hourly psql touch in WSL crontab documented at
https://github.com/garrytan/gbrain/issues/[link-to-issue]
Co-authored-by: lost9999 <lost9999@users.noreply.github.com>
|
||
|
|
78bc2fef09 |
fix(cli,config,doctor): CLI/config UX wave — config-get file plane, idempotent archive, honest help + doctor text (#2120 #2792 #1175 #1123 #2451) (#2918)
* fix(cli,config,doctor): CLI/config UX wave — config-get file plane, idempotent archive, honest help + doctor text, prefixed model defaults (#2120 #2792 #1175 #1123 #2451) - config get resolves the file/env plane before the DB plane (runtime precedence) and reports provenance on stderr; stdout stays a bare value. - sources archive distinguishes already-archived (friendly no-op, exit 0) from not-found (clear exit-4 error). - gbrain --help SOURCES block now lists archive/restore/archived/purge/status plus a pointer at `sources --help` for the long tail. - multi_source_drift doctor advice references only real CLI surfaces (drops the never-built 'sources rehome'; pins delete to GBRAIN_SOURCE=default). - #2451 (bare model ids in calibration defaults) verified already fixed + tested on master by #2892 — no change needed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: satisfy test-isolation gate for wave-B tests check-test-isolation R1 forbids direct process.env mutation in non-serial unit tests (env leaks across files sharing a shard process). Route the GBRAIN_HOME / GBRAIN_CHAT_MODEL / GBRAIN_PGLITE_SNAPSHOT overrides through the canonical withEnv() helper instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
26d2f8abfc |
fix(calibration,takes,cli): calibration CLI routing, source-scoped takes reads, BigInt-safe outputs (takeover of #2452) (#2892)
Rebase-port of #2452 (spinsirr:fix/calibration-profile-scope-and-cli) onto current master after tonight's merges made the fork branch conflict. - cli: add 'calibration' to CLI_ONLY so dispatch reaches its existing handler instead of falling through to "Unknown command" (#2035); honor --source / GBRAIN_SOURCE in the calibration CLI. - takes: route takes_list / takes_search / takes_scorecard / takes_calibration through sourceScopeOpts(ctx) (federated array > scalar > nothing) and scope engine reads via the take's page.source_id — JOIN filter for list/search, EXISTS for scorecard/curve — on both engines (#2200-class). - bigint: shared takeHitRowToHit coercion in searchTakes / searchTakesVector (both engines) + bigintToStringReplacer on the cli.ts output normalizer and the `gbrain call` exit, so int8/BIGSERIAL columns no longer crash JSON.stringify (#2450); calibration profile id BIGSERIAL → string. - calibration: default model ids route through TIER_DEFAULTS (provider-prefixed) instead of bare model strings; admin calibration chart endpoints fixed (takes has no page_slug column; month-precision since_date; Date generated_at; bigint id in drill-down). The think/gather source-scope slice of the original PR was dropped: it already landed on master via #2739. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: spinsirr <ID+spinsirr@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
2166545849 |
fix(pglite): platform-gate the init-failure banner — stop blaming the macOS 26.3 bug on every platform (#2674) (#2891)
classifyPgliteInitError() routes bare Emscripten aborts to the 'unknown' verdict, whose hint unconditionally printed "Most common cause: the macOS 26.3 WASM bug (#223)" — even on Windows and Linux (#2195, #1870). - buildPgliteInitErrorMessage now takes a platform param (default process.platform): darwin keeps the #223 link as a *possible* cause; other platforms get the plausible off-macOS causes (lock contention, damaged data dir) plus `gbrain doctor` / `gbrain reinit-pglite`. - New stringifyPgliteInitError(): non-Error rejections (Emscripten aborts can throw plain objects) no longer print "[object Object]". - Regression tests for both branches + the stringifier in test/pglite-init-classifier.test.ts. Canonical for a 7-report class: #2674, #1870, #1195, #1502, #2195, #939, #391. Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |