diff --git a/CHANGELOG.md b/CHANGELOG.md index 71555c843..662838cf5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,28 @@ All notable changes to GBrain will be documented in this file. +## [0.42.45.0] - 2026-06-13 + +**The daily sync cron stops wedging on cost, and the embedding-spend estimate finally matches what a sync actually does (gbrain#2139).** On an active brain the inline-embed cost gate priced the *entire* corpus every time the working tree was dirty — which is always, since agents and crons write to it constantly — so a routine daily sync estimated ~158M tokens / ~$8 when the real delta was a few hundred files / ~$0.04, then blocked the cron with a confirmation it could never answer. Embeds silently stalled until someone noticed. The estimate now mirrors execution: it fetches first and prices only the files this run will pull and import, through the same diff machinery the sync itself uses. A brain whose commits are caught up but whose tree is dirty estimates $0, because an attached-HEAD sync imports only the committed diff. + +When the gate does fire in a non-interactive session, it no longer exits with an error — it imports now and defers embedding to capped background jobs (which drain via the jobs worker or `gbrain embed --stale`), so a cron is never wedged again. Operators who have decided cost isn't the constraint get one switch — `gbrain config set spend.posture tokenmax` — that makes every cost gate informational across sync, reindex, enrich, and onboard (spend is still recorded; the switch removes the ceiling, not the accounting). The USD knobs accept `off` / `unlimited`, and every gate message now carries paste-ready commands so the controls are discoverable at the moment they fire. + +This release also lifts the rule that blocked `--skip-failed` / `--retry-failed` under parallel sync — failure recovery no longer has to drop to `--serial` (which is what armed the inline gate in the first place). + +### Added +- **`spend.posture` config** — `tokenmax` makes every embedding-cost gate informational (print the estimate, proceed, keep the ledger); `gated` (default) enforces as before. Documented end-to-end in `docs/operations/spend-controls.md`. +- **First-class off switches** — `sync.cost_gate_min_usd`, `embed.backfill_max_usd_per_source_24h`, `embed.backfill_max_usd`, and `reindex --max-cost` / `enrich --max-usd` accept `off` / `unlimited` / `none`. No more sentinel values like `100000`. +- **Single-source `gbrain sync` cost preview** — plain `gbrain sync` previously embedded inline with no preview; it now carries the same gate as `sync --all` (auto-defers in non-TTY sessions, never blocks). +- **Self-describing gate messages** — every cost-gate / FYI line ends with the exact `gbrain config set` commands to widen, disable, or switch posture, plus a docs pointer. + +### Changed +- **`gbrain sync --all` is no longer blocked by the cost gate in cron/agent contexts.** Above the floor in a non-interactive session it auto-defers embeds (exit 0) instead of emitting a `cost_preview_requires_yes` envelope and exiting 2. Cron wrappers that branched on exit 2 now see exit 0 with `status: "auto_deferred"`. A TTY still prompts `[y/N]`; `--yes` still embeds inline. +- **`--skip-failed` / `--retry-failed` now work under parallel sync.** The failure ledger is per-source and lock-serialized, so the previous "not supported under parallel — re-run with --serial" refusal is retired. +- **The six spend-control config keys are now first-class** (`gbrain config set` accepts them without `--force`). + +### To take advantage of v0.42.45.0 +`gbrain upgrade`. Nothing to configure for the headline fix — the daily sync cron stops wedging and the estimate is accurate out of the box. If you run a high-volume brain where cost genuinely isn't the constraint, `gbrain config set spend.posture tokenmax` makes every gate informational. To widen or disable a specific gate instead, see the table in `docs/operations/spend-controls.md` (e.g. `gbrain config set sync.cost_gate_min_usd off`). Failure-recovery syncs can now stay parallel: `gbrain sync --all --skip-failed` no longer forces `--serial`. + ## [0.42.44.0] - 2026-06-13 ### Fixed diff --git a/CLAUDE.md b/CLAUDE.md index 592b5e1dc..d75e61e90 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -97,6 +97,7 @@ detail on demand.) | any file in `src/` (what it does + its invariants) | `docs/architecture/KEY_FILES.md` — find the file's entry | | search / ranking / hybrid / retrieval | `docs/architecture/RETRIEVAL.md` + the `search/*` entries in `KEY_FILES.md` | | search modes / cost knobs | `docs/guides/search-modes.md` | +| embedding spend gates / cost gate / `spend.posture` / off switches | `docs/operations/spend-controls.md` | | push-based context (volunteer/watch/reflex window) | `docs/guides/push-context.md` | | schema packs / page types / extraction | `docs/architecture/schema-packs.md`, `type-taxonomy.md`, `lens-packs.md` | | thin-client / remote MCP / cross-modal | `docs/architecture/thin-client.md` | diff --git a/TODOS.md b/TODOS.md index 230a59860..ed6f5507c 100644 --- a/TODOS.md +++ b/TODOS.md @@ -1,5 +1,28 @@ # TODOS +## Spend-controls wave follow-ups (filed v0.42.45.0, #2139) + +Deferred from the #2139 delta-estimator wave. See plan + GSTACK REVIEW REPORT at +`~/.claude/plans/system-instruction-you-are-working-lovely-balloon.md`. + +- [ ] **P3 — Measured post-import chunk-count gating (#2139 proposal 2b).** + **What:** Gate the inline cost decision on the actual chunk count sync produced + (known after import, before embedding) instead of the pre-sync token estimate. + **Why:** A fully execution-accurate gate with zero estimate error. **Context:** + After v0.42.42.0 the estimator already mirrors execution (fetch-first delta via the + shared `computeSyncDelta`, `--full`=delta+stale, dirty-tree→$0). This is the + belt-and-suspenders fallback if a future case still drifts. **Trigger:** only if the + delta estimator proves insufficient in practice. **Start:** the gate call site in + `src/commands/sync.ts` (`runInlineCostGate`), gate on post-import `chunksCreated`. +- [ ] **P3 — Per-source defer granularity (#2139, D8A road-not-taken).** + **What:** When the aggregate inline gate trips in a non-TTY session, defer embeds + only for sources above a per-source floor; let cheap sources keep embedding inline. + **Why:** Cheap sources would get embeddings minutes sooner instead of waiting for a + backfill-worker drain. **Context:** v0.42.42.0 chose GLOBAL defer (one flag, strictly + dominates the exit-2 it replaced). This is the granularity upgrade. **Trigger:** a + filed embedding-latency-by-minutes complaint. **Start:** thread per-source estimates + through `runOne` (`src/commands/sync.ts`); design worked out at D8A in the plan. + ## gbrain#2095 push-based context follow-ups (v0.43+) Filed from the #2095 wave (volunteer_context op + reflex window + `gbrain watch`). diff --git a/VERSION b/VERSION index 4f011e032..360bb9599 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.42.44.0 \ No newline at end of file +0.42.45.0 diff --git a/docs/architecture/KEY_FILES.md b/docs/architecture/KEY_FILES.md index 005fd9e0c..ced7b8326 100644 --- a/docs/architecture/KEY_FILES.md +++ b/docs/architecture/KEY_FILES.md @@ -35,7 +35,7 @@ per-release `**vX.Y.Z:**` narration — CI enforces this - `src/core/embedding-pricing.ts` — `EMBEDDING_PRICING` map keyed `provider:model` for the post-upgrade reindex cost estimate. Sibling to `anthropic-pricing.ts`. Entries: OpenAI text-embedding-3-large ($0.13/1M), 3-small ($0.02/1M), ada-002 ($0.10/1M), Voyage 3-large ($0.18/1M), 3 ($0.06/1M). `lookupEmbeddingPrice(modelString)` returns a tagged union (`known` with price + `unknown` with provider name); `estimateCostFromChars(charCount, pricePerMTok)` uses 3.5 chars/token. Unknown providers degrade to "estimate unavailable" instead of fabricating numbers. - `src/core/post-upgrade-reembed.ts` — Pure functions backing the `gbrain upgrade` chunker-bump cost prompt. `computeReembedEstimate(engine, model)` queries real SQL (`COUNT(*)` + `COALESCE(SUM(LENGTH(compiled_truth)) + SUM(LENGTH(timeline)), 0)`) on `pages WHERE chunker_version < MARKDOWN_CHUNKER_VERSION`. `formatReembedPrompt(est, graceSeconds)` is the stderr-line formatter. `runPostUpgradeReembedPrompt(engine, model, opts)` orchestrates the 10-second Ctrl-C window; TTY-only wait (non-TTY auto-proceeds for CI / cron); `GBRAIN_NO_REEMBED=1` bails with a doctor-warning marker; `GBRAIN_REEMBED_GRACE_SECONDS=0` skips the wait. - `src/commands/reindex.ts` — `gbrain reindex --markdown [--limit N] [--dry-run] [--json] [--no-embed] [--repo PATH]`. Walks `pages WHERE page_kind = 'markdown' AND chunker_version < MARKDOWN_CHUNKER_VERSION` in 100-row batches ordered by id. Rows with non-null `source_path` re-import via `importFromFile`; rows without fall back to `importFromContent`. Both paths pass `forceRechunk: true` to bypass `importFromContent`'s `content_hash` short-circuit — without it the chunker version bump never reaches pages whose source content hasn't changed, AND the stripFactsFence privacy strip never applies to pre-strip chunks. Idempotent — partial-completion re-runs pick up via id-ordered batches. Wired into `src/commands/upgrade.ts:runPostUpgrade` after `apply-migrations`. The DB-only fallback (no source file on disk) does NOT pass body-only `compiled_truth` to `importFromContent` (that path re-parses with EMPTY frontmatter and OVERWRITES the page's real frontmatter/title/timeline); it `getPage`+`getTags`, reconstructs FULL markdown via `serializeMarkdown(frontmatter, compiled_truth, timeline, {type, title, tags})`, and re-imports THAT so re-chunking a DB-only page preserves everything while bumping `chunker_version`. Pinned by `test/reindex-preserve-tags.test.ts`. -- `src/commands/reindex-code.ts` — `gbrain reindex --code [--source ID] [--dry-run] [--yes] [--json] [--force] [--no-embed]`. Walks `pages WHERE type = 'code'` in 100-row batches, replays through `importCodeFile` for chunk + embed + content_hash folding. Idempotent unless `--force` bypasses the content_hash early-return. Cost-preview model field reads `getEmbeddingModelName()` from the gateway so preview reflects what the gateway will actually embed with. An informational stderr nudge inside `runReindexCode` (so dry-run + execute both surface it): when the configured embedding model isn't code-tuned (allowlist `{'voyage-code-3'}`, case-insensitive bare match), prints a recommendation to switch to `voyage:voyage-code-3`; suppress with `GBRAIN_NO_CODE_MODEL_NUDGE=1`, `--no-embed`, or `--json`. Pure `shouldNudgeCodeModel(bareName)` returns a tagged `NudgeDecision` union (takes the bare model name, emits qualified `voyage:voyage-code-3` for the paste-ready `gbrain config set` line). When `--yes` is absent and the caller is non-TTY or passed `--json`, the cost gate refuses (exit 2, no spend) via the pure exported `buildCostRefusal({json, previewMsg, preview, costUsd, model}): {stdout?, stderr?}` — JSON envelope only when `--json` is explicit, otherwise a human refusal on stderr (the spend guardrail is independent of the output format). Pinned by `test/ai/voyage-code-3-recipe.test.ts`, `test/reindex-code-nudge.serial.test.ts`, `test/reindex-code-model-source.serial.test.ts` (IRON-RULE regression for the cost-preview fix), `test/reindex-cost-refusal.test.ts`. +- `src/commands/reindex-code.ts` — `gbrain reindex --code [--source ID] [--dry-run] [--yes] [--json] [--force] [--no-embed]`. Walks `pages WHERE type = 'code'` in 100-row batches, replays through `importCodeFile` for chunk + embed + content_hash folding. Idempotent unless `--force` bypasses the content_hash early-return. Cost-preview model field reads `getEmbeddingModelName()` from the gateway so preview reflects what the gateway will actually embed with. An informational stderr nudge inside `runReindexCode` (so dry-run + execute both surface it): when the configured embedding model isn't code-tuned (allowlist `{'voyage-code-3'}`, case-insensitive bare match), prints a recommendation to switch to `voyage:voyage-code-3`; suppress with `GBRAIN_NO_CODE_MODEL_NUDGE=1`, `--no-embed`, or `--json`. Pure `shouldNudgeCodeModel(bareName)` returns a tagged `NudgeDecision` union (takes the bare model name, emits qualified `voyage:voyage-code-3` for the paste-ready `gbrain config set` line). When `--yes` is absent and the caller is non-TTY or passed `--json`, the cost gate refuses (exit 2, no spend) via the pure exported `buildCostRefusal({json, previewMsg, preview, costUsd, model}): {stdout?, stderr?}` — JSON envelope only when `--json` is explicit, otherwise a human refusal on stderr (the spend guardrail is independent of the output format). `spend.posture=tokenmax` OR an explicit `--max-cost off`/`unlimited` makes the gate informational and proceeds (#2139); `--max-cost off` also disables the runtime BudgetTracker cap. Pinned by `test/ai/voyage-code-3-recipe.test.ts`, `test/reindex-code-nudge.serial.test.ts`, `test/reindex-code-model-source.serial.test.ts` (IRON-RULE regression for the cost-preview fix), `test/reindex-cost-refusal.test.ts`. - `src/commands/sync.ts:resolveSlugByPathOrSourcePath` — Resolves a slug by `pages.source_path` first (returns the stored slug for frontmatter-fallback pages whose path doesn't derive a slug), then falls back to `resolveSlugForPath(path)`. Threaded into all 4 delete/rename call sites (`performSync`'s un-syncable cleanup at ~:531, deletes at ~:603, rename oldSlug at ~:622). Without this, emoji-only / Thai / Arabic filenames whose slug came from frontmatter would orphan on delete/rename (the delete path would compute the wrong path-derived slug). Best-effort query — pre-migration brains fall through to the legacy path. - `src/core/sources-ops.ts` — Multi-source registration + clone-lifecycle ops (`addSource`, `recloneIfMissing`, `defaultCloneDir`, `isOwnedClone`, `unownedHint`). **Reclone-ownership invariant (must-never-violate): gbrain may only delete/re-clone a clone it created, NEVER a user working tree.** `recloneIfMissing` deletes `local_path`, so it gates on `isOwnedClone(src)` and throws a `SourceOpError('unmanaged_path', ...)` BEFORE any filesystem op when ownership is unprovable — fail-closed. Ownership is proven by `config.managed_clone === true` (written by `addSource`'s `--url` path, covering default-location and `--clone-dir` clones) OR `local_path === defaultCloneDir(id)` (back-compat for pre-marker clones, via exact normalized-path equality, symlink-free). A row with `remote_url` + an unowned `local_path` (a user-registered working tree, e.g. `sources add --path`) is refused untouched; re-add with `--url` to regain auto-reclone. The reclone is EXDEV-safe: clone into a SIBLING temp of `local_path` (not the shared `clones/.tmp`, which may sit on a different mount than a `--clone-dir` target), then swap (move old aside → move new in → drop old) so `local_path` is never left missing-and-unrecoverable; on swap failure the original is restored, and if restore fails the error names the `aside` path so it's never reflexively deleted. A TOCTOU re-check re-confirms ownership immediately before the destructive move and rejects a symlink leaf swapped in after the entry check (`symlink_escape`). `unownedHint(src, state)` is the shared recovery message used by both the core error and the `gbrain sync --source` CLI error; `gbrain sources restore` special-cases `unmanaged_path` to print "DB row restored; gbrain syncs this path read-only" instead of the misleading "try sync to recover" guidance. `SourceOpErrorCode` includes `unmanaged_path`. Pinned by `test/sources-ops.test.ts`, `test/sources-resync-recovery.test.ts`. - `src/core/utils.ts` — Shared SQL utilities extracted from postgres-engine.ts. Exports `parseEmbedding(value)` (throws on unknown input, used by migration + ingest paths where data integrity matters) and `tryParseEmbedding(value)` (returns `null` + warns once per process, used by search/rescore paths where availability matters more than strictness). `isUndefinedColumnError(err)` predicate — pattern-matches Postgres SQLSTATE 42703 / "column ... does not exist" with engine-driver shape variation tolerated; replaces bare `catch {}` blocks in `oauth-provider.ts` so genuine errors (lock timeout, network blip, permission denied) propagate while column-missing falls through to the legacy fallback. `validateSourceId(id)` throws on anything outside `^[a-z0-9_-]+$`, used by the per-source disk-layout fix in patterns.ts/synthesize.ts before any `join(brainDir, '.sources', source_id, slug+'.md')` so source_id can't traverse out of brainDir. `rowToPage` populates the required `Page.source_id` from the SELECT projection (`scripts/check-source-id-projection.sh` enforces every projection feeding `rowToPage` includes the column). @@ -49,7 +49,7 @@ per-release `**vX.Y.Z:**` narration — CI enforces this - `src/core/disk-walk.ts` — `walkBrainRepo(repoPath)` returns `Map` from one recursive `readdirSync`. Skips dot-dirs, `node_modules`, non-`.md` files. Used by `gbrain storage status` to replace per-page `existsSync + statSync` (~400K syscalls on 200K-page brains → tens). - `src/core/git-head.ts` — local git HEAD freshness probe for `gbrain doctor`. `isSourceUnchangedSinceSync(localPath, lastCommit, opts?)` returns true iff `localPath` is a git repo whose current HEAD matches `lastCommit`; when `opts.requireCleanWorkingTree` is true also requires a clean working tree (mirrors `gbrain sync`'s force-walk gate at `sync.ts:1075` so doctor and sync agree on "is there work to do?"). `requireCleanWorkingTree` is `boolean | 'ignore-untracked'` — in `'ignore-untracked'` mode the clean probe runs `git status --porcelain --untracked-files=no` so a quiet repo with stray untracked dirs (`?? companies/`, `?? media/`) is still "unchanged" (sync's incremental path keys off the commit diff and never imports untracked files); `GitCleanProbe` gains an `ignoreUntracked?` second arg. Two probe seams (`_setGitHeadProbeForTests`, `_setGitCleanProbeForTests`) keep unit tests R2-compliant (no `mock.module`). Uses `execFileSync` with array args so shell metachars in `local_path` cannot escape to a shell (the regression test runs real `execFileSync` against `'/nonexistent/$(touch )/repo'` and asserts the sentinel is never created). Fail-open on every error (missing path, not a git repo, git not installed, timeout, NULL inputs, dirty-probe errored → false) preserving the caller's prior time-based behavior. The chunker-version-match check lives in the caller (doctor.ts) because it depends on engine state (`sources.chunker_version` vs `CHUNKER_VERSION` from `src/core/chunkers/code.ts`). Pinned by `test/core/git-head.test.ts` (incl. the shell-injection regression guard). - `src/core/source-health.ts` — per-source health metrics for `gbrain sources status` + doctor's `federation_health`. Commit-relative staleness: `newestCommitMs(localPath)` = HEAD committer time via `git log -1 --format=%ct` (fail-open null; NO working-tree mtime parsing — committed content only, robust against the porcelain-mtime bug farm); pure `lagFromContentMs(contentMs|null, lastSyncMs|null, nowMs)` = remote/column comparator (null lastSync → null; negative wall-clock → skew passthrough; `contentMs <= lastSync` → 0; else/null-content → wall-clock). `computeAllSourceMetrics(engine, sources, {probeContent?})`: LOCAL (`probeContent:true`, `gbrain sources status`) → `isSourceUnchangedSinceSync(..., {requireCleanWorkingTree:'ignore-untracked'}) ? 0 : wall-clock` (live commit-hash catches HEAD moving to an old-dated commit a timestamp compare would miss); REMOTE (default, `federation_health` on the HTTP MCP path) → `lagFromContentMs(row.newest_content_at, ...)`, NO git subprocess (trust boundary). `commitTimeMs(localPath, sha)` is the `newestCommitMs` sibling pinned to an arbitrary commit (committer time via `git show -s --format=%ct `, fail-open null, execFileSync array args) — the resumable sync stamps `newest_content_at` against its pinned target commit, not whatever HEAD raced to. Pinned by `test/source-health.test.ts`. -- `src/core/git-remote.ts` — SSRF-hardened git invocations for remote-source `cloneRepo` and `pullRepo`. Exports two distinct flag constants because `git`'s argv grammar treats them differently: `GIT_SSRF_FLAGS` (3 `-c` config flags — `protocol.allow=user`, `protocol.file.allow=never`, `http.allowRedirects=false`) is global config, spread BEFORE the subcommand verb; `GIT_SSRF_SUBCOMMAND_FLAGS = ['--no-recurse-submodules']` is subcommand-scoped, spread AFTER the verb (a combined array would spread `--no-recurse-submodules` before the verb where real git rejects it exit 129). `cloneRepo` argv: `git clone --depth=1 [--branch X] -- `. `pullRepo` argv: `git -C pull --ff-only`. Pinned by `test/git-remote.test.ts` position-anchored regression guard (`argv.indexOf('--no-recurse-submodules') > argv.indexOf(verb)`). +- `src/core/git-remote.ts` — SSRF-hardened git invocations for remote-source `cloneRepo`, `pullRepo`, and `fetchRemote(repoPath, branch)` (the last added for the sync cost-estimator's fetch-first path, #2139, so a cost preview / dry-run fetches through the same hardened flags + `GIT_TERMINAL_PROMPT=0` as real sync rather than a less-protected route). Exports two distinct flag constants because `git`'s argv grammar treats them differently: `GIT_SSRF_FLAGS` (3 `-c` config flags — `protocol.allow=user`, `protocol.file.allow=never`, `http.allowRedirects=false`) is global config, spread BEFORE the subcommand verb; `GIT_SSRF_SUBCOMMAND_FLAGS = ['--no-recurse-submodules']` is subcommand-scoped, spread AFTER the verb (a combined array would spread `--no-recurse-submodules` before the verb where real git rejects it exit 129). `cloneRepo` argv: `git clone --depth=1 [--branch X] -- `. `pullRepo` argv: `git -C pull --ff-only`. Pinned by `test/git-remote.test.ts` position-anchored regression guard (`argv.indexOf('--no-recurse-submodules') > argv.indexOf(verb)`). - `src/commands/storage.ts` — `gbrain storage status [--repo P] [--json]`. Split into pure data (`getStorageStatus`) + JSON formatter + human formatter (ASCII-only) matching the `orphans.ts` pattern. `PageCountsByTier` and `DiskUsageByTier` are distinct nominal types so swaps fail at compile time. - `gbrain.yml` (brain repo root) — Optional storage tiering config. Top-level `storage:` section with `db_tracked:` and `db_only:` array-valued keys. `gbrain sync` auto-manages `.gitignore` for `db_only` paths on successful sync (skips on dry-run, blocked-by-failures, submodule context, or `GBRAIN_NO_GITIGNORE=1`). `gbrain export --restore-only [--repo P] [--type T] [--slug-prefix S]` repopulates missing `db_only` files from the database. - `src/core/supabase-admin.ts` — Supabase admin API (project discovery, pgvector check). @@ -102,7 +102,9 @@ per-release `**vX.Y.Z:**` narration — CI enforces this - `src/core/search/hybrid.ts` — Cathedral II `Promise` return shape. `onMeta?: (m: HybridSearchMeta) => void` callback so op-layer capture records what hybridSearch actually did; existing callers leave it undefined. `HybridSearchOpts.types?: PageType[]` (on `SearchOpts`) threads a multi-type filter into per-engine `searchKeyword` + `searchVector` + `searchKeywordChunks` as `AND p.type = ANY($N::text[])` (primary consumer `gbrain whoknows`, filters to `['person','company']`); AND-applies alongside the single-value `type` filter. `hybridSearch` resolves the embedding column at the boundary via `resolveColumn(loadRegistry(cfg), opts.embedding_column, cfg)` from `src/core/search/embedding-column.ts`, threads the `ResolvedColumn` descriptor (not a raw string) into per-engine `searchVector`, and uses `isCacheSafe(resolved, cfg)` for the cache-skip decision so a repointed `embedding` builtin doesn't leak across vector spaces. `cosineReScore` calls `engine.getEmbeddingsByChunkIds(ids, resolved.name)` so rerank uses vectors from the active column, not the hardcoded OpenAI `embedding`. The `query` MCP op accepts `embedding_column` for per-call A/B; `search` (keyword-only) rejects it. Two post-fusion stages + evidence stamp: `applyTitleBoost(results, query, titleBoost, floorThreshold)` multiplies a result's score by the resolved `title_boost` when `isTitlePhraseMatch` fires, stamps `title_match_boost`, inherits the floor-ratio gate so a title match can't shove a much-stronger page below it; `applyAliasHop(engine, results, query, opts)` normalizes the query, calls `engine.resolveAliases`, and on exact normalized-alias match surfaces that page at top-of-organic + epsilon with `alias_hit=true`; `stampEvidence(...)` runs LAST (after the alias hop, before slice) on every path — keyword-only, no-embed, and full hybrid — so MCP callers and `--explain` read the same `evidence` + `create_safety` contract. `title_boost` resolved from the mode bundle and threaded in. - `docs/eval-capture.md` — stable NDJSON schema reference for gbrain-evals consumers. - `test/public-exports.test.ts` — runtime contract test (R2). Imports each of the 17 public subpaths via package name and pins a canary symbol per module. Paired with `scripts/check-exports-count.sh`. -- `src/core/embedding.ts` — OpenAI text-embedding-3-large, batch, retry, backoff. `BATCH_SIZE=100` (per-recipe pre-split + recursive halving + adaptive shrink-on-miss live in the gateway; the outer paginator is for progress-callback granularity, not batch protection). `estimateEmbeddingCostUsd(tokens)` prices against the currently-configured model's rate via `currentEmbeddingPricePerMTok()` (resolves the per-1M-token rate via `lookupEmbeddingPrice(gatewayGetModel())` from `embedding-pricing.ts`, falling back to the OpenAI 3-large rate 0.13 only when the gateway is unconfigured or the model is unknown to the pricing table). `EMBEDDING_COST_PER_1K_TOKENS` retained for back-compat with direct importers/tests. `currentEmbeddingSignature(): string` returns the embedding-provenance signature `:` (e.g. `openai:text-embedding-3-large:1536`) stamped onto `pages.embedding_signature` at every embed-write site; DELIBERATELY excludes the chunker version (tracked separately via `pages.chunker_version`) — this signature is strictly the EMBEDDING space, so a model OR dimension swap makes the stored signature differ from current and a page becomes stale. Same unconfigured-gateway fallback as the cost helpers. `willEmbedSynchronously({v2Enabled, serialFlag, noEmbed}): SyncEmbedMode` is the single source of truth for whether `gbrain sync --all` embeds at sync time (`'inline'`) or defers to per-source `embed-backfill` minion jobs (`'deferred'`) — mirrors `sync.ts`'s `effectiveNoEmbed` resolution exactly (`v2Enabled && !serialFlag && !noEmbed → deferred`) so cost gate and embed decision can't drift. `shouldBlockSync(costUsd, floorUsd, mode): boolean` is the pure cost-gate decision: blocks ONLY when `mode === 'inline' && costUsd > floorUsd` — deferred mode never blocks (the backfill's $X/source/24h cap is the real money gate). Pinned by `test/sync-cost-preview.test.ts` + `test/embedding-signature-stale.test.ts`. +- `src/core/embedding.ts` — OpenAI text-embedding-3-large, batch, retry, backoff. `BATCH_SIZE=100` (per-recipe pre-split + recursive halving + adaptive shrink-on-miss live in the gateway; the outer paginator is for progress-callback granularity, not batch protection). `estimateEmbeddingCostUsd(tokens)` prices against the currently-configured model's rate via `currentEmbeddingPricePerMTok()` (resolves the per-1M-token rate via `lookupEmbeddingPrice(gatewayGetModel())` from `embedding-pricing.ts`, falling back to the OpenAI 3-large rate 0.13 only when the gateway is unconfigured or the model is unknown to the pricing table). `EMBEDDING_COST_PER_1K_TOKENS` retained for back-compat with direct importers/tests. `currentEmbeddingSignature(): string` returns the embedding-provenance signature `:` (e.g. `openai:text-embedding-3-large:1536`) stamped onto `pages.embedding_signature` at every embed-write site; DELIBERATELY excludes the chunker version (tracked separately via `pages.chunker_version`) — this signature is strictly the EMBEDDING space, so a model OR dimension swap makes the stored signature differ from current and a page becomes stale. Same unconfigured-gateway fallback as the cost helpers. (See `src/core/sync-delta.ts` + `src/core/spend-posture.ts` for the #2139 cost-gate supporting modules.) `willEmbedSynchronously({v2Enabled, serialFlag, noEmbed}): SyncEmbedMode` is the single source of truth for whether `gbrain sync --all` embeds at sync time (`'inline'`) or defers to per-source `embed-backfill` minion jobs (`'deferred'`) — mirrors `sync.ts`'s `effectiveNoEmbed` resolution exactly (`v2Enabled && !serialFlag && !noEmbed → deferred`) so cost gate and embed decision can't drift. `shouldBlockSync(costUsd, floorUsd, mode, posture='gated'): boolean` is the pure cost-gate decision: blocks ONLY when `mode === 'inline' && costUsd > floorUsd` — deferred mode never blocks (the backfill's $X/source/24h cap is the real money gate), and `posture === 'tokenmax'` never blocks (the operator declared cost isn't the constraint; an `off`/`unlimited` floor is `Infinity` and so is never exceeded). Pinned by `test/sync-cost-preview.test.ts` + `test/embedding-signature-stale.test.ts`. +- `src/core/sync-delta.ts` — the single "what changed since last_commit" helper (#2139), consumed by BOTH `performSyncInner` (sync executor) and `estimateInlineNewTokens` (cost estimator) so the gate's dollar figure can't drift from what the sync imports. `computeSyncDelta(repoPath, fromCommit, toCommit, {detachedManifest?, detached?})` → `{status:'ok', manifest}` | `{status:'unavailable', reason:'anchor_missing'|'diff_failed'}`. Anchor reachability via `git cat-file -t` (#1970 discipline — a gc'd bookmark is `anchor_missing`, but a present-but-non-ancestor bookmark is still diffed tree-to-tree), then `git diff --name-status -M from..to` parsed by `buildSyncManifest`; merges the detached working-tree manifest when detached (`buildDetachedWorkingTreeManifest`, relocated here from sync.ts). NO dirty/untracked probe — attached-HEAD incremental sync imports only the commit diff, so pricing dirty files would re-introduce phantom costs on a busy brain. `execFileSync` array-args (shell-injection safe), 30s / 100 MiB budget. Test seam `_setGitRunnerForTests`. Pinned by `test/sync-delta.test.ts`. +- `src/core/spend-posture.ts` — spend-control surface (#2139). `resolveSpendPosture(engine): 'gated'|'tokenmax'` (DB-plane `spend.posture`, fail-open `gated`); `tokenmax` makes every cost gate informational across sync/reindex/enrich/onboard (spend still ledgered — removes the ceiling, not the accounting). `parseUsdLimit(raw, def, {allowZero?})` accepts `off`/`unlimited`/`none` → `Infinity`; `formatUsdLimit(n)` renders `Infinity` as the string `'unlimited'` (never raw — `JSON.stringify(Infinity)` is `null`); `usdLimitToCap(n)` maps `Infinity` → `undefined` at the BudgetTracker boundary so ledger rows never serialize null. `normalizeSpendPosture`/`isValidSpendPosture` back the `config set` validation. Doc: `docs/operations/spend-controls.md`. Pinned by `test/sync-cost-preview.test.ts` + `test/spend-off-switch.test.ts`. - `src/core/ai/dims.ts` — per-provider `providerOptions` resolver for embed-time dimension passthrough; the single source of truth for "which provider needs which knob to produce `vector(N)`". Exports `dimsProviderOptions(implementation, modelId, dims)` (called by `embed()` in `gateway.ts`), `VOYAGE_OUTPUT_DIMENSION_MODELS` (private const — the 7 hosted Voyage models that accept `output_dimension`: `voyage-4-large`, `voyage-4`, `voyage-4-lite`, `voyage-3-large`, `voyage-3.5`, `voyage-3.5-lite`, `voyage-code-3` — nano deliberately excluded), `VOYAGE_VALID_OUTPUT_DIMS = [256, 512, 1024, 2048] as const`, `supportsVoyageOutputDimension(modelId)`, `isValidVoyageOutputDim(dims)`. Voyage path uses the SDK-supported `dimensions` field (`{ openaiCompatible: { dimensions: N } }`), NOT Voyage's `output_dimension` wire-key — the `voyageCompatFetch` shim in `gateway.ts:541` translates `dimensions → output_dimension` before the HTTP body is built (the AI SDK's openai-compatible adapter doesn't recognize the wire-key, so sending it from here would be silently dropped and Voyage would return its default 1024-dim). Runtime guard: when a Voyage flexible-dim model is configured with `dims` outside `VOYAGE_VALID_OUTPUT_DIMS`, throws `AIConfigError` with a paste-ready `gbrain config set embedding_dimensions <256|512|1024|2048>` hint at the embed boundary (most common trigger: `embedding_model: voyage:voyage-4-large` without `embedding_dimensions`, falling back to `DEFAULT_EMBEDDING_DIMENSIONS=1536`, an OpenAI default not a Voyage one). - `src/core/ai/types.ts` — provider/recipe types. `EmbeddingTouchpoint` has optional `chars_per_token` (default 4, matching OpenAI tiktoken on English) and `safety_factor` (default 0.8, budget-utilization ceiling), both consulted only when `max_batch_tokens` is also set; Voyage declares `chars_per_token=1` + `safety_factor=0.5` to handle dense payloads (CJK/JSON/base64). Pre-split budget = `max_batch_tokens × safety_factor / chars_per_token`. `EmbeddingTouchpoint.multimodal_models?: string[]` model-level allow-list for recipes mixing text-only + multimodal models under one touchpoint (Voyage's 12 models share `supports_multimodal: true` but only `voyage-multimodal-3` accepts `/multimodalembeddings`); when omitted, recipe-level `supports_multimodal` is sufficient. `AIGatewayConfig.embedding_multimodal_model?: string` lets `embedMultimodal()` route to a different model than `embedding_model` (OpenAI text + Voyage images without flipping the primary pipeline). `Recipe.default_headers?: Record` (static) and `Recipe.resolveDefaultHeaders?(env)` (env-templated) seam for per-recipe headers riding alongside auth on every openai-compat touchpoint; mutually exclusive (declaring both throws `AIConfigError` at gateway-configure time); keys conflicting with the resolved auth header (`Authorization`, the resolver's custom header) rejected at `applyResolveAuth` call time so defaults can't shadow auth. Used by OpenRouter for the `HTTP-Referer` + `X-OpenRouter-Title` + `X-Title` attribution triple. - `src/core/ai/gateway.ts` — unified seam for every AI call. `embedQuery(text, opts?)` and `isAvailable(touchpoint, modelOverride?)` accept a model override so the resolved-column path embeds via the column's provider (Voyage / ZeroEntropy / OpenAI) instead of the global default; the hybrid path passes `{embeddingModel: resolved.provider, dimensions: resolved.dimensions}` and the gateway resolves the matching recipe via `instantiateEmbedding()`. `isAvailable('embedding', 'voyage:voyage-3-large')` checks the override's recipe (not the default) so hybrid skips vector search only when the active column's provider is actually down. `zeroEntropyCompatFetch` shim (sibling to `voyageCompatFetch`) handles ZE's non-OpenAI-compatible wire shape — rewrites the request URL `/embeddings → /models/embed`, injects `input_type` (default `'document'`; the threaded `'query'|'document'` crosses the SDK boundary via the module-level `__embedInputTypeStore` AsyncLocalStorage populated in `embedSubBatch()`, because the AI SDK's openai-compatible adapter strips `input_type` from `providerOptions` before building the wire body — #1400; `voyageCompatFetch` injects it opt-in the same way, and `openAICompatAsymmetricFetch` is the fallthrough shim for every other openai-compat recipe — llama-server/litellm/ollama — a strict pass-through when nothing was threaded) and explicit `encoding_format: 'float'`, and rewrites the response `{results: [{embedding}], usage: {total_bytes, total_tokens}}` → `{data: [{embedding, index}], usage: {prompt_tokens, total_tokens}}` so the SDK's openai-compatible Zod schema validates. Layer 1 (Content-Length) + Layer 2 (per-embedding) OOM caps via tagged `ZeroEntropyResponseTooLargeError` (kept separate from `VoyageResponseTooLargeError` because `test/voyage-response-cap.test.ts` does structural source-text greps pinning the Voyage name). Wired in `instantiateEmbedding()` via the `recipe.id === 'zeroentropyai'` branch. `gateway.rerank()` native HTTP path (no AI-SDK reranking abstraction): resolves the configured reranker via `getRerankerModel()`, posts to `${recipe.base_url}/models/rerank` with bearer auth, returns `RerankResult[]` sorted by relevance. `RerankError.reason` classifier: `auth | rate_limit | network | timeout | payload_too_large | unknown`. 5s default timeout (search hot path). Pre-flight payload guard rejects bodies over `recipe.touchpoints.reranker.max_payload_bytes` with `reason: 'payload_too_large'`. `_rerankTransport` test seam mirrors `_embedTransport`. `embedQuery(text)` threads `inputType: 'query'` through `dimsProviderOptions()` (4-arg). `getRerankerModel()` accessor + `isAvailable('reranker')` branch; `configureGateway` + `reconfigureGatewayWithEngine` thread `reranker_model`; `applyResolveAuth` + `defaultResolveAuth` widen touchpoint param to include `'reranker'`. `embedMultimodalOpenAICompat()` routes recipes with `implementation: 'openai-compatible'` (LiteLLM, Anyscale, vLLM, Gemini multimodal via proxy) through the standard `/embeddings` endpoint with content arrays carrying `image_url` entries; the Voyage `/multimodalembeddings` path is unchanged (gateway selects by recipe `implementation` tag). Runtime dimension validation throws `AIConfigError` (with model id + observed + expected) before the vector reaches storage when the provider returns a width that doesn't match the recipe's `default_dims` or the brain's `embedding_dimensions`. Pinned by `test/openai-compat-multimodal.test.ts`. Module-scoped `_embedTransport` defaults to AI SDK `embedMany`, with `__setEmbedTransportForTests(fn)` test seam so tests drive `embed()` with a stubbed transport. `splitByTokenBudget` and `isTokenLimitError` exported `@internal` (pure functions reused by the test file). Module-level `_shrinkState: Map` halves the recipe's effective `safety_factor` on token-limit miss (floor 0.05) and heals back ×1.5 after `SHRINK_HEAL_AFTER=10` consecutive successes. `configureGateway()` walks every registered recipe at construction and emits a once-per-process stderr warning for any embedding touchpoint missing `max_batch_tokens` (excluding the canonical OpenAI fast-path). `resetGateway()` clears `_shrinkState`, the warned-set, and restores the real transport. `embedMultimodal()` reads `cfg.embedding_multimodal_model` first (falls back to `cfg.embedding_model`); after the recipe-level `supports_multimodal` fast-fail, validates the resolved model against `touchpoint.multimodal_models` when declared (closes the Voyage-text-only-into-multimodal-endpoint footgun before any HTTP call). `getMultimodalModel()` accessor mirrors `getEmbeddingModel` / `getChatModel`. Exported `VoyageResponseTooLargeError` tagged class: `voyageCompatFetch`'s two OOM-defense caps (Layer 1 Content-Length at `:595`, Layer 2 per-embedding base64 at `:619`) throw it; the inbound response-rewriter's try/catch (which swallows parse failures so misshaped responses fall through to the SDK parser) checks `instanceof VoyageResponseTooLargeError` and rethrows so the cap is actually effective (regression assertion in `test/voyage-response-cap.test.ts` pins the `instanceof ⇒ throw err` line). AI SDK v6 toolLoop compat (`gbrain skillopt` rollouts AND production background `subagent` jobs both route through `chat()` / `toolLoop`): in `chat()`, tool defs wrap the raw JSON Schema with the SDK's `jsonSchema()` helper (`inputSchema: jsonSchema(t.inputSchema)`) — v6's `asSchema()` treats a bare `{jsonSchema: ...}` object as a thunk and throws "schema is not a function"; new exported pure `toModelMessages(messages: ChatMessage[]): unknown[]` converts gbrain's provider-neutral `ChatMessage[]` into v6 `ModelMessage[]` — tool results (pushed by `toolLoop` as `role:'user'` with bare-value tool-result blocks) become a dedicated `role:'tool'` message with structured `output:{type:'json'|'text'|'error-text', value}` parts; `null` output preserved as `{type:'json', value:null}` (not dropped); text/tool-call blocks pass through with v6 field names (`toolCallId`/`toolName`/`input`); applied at the `generateText` call (`messages: toModelMessages(opts.messages)`). The converter is the load-bearing fix for the production subagent path, not just skillopt. Pinned by `test/gateway-model-messages.test.ts`. Companion fix in `src/core/skillopt/rollout.ts`: the inline `paramsToSchema` dropped `items` on array params; it now uses the shared `paramDefToSchema` from `src/mcp/tool-defs.ts` (single source of truth, recursive on items/enum/default). @@ -296,7 +298,7 @@ per-release `**vX.Y.Z:**` narration — CI enforces this - `src/core/embedding-dim-check.ts` extension — facts.embedding dim drift surface. `readFactsEmbeddingDim(engine): Promise` covers both `vector(N)` and `halfvec(N)` shapes (migration v40 falls back to `vector` on pgvector < 0.7); regex ordering is halfvec-before-vector (substring "vec" appears in "halfvec"; naive `/vector/i` would shadow). `buildFactsAlterRecipe(dims, configured, type)` emits the paste-ready `DROP INDEX IF EXISTS idx_facts_embedding_hnsw; ALTER TABLE facts ALTER COLUMN embedding TYPE halfvec(N) USING embedding::halfvec(N); CREATE INDEX idx_facts_embedding_hnsw ON facts USING hnsw (embedding halfvec_cosine_ops) WHERE ...` flow (NOT bare REINDEX, which doesn't rewrite the index after a column-type change). `assertFactsEmbeddingDimMatchesConfig(engine)` is the preflight — throws `FactsEmbeddingDimMismatchError` (tagged `tag: 'FACTS_EMBEDDING_DIM_MISMATCH'` for parity with the worker-pool MUST_ABORT semantics) when configured dim ≠ column width; cached per-engine via `WeakMap`; PGLite engines silently skip. Doctor check `facts_embedding_width_consistency` (registered after `embedding_width_consistency`) reuses the same helpers with an identical ALTER recipe. Pinned by `test/embedding-dim-check-facts.test.ts`. - `src/core/postgres-engine.ts` extension — `insertFact` + `insertFacts` no longer hardcode `tx.unsafe(\`'${embedLit}'::vector\`)` for the embedding column. `resolveFactsEmbeddingCast()` (private) probes `pg_attribute` once per engine instance (cached in `_factsEmbeddingCastSuffix`) and returns `'::halfvec'` when migration v40 created the column as halfvec, else `'::vector'`; both insert paths use the cached suffix so the cast matches the actual column type (works on older pgvector that lacks implicit auto-cast). Test seam `__resetFactsEmbeddingCastCacheForTest()` clears the per-engine cache. - `src/core/cycle.ts` + `src/core/cycle/extract-atoms.ts` + `src/core/cycle/synthesize-concepts.ts` + `src/commands/extract.ts` + `src/commands/doctor.ts` + `src/core/op-checkpoint.ts` extensions — six daily-driver ops fixes. (1) Batch idempotency: `atomsExistingForHashes(engine, sourceId, hashes[])` (exported from `src/core/cycle/extract-atoms.ts`) replaces the per-hash loop (7K individual queries) with one batched SQL roundtrip returning already-extracted `content_hash16` values; fail-open (SQL error → empty set, extraction proceeds); powered by migration v104 `pages_atom_source_hash_idx` (partial expression index on `frontmatter->>'source_hash'` for atom rows where `deleted_at IS NULL`; Postgres `CREATE INDEX CONCURRENTLY` with invalid-remnant pre-drop, PGLite plain). (2) Shorter cycle lock TTL + active in-phase refresh: `LOCK_TTL_MINUTES = 5` (was 30); `buildYieldDuringPhase(lock, outer)` (exported, with `LockHandle`) calls `lock.refresh()` + any external hook on every fire, throttled to 30s via `maybeYield`, firing both in the main loop AND immediately after every `await chat(...)`; `synthesize_concepts` uses the same throttled hook. A crashed cycle releases its lock 6x faster while a healthy long-running cycle keeps it alive (residual: a single `await chat()` past 5 min can expire the lock mid-await — TODO-OPS-2). (3) Progress wiring: `progress?: ProgressReporter` opt on `ExtractAtomsOpts` and `SynthesizeConceptsOpts`; cycle.ts passes its phase-level reporter down (NOT a child reporter, which would collide on `cycle.extract_atoms.extract_atoms.work`); phases only call `tick()`/`heartbeat()`, cycle.ts owns `start()`/`finish()`. (4) `by-mention` resume: `mentionsFingerprint({source, type, since, gazetteerHash})` in `src/core/op-checkpoint.ts` — the gazetteer hash is load-bearing (adding entity pages mid-pause shifts the hash → new fingerprint → fresh scan against the new gazetteer, never silent skip); `gbrain extract links --by-mention` resumes via `op_checkpoints` with `flushAndCheckpoint` ordering (links flush to DB FIRST, page keys commit to checkpoint SECOND, persist THIRD, so a crash mid-batch leaves the page un-checkpointed and resume re-scans it); persist every 1000 items OR 30s; clean exit clears the checkpoint; `--dry-run` skips both load and write. (5) `sync_consolidation` doctor check (multi-source brains see a paste-ready `gbrain sync --all --parallel 4 --workers 4 --skip-failed`; single-source "not applicable"; SQL errors return `warn` via the check's own try/catch). (6) Test-isolation: `test/cycle-last-full-cycle-at.test.ts` + `test/schema-cli.test.ts` use per-test `GBRAIN_HOME=tempdir`. Pinned by `test/cycle/extract-atoms-batch.test.ts`, `test/cycle/cycle-lock-ttl.test.ts` (regression pin on `LOCK_TTL_MINUTES === 5`), `test/op-checkpoint-mentions-fingerprint.test.ts`, `test/cycle/extract-atoms-progress.test.ts`, `test/cycle/synthesize-concepts-progress.test.ts`, `test/cycle/yield-during-phase-refresh.test.ts`, `test/cycle/yield-during-phase-throttle.test.ts`, `test/extract-by-mention-resume.test.ts`, `test/doctor-sync-consolidation.test.ts`. Companion `sync --all` recipe block in `skills/cron-scheduler/SKILL.md`. -- `src/commands/sync.ts` — `gbrain sync` CLI + the `performSync` / `performFullSync` library entrypoints (consumed by the autopilot cycle and the Minion sync handler). `performSync` runs under a writer lock: per-source `gbrain-sync:` whenever `opts.sourceId` is set, wrapped in `withRefreshingLock` from `src/core/db-lock.ts` so long-running sources (250K+ chunks) don't lose the lock at the 30-min TTL mid-run; the bare no-source path uses the SAME refreshing lock (#1794 — it was previously a non-refreshing `tryAcquireDbLock`, stealable mid-run during an incident); `SyncOpts.lockId?: string` is the explicit override. The lock refresh AND its health probe route through the DIRECT session pool so Supavisor transaction-pool exhaustion (`EMAXCONNSESSION`) can't kill renewal; takeover is heartbeat-aware (it will NOT steal a holder whose `last_refreshed_at` is within `GBRAIN_LOCK_STEAL_GRACE_SECONDS`, defending an alive-but-starved holder); the import loop yields the event loop every `GBRAIN_SYNC_YIELD_EVERY` files (`setTimeout(0)`, not `setImmediate` — Bun starves the timers phase) so the refresh `setInterval` heartbeat fires mid-import. This lock-identity invariant prevents a `sync --all` per-source worker racing `sync --source foo` on the global lock from corrupting the same source. `performSync` throws a typed `SyncLockBusyError` when the writer lock is held; the Minion `sync` handler (`src/commands/jobs.ts`) catches it and marks the job SKIPPED (not failed) so a cron/autopilot tick defers to the holder without polluting crash metrics. `performSyncInner` is RESUMABLE (incremental path): it drains a PINNED target commit (`lastCommit..pin`), banking drained file paths via `appendCompleted` (append-only delta into the `op_checkpoint_paths` child table, migration v115 — one row per path, O(delta) not the old O(N²) full-array rewrite), keyed by `syncFingerprint({sourceId, lastCommit})` from `src/core/op-checkpoint.ts` (paths under `op:'sync'`; the pinned target under `op:'sync-target'`), and advances `last_commit`/`last_sync_at` ONLY at full import completion. Checkpoint writes route through the DIRECT session pool + bounded retry so they survive `EMAXCONNSESSION`; the flush cadence is first-file then every `GBRAIN_SYNC_CHECKPOINT_EVERY` (default 1000) files OR `GBRAIN_SYNC_CHECKPOINT_SECONDS` (default 10s), with a race-safe `pendingCheckpointPaths` delta (single-flight swap, re-merge on failure) under parallel workers; a SIGTERM banks the in-flight delta via a no-retry one-shot (`appendCompletedOnce`, ordered before lock release through `registerCleanup`); and sustained flush failure aborts the run with `reason:'checkpoint_unavailable'` after `GBRAIN_SYNC_MAX_CHECKPOINT_FAILURES` consecutive fails rather than importing work it can never bank (every partial/blocked exit logs the banked-file count). A sync killed mid-import banks its progress, leaves the anchor unmoved (the source stays correctly stale to the autopilot scheduler — `last_sync_at` is never bumped on a partial), and the next run `resumeFilter`s the same fixed diff to skip done files. The pin is the checkpoint's stored target when still reachable from HEAD, else live HEAD (a history rewrite / reset re-pins); completion advances to the pin, NOT live HEAD, so commits landing past the pin are a clean next-sync diff — this closes the cross-run staleness window. After import a pin-reachability gate (`git merge-base --is-ancestor pin HEAD`) replaces the old strict head-drift gate: forward commits on top of the pin (e.g. a background enrich process committing to the same repo every ~2 min) are SAFE and no longer block the run; only a real rewrite (pin not an ancestor of HEAD) blocks and discards the checkpoint. A file added in `lastCommit..pin` but gone from disk (deleted by a commit after the pin) is SKIPPED and marked completed, not recorded as a failure. Downstream extract/facts/embed are size-gated: inline only for `totalChanges <= 100`; large syncs defer to the resumable `extract --stale` watermark + `embed --stale`/backfill + the facts cycle phases, so a 44K-page facts/embed pass never re-blocks import convergence (sync convergence == import convergence). Worker engines wrap in try/finally so disconnect always fires; both PGLite-detection sites use `engine.kind === 'pglite'`. CLI accepts `--workers N` (alias `--concurrency N`) validated via `parseWorkers` (explicit bypasses the file-count floor; auto path defers to `autoConcurrency()`). The newest-first descending-lex order uses `sortNewestFirst(addsAndMods)` from `src/core/sort-newest-first.ts` (shared with `gbrain import`). `gbrain sync --all` runs a continuous worker pool: `parseWorkers`-validated `--parallel N` (default `min(sourceCount, --workers, DEFAULT_PARALLEL_SOURCES=4)`), long-lived async workers pulling from a shared FIFO queue (no head-of-line blocking), per-source `withSourcePrefix(src.id, ...)` so every `slog`/`serr` line carries `[]`; `--skip-failed`/`--retry-failed` reject when combined with `--parallel > 1` (the brain-global `sync-failures.jsonl` has no per-source scope); a connection-budget stderr warning fires when `parallel × workers × 2 > 16` (the `× 2 per-file pool` factor: each per-file worker opens its own `PostgresEngine` with `poolSize=2`). Exports `resolveParallelism`, `syncOneSource`, `buildSyncStatusReport`, `printSyncStatusReport`, `SyncStatusReport` back the `gbrain sources status` dashboard. `--json` envelope `{schema_version: 1, sources, parallel, ok_count, error_count}` on stdout; human banners route to stderr via `humanSink` so `jq` parses cleanly. Exit matrix: 0 all ok, 1 any error, 2 cost-prompt-not-confirmed. The dashboard SQL is `content_chunks ch JOIN pages pg ON pg.id = ch.page_id WHERE pg.deleted_at IS NULL` with `archived = false` at the caller; embedding column resolved via `resolveEmbeddingColumn(undefined, cfg)` from `src/core/search/embedding-column.ts` so Voyage/multimodal/non-default-column brains count against the column they use; errors propagate (no swallow-catch). The sync delete loop is interleaved per-batch resolve+delete using `engine.resolveSlugsByPaths` + `engine.deletePages` from `src/core/engine.ts` (73K-delete commit: ~146K SQL round-trips → ~292, closing the cascade-staleness class where one big-delete commit jammed every other source's sync); per-batch try-catch decomposes batch DELETE failures to per-slug `deletePage` fallback, unrecoverable per-slug failures land in `failedFiles`; `pagesAffected` filters to confirmed-deleted slugs. The rename loop gets the same batched slug-resolve; a rename whose destination is un-syncable folds the source path into the delete set so the old page can't orphan (#1970). An entry-time bookmark-reachability guard distinguishes a gc'd anchor (`cat-file` fails → `performFullSync`) from a history-rewrite anchor that is merely no-longer-an-ancestor but still on disk: the latter is diffed tree-to-tree directly (`git diff lastCommit..pin` is an endpoint-tree compare, ancestry not required) so a force-push / `master`→`main` consolidation imports only the real delta instead of re-walking the whole tree forever (#1970); an oversized or failed diff degrades to `performFullSync`. `performFullSync` is itself authoritative for deletes — after an advancing full import it purges file-backed pages (`source_path != null` AND strategy-aware `isSyncable`) whose source file no longer exists, sparing `put_page`/manual pages (null `source_path`) and metafiles. `resolveSlugByPathOrSourcePath` at `sync.ts:267` delegates to `engine.resolveSlugsByPaths` when `sourceId` is set, keeping legacy `executeRaw` fallback for the no-sourceId path. `failedFiles` is hoisted to the top of `performSyncInner` so both delete-decompose and import loops feed the same bookmark gate. The `sync --all` cost gate is mode-aware via `willEmbedSynchronously` + `shouldBlockSync` from `src/core/embedding.ts` (federated_v2 resolved once up front): the DEFERRED path (v2 on, parallel) is INFORMATIONAL (embedding goes to per-source `embed-backfill` jobs with their own `$X/source/24h` cap, default $25 via `SPEND_CAP_CONFIG_KEY` from `embed-backfill-submit.ts`; prints the cap + backlog `sumStaleChunkChars({signature})` priced via `estimateCostFromChars` at `currentEmbeddingPricePerMTok()` + queued-job count, NEVER exits 2); the INLINE path BLOCKS only when the new-content estimate exceeds `sync.cost_gate_min_usd` (default $0.50). The new-content estimate (`estimateInlineNewTokens`) contributes ZERO for sources provably unchanged since last sync (`isSourceUnchangedSinceSync` HEAD==last_commit + clean tree AND `chunker_version === CURRENT`) and the full-tree ceiling for changed sources; the pre-existing stale backlog is shown informationally but NEVER gated on (sync doesn't sweep it inline). Helpers `resolveCostGateFloorUsd(engine)` (fail-open $0.50, accepts 0) + `resolveBackfillCapUsd(engine)`. JSON envelopes gain `mode` + `gate` discriminators (`dry_run | deferred_notice | below_floor | confirmation_required`); `SyncStatusReportSource` gains `backfill_queued`/`backfill_active`/`backfill_last_completed_at`; cost previews read `getEmbeddingModelName()` (no hardcoded OpenAI). `SyncOpts.noSchemaPack` (CLI `--no-schema-pack`, threaded through `performSync` AND `syncOneSource`) skips `loadActivePack` so pages fall back to legacy prefix typing — an escape hatch when a suspect pack regex wedges a sync. A per-file BEGIN heartbeat `if (process.env.GBRAIN_SYNC_TRACE) serr('[sync] begin import: ')` fires BEFORE `importFile` (the `progress.tick` fires only AFTER) so a stuck file is a begin-line with no matching completion. Triage doc: `docs/architecture/serve-sync-concurrency.md` (PGLite single-writer serve↔sync contention + the `GBRAIN_SYNC_TRACE` + `--no-schema-pack` recipes). Pinned by `test/e2e/sync-status-pglite.test.ts` (IRON-RULE: PGLite seeds 2 sources × pages × chunks, soft-deletes 1 page, archives 1 source, validates the SQL excludes both AND uses the active embedding column), `test/sync-cost-gate.serial.test.ts`, `test/sync-cost-preview.test.ts`. Runaway-sync protection: `resolveSyncHardDeadline(args, {isTty, env, defaultNonTtySec?})` resolves a wall-clock hard deadline (precedence `--no-hard-deadline` > `--hard-deadline ` > `--timeout `(non-`--all`, which auto-arms the backstop) > `GBRAIN_SYNC_MAX_RUNTIME_SECONDS` env > non-TTY default 3600s > none; `HARD_DEADLINE_GRACE_SEC=30`). `src/cli.ts` installs the out-of-band watchdog (see `src/core/process-watchdog.ts`) for the sync command BEFORE `connectEngine` and disposes it in the dispatch `finally`, so even an event-loop-starved sync — or a connect-phase hang — is SIGTERM-then-SIGKILLed by the deadline instead of orphaning under cron. `runSync` registers a SIGINT handler that aborts an interrupt `AbortController` composed via `composeAbortSignals(...)` (an `AbortSignal.any` wrapper over the defined signals) with the per-source `--timeout` signal, so Ctrl-C returns a clean `partial` and releases the lock through the normal `finally` (process-cleanup.ts owns SIGTERM lock-release; the watchdog owns the hard kill). `withRefreshingLock` `unref()`s its refresh `setInterval`. The spin's own root cause is not yet pinned (leading lead is catastrophic-backtracking in a pack link-inference regex, bounded by the redos-guard); the watchdog heartbeat plus the existing `[gbrain phase]` breadcrumbs are the diagnosis surface. Pinned by `test/sync-hard-deadline.test.ts` (resolution precedence + `composeAbortSignals`). +- `src/commands/sync.ts` — `gbrain sync` CLI + the `performSync` / `performFullSync` library entrypoints (consumed by the autopilot cycle and the Minion sync handler). `performSync` runs under a writer lock: per-source `gbrain-sync:` whenever `opts.sourceId` is set, wrapped in `withRefreshingLock` from `src/core/db-lock.ts` so long-running sources (250K+ chunks) don't lose the lock at the 30-min TTL mid-run; the bare no-source path uses the SAME refreshing lock (#1794 — it was previously a non-refreshing `tryAcquireDbLock`, stealable mid-run during an incident); `SyncOpts.lockId?: string` is the explicit override. The lock refresh AND its health probe route through the DIRECT session pool so Supavisor transaction-pool exhaustion (`EMAXCONNSESSION`) can't kill renewal; takeover is heartbeat-aware (it will NOT steal a holder whose `last_refreshed_at` is within `GBRAIN_LOCK_STEAL_GRACE_SECONDS`, defending an alive-but-starved holder); the import loop yields the event loop every `GBRAIN_SYNC_YIELD_EVERY` files (`setTimeout(0)`, not `setImmediate` — Bun starves the timers phase) so the refresh `setInterval` heartbeat fires mid-import. This lock-identity invariant prevents a `sync --all` per-source worker racing `sync --source foo` on the global lock from corrupting the same source. `performSync` throws a typed `SyncLockBusyError` when the writer lock is held; the Minion `sync` handler (`src/commands/jobs.ts`) catches it and marks the job SKIPPED (not failed) so a cron/autopilot tick defers to the holder without polluting crash metrics. `performSyncInner` is RESUMABLE (incremental path): it drains a PINNED target commit (`lastCommit..pin`), banking drained file paths via `appendCompleted` (append-only delta into the `op_checkpoint_paths` child table, migration v115 — one row per path, O(delta) not the old O(N²) full-array rewrite), keyed by `syncFingerprint({sourceId, lastCommit})` from `src/core/op-checkpoint.ts` (paths under `op:'sync'`; the pinned target under `op:'sync-target'`), and advances `last_commit`/`last_sync_at` ONLY at full import completion. Checkpoint writes route through the DIRECT session pool + bounded retry so they survive `EMAXCONNSESSION`; the flush cadence is first-file then every `GBRAIN_SYNC_CHECKPOINT_EVERY` (default 1000) files OR `GBRAIN_SYNC_CHECKPOINT_SECONDS` (default 10s), with a race-safe `pendingCheckpointPaths` delta (single-flight swap, re-merge on failure) under parallel workers; a SIGTERM banks the in-flight delta via a no-retry one-shot (`appendCompletedOnce`, ordered before lock release through `registerCleanup`); and sustained flush failure aborts the run with `reason:'checkpoint_unavailable'` after `GBRAIN_SYNC_MAX_CHECKPOINT_FAILURES` consecutive fails rather than importing work it can never bank (every partial/blocked exit logs the banked-file count). A sync killed mid-import banks its progress, leaves the anchor unmoved (the source stays correctly stale to the autopilot scheduler — `last_sync_at` is never bumped on a partial), and the next run `resumeFilter`s the same fixed diff to skip done files. The pin is the checkpoint's stored target when still reachable from HEAD, else live HEAD (a history rewrite / reset re-pins); completion advances to the pin, NOT live HEAD, so commits landing past the pin are a clean next-sync diff — this closes the cross-run staleness window. After import a pin-reachability gate (`git merge-base --is-ancestor pin HEAD`) replaces the old strict head-drift gate: forward commits on top of the pin (e.g. a background enrich process committing to the same repo every ~2 min) are SAFE and no longer block the run; only a real rewrite (pin not an ancestor of HEAD) blocks and discards the checkpoint. A file added in `lastCommit..pin` but gone from disk (deleted by a commit after the pin) is SKIPPED and marked completed, not recorded as a failure. Downstream extract/facts/embed are size-gated: inline only for `totalChanges <= 100`; large syncs defer to the resumable `extract --stale` watermark + `embed --stale`/backfill + the facts cycle phases, so a 44K-page facts/embed pass never re-blocks import convergence (sync convergence == import convergence). Worker engines wrap in try/finally so disconnect always fires; both PGLite-detection sites use `engine.kind === 'pglite'`. CLI accepts `--workers N` (alias `--concurrency N`) validated via `parseWorkers` (explicit bypasses the file-count floor; auto path defers to `autoConcurrency()`). The newest-first descending-lex order uses `sortNewestFirst(addsAndMods)` from `src/core/sort-newest-first.ts` (shared with `gbrain import`). `gbrain sync --all` runs a continuous worker pool: `parseWorkers`-validated `--parallel N` (default `min(sourceCount, --workers, DEFAULT_PARALLEL_SOURCES=4)`), long-lived async workers pulling from a shared FIFO queue (no head-of-line blocking), per-source `withSourcePrefix(src.id, ...)` so every `slog`/`serr` line carries `[]`; `--skip-failed`/`--retry-failed` are scoped per source (`acknowledgeFailures(sourceId)`; `--all` acks every source, single-source acks only its own) and run UNDER parallel — the #1939 failure ledger is per-`(source_id, path)` and serialized through `withLedgerLock`, so the old "not supported under parallel, re-run with --serial" refusal is lifted (#2139, which also removed the forcing-function that pushed recovery syncs to `--serial` and thus armed the inline cost gate); a connection-budget stderr warning fires when `parallel × workers × 2 > 16` (the `× 2 per-file pool` factor: each per-file worker opens its own `PostgresEngine` with `poolSize=2`). Exports `resolveParallelism`, `syncOneSource`, `buildSyncStatusReport`, `printSyncStatusReport`, `SyncStatusReport` back the `gbrain sources status` dashboard. `--json` envelope `{schema_version: 1, sources, parallel, ok_count, error_count}` on stdout; human banners route to stderr via `humanSink` so `jq` parses cleanly. Exit matrix: 0 all ok, 1 any error. (The non-TTY cost gate no longer exits 2 — it auto-defers; #2139.) The dashboard SQL is `content_chunks ch JOIN pages pg ON pg.id = ch.page_id WHERE pg.deleted_at IS NULL` with `archived = false` at the caller; embedding column resolved via `resolveEmbeddingColumn(undefined, cfg)` from `src/core/search/embedding-column.ts` so Voyage/multimodal/non-default-column brains count against the column they use; errors propagate (no swallow-catch). The sync delete loop is interleaved per-batch resolve+delete using `engine.resolveSlugsByPaths` + `engine.deletePages` from `src/core/engine.ts` (73K-delete commit: ~146K SQL round-trips → ~292, closing the cascade-staleness class where one big-delete commit jammed every other source's sync); per-batch try-catch decomposes batch DELETE failures to per-slug `deletePage` fallback, unrecoverable per-slug failures land in `failedFiles`; `pagesAffected` filters to confirmed-deleted slugs. The rename loop gets the same batched slug-resolve; a rename whose destination is un-syncable folds the source path into the delete set so the old page can't orphan (#1970). An entry-time bookmark-reachability guard distinguishes a gc'd anchor (`cat-file` fails → `performFullSync`) from a history-rewrite anchor that is merely no-longer-an-ancestor but still on disk: the latter is diffed tree-to-tree directly (`git diff lastCommit..pin` is an endpoint-tree compare, ancestry not required) so a force-push / `master`→`main` consolidation imports only the real delta instead of re-walking the whole tree forever (#1970); an oversized or failed diff degrades to `performFullSync`. `performFullSync` is itself authoritative for deletes — after an advancing full import it purges file-backed pages (`source_path != null` AND strategy-aware `isSyncable`) whose source file no longer exists, sparing `put_page`/manual pages (null `source_path`) and metafiles. `resolveSlugByPathOrSourcePath` at `sync.ts:267` delegates to `engine.resolveSlugsByPaths` when `sourceId` is set, keeping legacy `executeRaw` fallback for the no-sourceId path. `failedFiles` is hoisted to the top of `performSyncInner` so both delete-decompose and import loops feed the same bookmark gate. The cost gate is the shared `runInlineCostGate` (one implementation on BOTH the `--all` and single-source paths; runs at the command layer, never inside `performSync`), mode-aware via `willEmbedSynchronously` + posture-aware `shouldBlockSync` from `src/core/embedding.ts` (#2139). The DEFERRED path (v2 on, parallel) is INFORMATIONAL (embedding goes to per-source `embed-backfill` jobs with their own `$X/source/24h` cap, default $25 via `SPEND_CAP_CONFIG_KEY` from `embed-backfill-submit.ts`; prints the cap + backlog + queued-job count, NEVER exits 2). The INLINE path (v2 off, or `--serial`) gates on the DELTA estimate vs `sync.cost_gate_min_usd` (default $0.50): below floor proceeds; above floor in a TTY prompts `[y/N]`; above floor in a non-TTY/`--json` session AUTO-DEFERS embeds to capped backfill jobs and exits 0 (NEVER exit 2 — the wedged-cron fix); `spend.posture=tokenmax` makes it informational and proceeds inline. The estimate MIRRORS EXECUTION instead of pricing the whole tree: `estimateInlineNewTokens` routes through the shared `computeSyncDelta` (`src/core/sync-delta.ts`) — fetch-first against `origin/`, prices only the committed delta (a dirty-but-caught-up tree → $0), with a full-tree CEILING only on the fail-open rungs (chunker drift, first sync, git-unavailable) honestly labeled; `--full` adds the stale backlog (full sync sweeps it inline). Return shape carries `estimateKind: 'delta'|'ceiling'|'mixed'|'unchanged'` + `ceilingReasons`. Helpers `resolveCostGateFloorUsd(engine)` + `resolveBackfillCapUsd(engine)` resolve via `parseUsdLimit` (`off`/`unlimited` → `Infinity`; floor accepts `0` = block-on-any-spend). JSON envelopes carry `mode` + `gate` discriminators (`dry_run | deferred_notice | below_floor | auto_deferred_embeds | posture_tokenmax`) + a paste-ready `hint`; `Infinity` floors/caps render as the string `'unlimited'` (never raw, which JSON-serializes to `null`); `SyncStatusReportSource` gains `backfill_queued`/`backfill_active`/`backfill_last_completed_at`; cost previews read `getEmbeddingModelName()` (no hardcoded OpenAI). Format splits on the explicit `--json` flag only (human text otherwise). `SyncOpts.noSchemaPack` (CLI `--no-schema-pack`, threaded through `performSync` AND `syncOneSource`) skips `loadActivePack` so pages fall back to legacy prefix typing — an escape hatch when a suspect pack regex wedges a sync. A per-file BEGIN heartbeat `if (process.env.GBRAIN_SYNC_TRACE) serr('[sync] begin import: ')` fires BEFORE `importFile` (the `progress.tick` fires only AFTER) so a stuck file is a begin-line with no matching completion. Triage doc: `docs/architecture/serve-sync-concurrency.md` (PGLite single-writer serve↔sync contention + the `GBRAIN_SYNC_TRACE` + `--no-schema-pack` recipes). Pinned by `test/e2e/sync-status-pglite.test.ts` (IRON-RULE: PGLite seeds 2 sources × pages × chunks, soft-deletes 1 page, archives 1 source, validates the SQL excludes both AND uses the active embedding column), `test/sync-cost-gate.serial.test.ts`, `test/sync-cost-preview.test.ts`. Runaway-sync protection: `resolveSyncHardDeadline(args, {isTty, env, defaultNonTtySec?})` resolves a wall-clock hard deadline (precedence `--no-hard-deadline` > `--hard-deadline ` > `--timeout `(non-`--all`, which auto-arms the backstop) > `GBRAIN_SYNC_MAX_RUNTIME_SECONDS` env > non-TTY default 3600s > none; `HARD_DEADLINE_GRACE_SEC=30`). `src/cli.ts` installs the out-of-band watchdog (see `src/core/process-watchdog.ts`) for the sync command BEFORE `connectEngine` and disposes it in the dispatch `finally`, so even an event-loop-starved sync — or a connect-phase hang — is SIGTERM-then-SIGKILLed by the deadline instead of orphaning under cron. `runSync` registers a SIGINT handler that aborts an interrupt `AbortController` composed via `composeAbortSignals(...)` (an `AbortSignal.any` wrapper over the defined signals) with the per-source `--timeout` signal, so Ctrl-C returns a clean `partial` and releases the lock through the normal `finally` (process-cleanup.ts owns SIGTERM lock-release; the watchdog owns the hard kill). `withRefreshingLock` `unref()`s its refresh `setInterval`. The spin's own root cause is not yet pinned (leading lead is catastrophic-backtracking in a pack link-inference regex, bounded by the redos-guard); the watchdog heartbeat plus the existing `[gbrain phase]` breadcrumbs are the diagnosis surface. Pinned by `test/sync-hard-deadline.test.ts` (resolution precedence + `composeAbortSignals`). - `src/commands/import.ts` — `gbrain import` CLI + `runImport` library entrypoint. Uses a path-set checkpoint via `src/core/import-checkpoint.ts` (the walk still applies `sortNewestFirst()` for embed-cost ordering, but checkpoint correctness no longer depends on sort order). A file enters `completed: Set` only when its `processFile` returns success (including content-hash short-circuit no-ops); failed files never enter the set so the next run retries them automatically with no manual `~/.gbrain/import-checkpoint.json` delete. This closes three classes: parallel-import-with-slow-worker dropping the slow file on crash-resume (the slow file isn't in `completed` until its own `processFile` resolves), failed-file-bumps-counter-past-itself (failures don't add to `completed`), and sort-flip-drops-newest-N-on-cross-version-resume (order is no longer part of the checkpoint). Old positional checkpoints are detected and discarded with a stderr line on first resume (re-walking is cheap because `content_hash` short-circuits unchanged files). Checkpoint persists every 100 successful adds, not every 100 processed files. The `managedBookmark` opt (set by `performFullSync` when `runImport` is the full-sync engine) suppresses `runImport`'s own `sync.last_commit` advance so the shared `applySyncFailureGate` (`src/core/sync-failure-ledger.ts`) owns the bookmark + failure-ledger gating on that path — one gate decides advance/block/auto-skip across both sync paths. Pinned by `test/import-checkpoint.test.ts` + `test/import-resume.test.ts` (incl. the SLUG_MISMATCH retry regression). - `src/core/import-checkpoint.ts` — `loadCheckpoint(brainDir)`, `saveCheckpoint(brainDir, completed)`, `resumeFilter(files, completed, brainDir)`, `clearCheckpoint()`, plus the `ImportCheckpoint` type. Path-set format `{schema_version, brainDir, completed: string[]}`. Atomic write via `.tmp` + `rename()` so a mid-write crash never leaves a partial JSON. `loadCheckpoint` returns `null` on: missing file, malformed JSON, brainDir mismatch (ran against a different brain), and the old positional format (logged to stderr before discard). `resumeFilter` returns `{toProcess, skippedCount}` — pure, no I/O, deterministic. `clearCheckpoint` is no-op-on-missing for clean-exit cleanup. Honors `GBRAIN_HOME` via `gbrainPath()` so `withEnv({GBRAIN_HOME: tmpdir})` test isolation works without monkey-patching fs. Best-effort persistence — `saveCheckpoint` logs warnings on write errors but never throws. - `src/core/sort-newest-first.ts` — single source of truth for the descending-lex sort that `gbrain import` and `gbrain sync` both apply. Mutates in place (Array.prototype.sort semantics), returns the same array reference for fluent chaining. Empty/single-element inputs short-circuit. Future ordering changes flip one line here instead of touching two CLI commands. Pinned by `test/sort-newest-first.test.ts` (descending order, mixed prefixes, empty, single-element, in-place-mutation contract). diff --git a/docs/operations/spend-controls.md b/docs/operations/spend-controls.md new file mode 100644 index 000000000..1e2fca9ee --- /dev/null +++ b/docs/operations/spend-controls.md @@ -0,0 +1,111 @@ +# Spend controls + +GBrain's embedding-spend gates in one place: every gate, its config key, default, +whether it blocks or just informs, how to widen or disable it, and how the +`spend.posture` switch governs all of them. + +The orienting idea: **GBrain itself is rounding error; the spend that matters is +downstream embedding.** These gates exist so a routine sync or enrich can't run up +an unexpected embedding bill, while never wedging an unattended cron. + +## `spend.posture` — one switch for "cost is not my constraint" + +```bash +gbrain config set spend.posture tokenmax # all cost gates become informational +gbrain config set spend.posture gated # default — gates enforce +``` + +| Value | Effect | +|-------|--------| +| `gated` (default) | Every cost gate enforces its limit as documented below. | +| `tokenmax` | Every cost gate prints its estimate and **proceeds** — informational only. Spend is still recorded to the ledger; posture removes the *ceiling*, not the *accounting*. | + +`spend.posture` is deliberately separate from `search.mode=tokenmax` (which governs +retrieval payload size, not embedding spend). When a gate fires and +`search.mode=tokenmax` but `spend.posture` is unset, the gate prints a one-line hint +pointing at this switch. + +**Precedence:** an explicit per-call cap (`--max-usd N`, `--max-cost N`) always wins +over posture. `tokenmax` only governs the default/absent case — it never overrides a +number you typed on the command line. + +## Off switches (`off` / `unlimited` / `none`) + +The USD-limit knobs accept `off`, `unlimited`, or `none` (case-insensitive) to mean +"no limit" — no more setting sentinel values like `100000`. + +- `0` is **not** "off". On `sync.cost_gate_min_usd`, `0` means "block on any nonzero + spend" (a real choice). On the backfill caps, `0` falls back to the default. +- Internally "no limit" is the string `unlimited` in any printed/JSON output and "no + cap" inside the budget tracker — never a raw `Infinity` (which would serialize to + `null` in ledger rows). + +## The gates + +| Gate | Config key | Default | Blocks? | Off switch | tokenmax | +|------|-----------|---------|---------|-----------|----------| +| Sync inline-embed cost gate | `sync.cost_gate_min_usd` | `0.50` | TTY prompt / non-TTY auto-defer | `off` (or `0` = block-on-any) | informational | +| Backfill 24h per-source spend cap | `embed.backfill_max_usd_per_source_24h` | `25` | refuses submission | `off` (`0` → default) | bypassed (still ledgered) | +| Backfill per-job budget | `embed.backfill_max_usd` | `10` | caps the job's tracker | `off` (`0` → default) | uncapped (still ledgered) | +| Backfill cooldown | `embed.backfill_cooldown_min` | `10` | skips re-submission inside window | — (latency knob, not spend) | **not** bypassed | +| `reindex-code` cost gate | — (preview before re-embed) | — | TTY prompt / non-TTY refuse + exit 2 | `--max-cost off` | informational | +| `enrich` / `onboard --auto` | `--max-usd` (per-call) | — | refuse without a cap (non-TTY) | `--max-usd off` | runs uncapped (still ledgered) | + +### Sync inline-embed cost gate + +Fires only when sync embeds **inline** (federated_v2 off, or `--serial` without +`--no-embed`). Under federated_v2 + parallel, embedding is deferred to capped backfill +jobs and the gate is informational. The estimate prices the **delta** — the files this +sync will actually import (fetched-first, so it sees commits the run is about to pull) — +not the whole tree. A busy brain with a dirty working tree but caught-up commits +estimates `$0`, because an attached-HEAD sync imports only the committed diff. + +Behavior above the floor: +- **TTY:** prompts `[y/N]`. +- **Non-interactive (cron/agent):** **auto-defers** embeds to capped backfill jobs and + exits 0 — it never wedges the pipeline. The backlog drains via the jobs worker or + `gbrain embed --stale`. Pass `--yes` to embed inline instead. + +Output format splits on the explicit `--json` flag: `--json` emits a structured +envelope; otherwise human text. Every gate message carries paste-ready knobs. + +`--full` re-embeds the stale backlog inline (full sync sweeps it), so a `--full` +estimate is `delta + stale backlog`, labeled as such. + +### Estimate labels + +- `~N tokens (delta: changed files since last sync)` — the precise estimate. +- `<=N tokens (full-tree ceiling for K source(s): …)` — a conservative + over-count used only when a precise delta can't be computed: a first sync, a chunker + version drift (forces a full re-chunk), or git being unavailable. Unchanged files + still skip via `content_hash` at execution, so the ceiling over-states real spend. + +## Notes & limits + +- **Pre-pull window:** the gate fetches before estimating, so it prices what the run + will pull. If a fetch fails (offline), it estimates against local HEAD and labels the + result; the bounded residual is priced on the next run. +- **Single-source `gbrain sync`** carries the same gate as `sync --all` (it previously + embedded inline with no preview). +- **Recovery under parallel:** `--skip-failed` / `--retry-failed` work under parallel + sync (the failure ledger is per-source and lock-serialized) — you no longer have to + drop to `--serial`, which is what used to arm the inline gate. + +## Escape hatches at a glance + +```bash +# Never gate this brain on cost: +gbrain config set spend.posture tokenmax + +# Widen the sync inline floor to $5: +gbrain config set sync.cost_gate_min_usd 5 + +# Disable the sync inline floor entirely: +gbrain config set sync.cost_gate_min_usd off + +# Lift the backfill 24h spend cap: +gbrain config set embed.backfill_max_usd_per_source_24h off + +# Run enrich uncapped non-interactively: +gbrain enrich --max-usd off # or: gbrain config set spend.posture tokenmax +``` diff --git a/llms-full.txt b/llms-full.txt index 1ef630b27..8e180ce41 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -246,6 +246,7 @@ detail on demand.) | any file in `src/` (what it does + its invariants) | `docs/architecture/KEY_FILES.md` — find the file's entry | | search / ranking / hybrid / retrieval | `docs/architecture/RETRIEVAL.md` + the `search/*` entries in `KEY_FILES.md` | | search modes / cost knobs | `docs/guides/search-modes.md` | +| embedding spend gates / cost gate / `spend.posture` / off switches | `docs/operations/spend-controls.md` | | push-based context (volunteer/watch/reflex window) | `docs/guides/push-context.md` | | schema packs / page types / extraction | `docs/architecture/schema-packs.md`, `type-taxonomy.md`, `lens-packs.md` | | thin-client / remote MCP / cross-modal | `docs/architecture/thin-client.md` | diff --git a/package.json b/package.json index 9ae1a255f..9ba4444fd 100644 --- a/package.json +++ b/package.json @@ -143,5 +143,5 @@ "bun": ">=1.3.10" }, "license": "MIT", - "version": "0.42.44.0" + "version": "0.42.45.0" } diff --git a/src/commands/config.ts b/src/commands/config.ts index 075de0ca5..d4e0bb1c6 100644 --- a/src/commands/config.ts +++ b/src/commands/config.ts @@ -181,6 +181,20 @@ export async function runConfig(engine: BrainEngine, args: string[]) { const coverageOverride = args.includes('--coverage-override') || args.includes('--yes'); + // v0.42.42.0 (#2139): validate spend.posture at set time so a typo + // ('tokenMax', 'max') doesn't silently fall back to gated. + if (key === 'spend.posture') { + const { isValidSpendPosture } = await import('../core/spend-posture.ts'); + if (!isValidSpendPosture(value)) { + console.error( + `[config] spend.posture must be 'gated' or 'tokenmax' (got '${value}').\n` + + `[config] gbrain config set spend.posture tokenmax # cost gates become informational\n` + + `[config] gbrain config set spend.posture gated # default — gates enforce`, + ); + process.exit(1); + } + } + if (key === 'embedding_columns') { try { const parsed = JSON.parse(value); diff --git a/src/commands/enrich.ts b/src/commands/enrich.ts index b859f0669..99a04ed24 100644 --- a/src/commands/enrich.ts +++ b/src/commands/enrich.ts @@ -508,8 +508,13 @@ export async function runEnrichCore( // One tracker reference for both the run and the post-hoc overage check. // External tracker (cycle phase): used as-is, no withBudgetTracker wrap (that // would REPLACE not stack). Internal: capped at maxCostUsd ?? DEFAULT. + // v0.42.42.0 (#2139): Infinity = explicit "uncapped" (off / tokenmax) → pass + // `undefined` so BudgetTracker runs without a ceiling (NOT raw Infinity, which + // would serialize to null in audit rows). undefined-when-unset still → DEFAULT. + const resolvedCap = + opts.maxCostUsd === Infinity ? undefined : (opts.maxCostUsd ?? DEFAULT_MAX_COST_USD); const tracker = opts.budgetTracker ?? new BudgetTracker({ - maxCostUsd: opts.maxCostUsd ?? DEFAULT_MAX_COST_USD, + maxCostUsd: resolvedCap, label: `enrich:${sourceId}`, }); try { @@ -622,8 +627,15 @@ export function parseArgs(args: string[]): ParsedArgs { continue; } if (a === '--max-usd' || a === '--max-cost-usd') { - const n = parseFloat(args[++i] ?? ''); - if (Number.isFinite(n) && n > 0) out.maxCostUsd = n; + const raw = args[++i] ?? ''; + // v0.42.42.0 (#2139): off/unlimited/none → run uncapped (Infinity sentinel; + // mapped to "no BudgetTracker ceiling" in runEnrichCore). Spend still ledgered. + if (['off', 'unlimited', 'none'].includes(raw.trim().toLowerCase())) { + out.maxCostUsd = Infinity; + } else { + const n = parseFloat(raw); + if (Number.isFinite(n) && n > 0) out.maxCostUsd = n; + } continue; } if (a === '--min-context') { @@ -801,9 +813,24 @@ export async function runEnrich(engine: BrainEngine, args: string[]): Promise posture). + const explicitOff = parsed.maxCostUsd === Infinity; + const { resolveSpendPosture } = await import('../core/spend-posture.ts'); + const posture = parsed.dryRun ? 'gated' : await resolveSpendPosture(engine); + const uncapped = + !parsed.dryRun && (explicitOff || (parsed.maxCostUsd === undefined && posture === 'tokenmax')); + if (uncapped) { + console.error(`${explicitOff ? '--max-usd off' : 'spend.posture=tokenmax'}: running uncapped, spend ledgered. docs: docs/operations/spend-controls.md`); + } + // Non-TTY execute without --max-usd or --yes is refused (cost guardrail). - if (!parsed.dryRun && parsed.maxCostUsd === undefined && !parsed.yes && !process.stdout.isTTY) { - console.error('Refusing to spend without a cap in a non-interactive context. Pass --max-usd or --yes.'); + if (!parsed.dryRun && parsed.maxCostUsd === undefined && !parsed.yes && !process.stdout.isTTY && !uncapped) { + console.error('Refusing to spend without a cap in a non-interactive context. Pass --max-usd (or `off`), --yes, or set spend.posture=tokenmax.'); process.exit(1); } @@ -812,7 +839,7 @@ export async function runEnrich(engine: BrainEngine, args: string[]): Promise s.id); // Dry-run cost preview (TTY) before spending. - if (!parsed.dryRun && process.stdout.isTTY && !parsed.yes && parsed.maxCostUsd === undefined) { + if (!parsed.dryRun && process.stdout.isTTY && !parsed.yes && parsed.maxCostUsd === undefined && !uncapped) { const limit = parsed.limit ?? DEFAULT_LIMIT; const est = (limit * sourceIds.length * COST_ESTIMATE_PER_PAGE_USD).toFixed(2); console.error(`About to enrich up to ${limit} page(s) per source across ${sourceIds.length} source(s), est. ~$${est}. Re-run with --max-usd or --yes to confirm.`); @@ -834,7 +861,9 @@ export async function runEnrich(engine: BrainEngine, args: string[]): Promise= 0 ? (args[maxUsdIdx + 1] ?? '').trim().toLowerCase() : ''; + const maxUsdOff = ['off', 'unlimited', 'none'].includes(maxUsdVal); const maxUsdRaw = parseFloat10(args, '--max-usd'); const maxUsd = maxUsdRaw === null ? undefined : maxUsdRaw; @@ -91,13 +98,23 @@ export async function runOnboard(engine: BrainEngine, args: string[]): Promise= 0) { const v = args[idx + 1]; + const t = (v ?? '').trim().toLowerCase(); + if (['off', 'unlimited', 'none'].includes(t)) { + maxCostUsd = undefined; // no runtime cap (reindex skips the tracker when unset) + maxCostOff = true; + break; + } const n = v ? parseFloat(v) : NaN; if (!Number.isFinite(n) || n <= 0) { - console.error(`gbrain reindex --code: ${flag} requires a positive number in USD (got ${v ?? '(missing)'})`); + console.error(`gbrain reindex --code: ${flag} requires a positive number in USD, or off/unlimited (got ${v ?? '(missing)'})`); process.exit(2); } maxCostUsd = n; @@ -493,20 +503,36 @@ export async function runReindexCodeCli(engine: BrainEngine, args: string[]): Pr } if (!yes) { - const isTTY = Boolean(process.stdout.isTTY) && Boolean(process.stdin.isTTY); - if (!isTTY || json) { - // Guardrail unchanged: refuse + exit 2, no spend. Only the FORMAT splits - // on --json now (human refusal on stderr otherwise) — #1784. - const refusal = buildCostRefusal({ json, previewMsg, preview, costUsd, model: getEmbeddingModelName() }); - if (refusal.stdout) console.log(refusal.stdout); - if (refusal.stderr) console.error(refusal.stderr); - process.exit(2); - } - console.log(previewMsg); - const answer = await promptYesNo('Proceed? [y/N] '); - if (!answer) { - console.log('Cancelled.'); - return; + // v0.42.42.0 (#2139): spend.posture=tokenmax makes the gate informational + // — print the estimate and proceed (the operator declared cost isn't the + // constraint). The spend is still ledgered by the runtime BudgetTracker. + const { resolveSpendPosture } = await import('../core/spend-posture.ts'); + const posture = await resolveSpendPosture(engine); + // An explicit `--max-cost off` is the same "cost isn't the constraint" + // signal as spend.posture=tokenmax — proceed past the confirmation gate. + if (posture === 'tokenmax' || maxCostOff) { + const gate = maxCostOff ? 'max_cost_off' : 'posture_tokenmax'; + if (json) { + console.log(JSON.stringify({ status: 'proceeding', gate, codePages: preview.totalPages, totalTokens: preview.totalTokens, costUsd, model: getEmbeddingModelName() })); + } else { + console.log(`${previewMsg} ${maxCostOff ? '--max-cost off' : 'spend.posture=tokenmax'}: proceeding (informational). docs: docs/operations/spend-controls.md`); + } + } else { + const isTTY = Boolean(process.stdout.isTTY) && Boolean(process.stdin.isTTY); + if (!isTTY || json) { + // Guardrail unchanged: refuse + exit 2, no spend. Only the FORMAT splits + // on --json now (human refusal on stderr otherwise) — #1784. + const refusal = buildCostRefusal({ json, previewMsg, preview, costUsd, model: getEmbeddingModelName() }); + if (refusal.stdout) console.log(refusal.stdout); + if (refusal.stderr) console.error(refusal.stderr); + process.exit(2); + } + console.log(previewMsg); + const answer = await promptYesNo('Proceed? [y/N] '); + if (!answer) { + console.log('Cancelled.'); + return; + } } } } diff --git a/src/commands/sync.ts b/src/commands/sync.ts index e503b4e6c..09f8bafa9 100644 --- a/src/commands/sync.ts +++ b/src/commands/sync.ts @@ -7,12 +7,11 @@ import { importFile } from '../core/import-file.ts'; import { collectSyncableFiles } from './import.ts'; import { createInterface } from 'readline'; import { - buildSyncManifest, isSyncable, unsyncableReason, resolveSlugForPath, unacknowledgedSyncFailures, - acknowledgeSyncFailures, + acknowledgeFailures, loadSyncFailures, formatCodeBreakdown, applySyncFailureGate, @@ -20,6 +19,16 @@ import { resolveAutoSkipThreshold, DEFAULT_SOURCE_ID, } from '../core/sync.ts'; +import { + computeSyncDelta, + buildDetachedWorkingTreeManifest, +} from '../core/sync-delta.ts'; +import { fetchRemote } from '../core/git-remote.ts'; +import { + parseUsdLimit, + formatUsdLimit, + resolveSpendPosture, +} from '../core/spend-posture.ts'; import { estimateTokens, CHUNKER_VERSION } from '../core/chunkers/code.ts'; import { estimateEmbeddingCostUsd, @@ -28,11 +37,10 @@ import { currentEmbeddingSignature, willEmbedSynchronously, shouldBlockSync, + type SyncEmbedMode, } from '../core/embedding.ts'; import { estimateCostFromChars } from '../core/embedding-pricing.ts'; -import { isSourceUnchangedSinceSync } from '../core/git-head.ts'; import { SPEND_CAP_CONFIG_KEY } from '../core/embed-backfill-submit.ts'; -import { errorFor, serializeError } from '../core/errors.ts'; import type { SyncManifest } from '../core/sync.ts'; import { createProgress } from '../core/progress.ts'; import { getCliOptions, cliOptsToProgressOptions } from '../core/cli-options.ts'; @@ -214,16 +222,17 @@ export interface SyncResult { /** * Walk ONE source's working tree and sum tokens for every syncable file. - * Conservative full-tree ceiling (full file content, not the incremental - * diff) — over-counts, never under-counts, and matches the filesystem set - * `sync` actually imports (collectSyncableFiles + content_hash, NOT a git - * commit diff). Best-effort per file and per source: anything unreadable - * contributes 0 rather than blocking the preview. + * Conservative full-tree CEILING (full file content, not the incremental + * diff) — over-counts, never under-counts. Used only on the ceiling rungs of + * `estimateInlineNewTokens` (first sync, chunker drift, git-unavailable), + * where the delta is genuinely the whole tree or can't be computed. * * v0.31.2: routed through collectSyncableFiles (lstat + inode-cycle + * max-depth) so the preview walks exactly what the real sync walks. + * + * Exported (v0.42.42.0, #2139) for direct unit testing. */ -function estimateSourceTreeTokens( +export function estimateSourceTreeTokens( localPath: string, strategy: 'markdown' | 'code' | 'auto', ): { tokens: number; files: number } { @@ -248,18 +257,114 @@ function estimateSourceTreeTokens( return { tokens, files }; } +/** Sum tokens for an explicit set of repo-relative paths read at live working-tree content. */ +function estimateDeltaTokens(localPath: string, relPaths: string[]): number { + let tokens = 0; + for (const rel of relPaths) { + try { + const full = join(localPath, rel); + const stat = statSync(full); + if (stat.size > 5_000_000) continue; // skip large binaries (matches tree walk) + tokens += estimateTokens(readFileSync(full, 'utf-8')); + } catch { + // Listed in the diff but unreadable (e.g. since deleted) → 0, like the tree walk. + } + } + return tokens; +} + /** - * v0.41.31 — INLINE-path new-content estimate. Per source, contribute ZERO - * when the source is provably unchanged since its last sync (HEAD == - * last_commit AND clean working tree AND chunker_version matches CURRENT) — - * `content_hash` short-circuits every file so nothing re-embeds. Otherwise - * contribute the full-tree ceiling. The unchanged predicate mirrors - * doctor's `sync_freshness` and sync's own "do work?" gate (sync.ts:1057+ - * 1075). `isSourceUnchangedSinceSync` is fail-open (probe error → false), so - * a source we can't prove unchanged is conservatively re-estimated rather - * than silently priced at $0. + * v0.42.42.0 (#2139): resolve the commit the estimate should diff AGAINST. + * + * The cost gate runs BEFORE sync's own `git pull`, so a stale local HEAD would + * make the estimate blind to commits the run is about to pull (codex #1). So + * we FETCH first (fail-open) and target `origin/` — the estimate then + * prices exactly what this run will sync. The subsequent pull fast-forwards + * the already-fetched objects, so net new network cost ≈ 0. + * + * - detached HEAD → no upstream; target = local HEAD (+ caller merges the + * detached working-tree manifest, which sync imports on a detached repo). + * - attached + origin remote → fetch origin/ (best-effort), target = + * origin/ if resolvable, else local HEAD (offline / no upstream). + * - HEAD unresolvable (not a git repo) → null (caller treats as unavailable). + * + * NOTE: this makes `--dry-run` perform a network fetch so the preview reflects + * what a real run would pull. Fail-open: offline dry-run still previews against + * local HEAD. Uses the shared `git()` 30s budget (the fetch cost is the pull + * cost paid a few seconds early — a tighter cap would frequently fall back to + * local HEAD and underestimate the remote delta). */ -function estimateInlineNewTokens( +function resolveEstimateTarget(localPath: string): { target: string; detached: boolean } | null { + let head: string; + try { + head = git(localPath, ['rev-parse', 'HEAD']); + } catch { + return null; + } + const detached = isDetachedHead(localPath); + if (detached) return { target: head, detached: true }; + + let branch: string | null = null; + try { + branch = git(localPath, ['rev-parse', '--abbrev-ref', 'HEAD']).trim() || null; + } catch { + branch = null; + } + if (branch && branch !== 'HEAD' && hasOriginRemote(localPath)) { + try { + // v0.42.42.0 (#2139): route through the SSRF-hardened fetch (same flags + + // no-prompt env as pullRepo) — a cost preview / dry-run must NOT hit a + // remote through a less-protected path than real sync. + fetchRemote(localPath, branch, { timeoutMs: 30_000 }); + } catch { + // fail-open: offline, auth failure, no upstream — fall through to local HEAD. + } + try { + const remoteSha = git(localPath, ['rev-parse', `origin/${branch}`]); + if (remoteSha) return { target: remoteSha, detached: false }; + } catch { + // no remote-tracking ref for this branch — use local HEAD. + } + } + return { target: head, detached: false }; +} + +export type EstimateKind = 'delta' | 'ceiling' | 'mixed' | 'unchanged'; + +export interface InlineEstimate { + tokens: number; + changedSources: number; + unchangedSources: number; + estimateKind: EstimateKind; + /** Per-source ceiling reasons (chunker_drift / first_sync / git_unavailable) for honest labeling. */ + ceilingReasons: string[]; +} + +/** + * v0.42.42.0 (#2139) — INLINE-path new-content estimate. The estimate now + * MIRRORS EXECUTION instead of pricing the whole tree on every dirty sync (the + * 400x overestimate that wedged the daily cron). Per-source fail-open ladder: + * + * 1. syncEnabled === false → skip (unchanged) + * 2. chunker drift (stored !== current) → full-tree CEILING (a drift forces + * performFullSync → full re-chunk → full re-embed; a delta would + * underestimate by the whole corpus). kind: ceiling_chunker_drift + * 3. last_commit === fetch target → 0 (mirrors `up_to_date` at + * sync.ts:1402 — NO clean-working-tree requirement; a dirty tree whose + * commits are caught up imports nothing). kind: unchanged + * 4. last_commit === null (first sync) → full-tree CEILING. kind: ceiling_first_sync + * 5. computeSyncDelta ok → price added∪modified∪renamed.to + * (syncable, live working-tree content); deletes cost 0. kind: delta + * 6. computeSyncDelta unavailable → full-tree CEILING. kind: ceiling_git_unavailable + * + * The delta rung routes through the SAME `computeSyncDelta` the executor uses + * (src/core/sync-delta.ts), so the gate's dollar figure can't drift from what + * the sync imports. `--full`'s extra stale-backlog sweep is added by the gate + * (it already has `staleCostUsd`), not here — see the call site. + * + * Exported for direct unit testing. + */ +export function estimateInlineNewTokens( sources: Array<{ local_path: string | null; config: Record; @@ -267,25 +372,85 @@ function estimateInlineNewTokens( chunker_version: string | null; }>, currentChunkerVersion: string, -): { tokens: number; changedSources: number; unchangedSources: number } { +): InlineEstimate { let tokens = 0; let changedSources = 0; let unchangedSources = 0; + let hadDelta = false; + let hadCeiling = false; + const ceilingReasons: string[] = []; + + const ceiling = (localPath: string, strategy: 'markdown' | 'code' | 'auto', reason: string) => { + tokens += estimateSourceTreeTokens(localPath, strategy).tokens; + changedSources++; + hadCeiling = true; + ceilingReasons.push(reason); + }; + for (const src of sources) { if (!src.local_path) continue; const cfg = (src.config || {}) as { syncEnabled?: boolean; strategy?: 'markdown' | 'code' | 'auto' }; if (cfg.syncEnabled === false) continue; - const unchanged = - isSourceUnchangedSinceSync(src.local_path, src.last_commit, { requireCleanWorkingTree: true }) && - src.chunker_version === currentChunkerVersion; - if (unchanged) { + const strategy = cfg.strategy ?? 'markdown'; + const localPath = src.local_path; + + // Rung 2: chunker drift forces a full re-chunk → full re-embed. CEILING. + if (src.chunker_version !== currentChunkerVersion) { + ceiling(localPath, strategy, 'chunker_drift'); + continue; + } + + // Rung 4 (early): no bookmark → first sync imports everything. CEILING. + if (!src.last_commit) { + ceiling(localPath, strategy, 'first_sync'); + continue; + } + + const resolved = resolveEstimateTarget(localPath); + if (!resolved) { + // HEAD unresolvable (not a git repo / gone) — can't compute a delta. CEILING. + ceiling(localPath, strategy, 'git_unavailable'); + continue; + } + + // Rung 3: caught up to the fetch target AND no detached working-tree changes. + // Mirrors the executor's `up_to_date` predicate — a dirty-but-committed-current + // tree imports nothing, so it must price $0 (the heart of the false-fire fix). + const detachedManifest = resolved.detached + ? buildDetachedWorkingTreeManifest(localPath) + : null; + const detachedHasChanges = detachedManifest !== null && + (detachedManifest.added.length > 0 || + detachedManifest.modified.length > 0 || + detachedManifest.deleted.length > 0 || + detachedManifest.renamed.length > 0); + if (src.last_commit === resolved.target && !detachedHasChanges) { unchangedSources++; continue; } + + // Rung 5/6: the delta itself — SAME helper the executor diffs with. + const delta = computeSyncDelta(localPath, src.last_commit, resolved.target, { + detachedManifest, + }); + if (delta.status === 'unavailable') { + ceiling(localPath, strategy, 'git_unavailable'); + continue; + } + const syncOpts = { strategy }; + const changedPaths = unique([ + ...delta.manifest.added.filter(p => isSyncable(p, syncOpts)), + ...delta.manifest.modified.filter(p => isSyncable(p, syncOpts)), + ...delta.manifest.renamed.filter(r => isSyncable(r.to, syncOpts)).map(r => r.to), + ]); + tokens += estimateDeltaTokens(localPath, changedPaths); changedSources++; - tokens += estimateSourceTreeTokens(src.local_path, cfg.strategy ?? 'markdown').tokens; + hadDelta = true; } - return { tokens, changedSources, unchangedSources }; + + const estimateKind: EstimateKind = + hadCeiling && hadDelta ? 'mixed' : hadCeiling ? 'ceiling' : hadDelta ? 'delta' : 'unchanged'; + return { tokens, changedSources, unchangedSources, estimateKind, ceilingReasons }; } /** @@ -299,9 +464,10 @@ function estimateInlineNewTokens( async function resolveCostGateFloorUsd(engine: BrainEngine): Promise { try { const raw = await engine.getConfig('sync.cost_gate_min_usd'); - if (raw === null || raw === undefined) return 0.5; - const n = Number(raw); - return Number.isFinite(n) && n >= 0 ? n : 0.5; + // v0.42.42.0 (#2139): `0` keeps meaning "block on any nonzero spend" + // (allowZero); `off`/`unlimited`/`none` → Infinity so the gate never + // blocks (`costUsd > Infinity` is always false). + return parseUsdLimit(raw, 0.5, { allowZero: true }); } catch { return 0.5; } @@ -315,9 +481,9 @@ async function resolveCostGateFloorUsd(engine: BrainEngine): Promise { async function resolveBackfillCapUsd(engine: BrainEngine): Promise { try { const raw = await engine.getConfig(SPEND_CAP_CONFIG_KEY); - if (raw === null || raw === undefined) return 25; - const n = Number(raw); - return Number.isFinite(n) && n > 0 ? n : 25; + // v0.42.42.0 (#2139): no allowZero — `0` falls back to the default + // (off semantics ≠ 0); `off`/`unlimited`/`none` → Infinity (cap disabled). + return parseUsdLimit(raw, 25); } catch { return 25; } @@ -335,6 +501,214 @@ async function promptYesNo(question: string): Promise { }); } +// v0.42.42.0 (#2139): paste-ready knobs appended to every gate message so the +// spend-control surface is discoverable at the moment of need (humans + agents), +// not only after reading source. Closes the issue's "takes archaeology" complaint. +const SPEND_HINT = + 'widen: gbrain config set sync.cost_gate_min_usd 5 | ' + + 'never gate: gbrain config set spend.posture tokenmax | ' + + 'docs: docs/operations/spend-controls.md'; + +/** Honest token label — delta vs full-tree ceiling, with the ceiling reasons. */ +function labelEstimate(inline: InlineEstimate): string { + if (inline.estimateKind === 'unchanged') return '0 new tokens (sources caught up)'; + if (inline.estimateKind === 'delta') { + return `~${inline.tokens.toLocaleString()} new tokens (delta: changed files since last sync)`; + } + // ceiling | mixed — be explicit that this is an over-count, not the real spend. + const reasons = unique(inline.ceilingReasons).join(', ') || 'unknown'; + return ( + `<=${inline.tokens.toLocaleString()} tokens (full-tree ceiling for ${inline.changedSources} ` + + `source(s): ${reasons} — unchanged files skip via content_hash at execution)` + ); +} + +type CostGateSource = { + local_path: string | null; + config: Record; + last_commit: string | null; + chunker_version: string | null; +}; + +interface CostGateContext { + sources: CostGateSource[]; + /** Resolved embed mode. Single-source is always 'inline' (not the parallel-deferred fan-out). */ + mode: SyncEmbedMode; + dryRun: boolean; + jsonOut: boolean; + yesFlag: boolean; + full: boolean; + /** Message prefix ('sync --all' | 'sync'). */ + label: string; +} + +type CostGateOutcome = + | { action: 'proceed'; autoDeferEmbeds: boolean } + | { action: 'stop' }; + +/** + * v0.42.42.0 (#2139): the inline-embed cost gate, shared by BOTH `sync --all` + * and single-source `sync` so the spend surface is consistent. Runs at the + * COMMAND layer (never inside performSync, which `runOne` also calls — that + * would double-gate `--all`). + * + * Behavior: + * - deferred mode → FYI only, never blocks (backfill cap is the money gate). + * - inline + below floor → proceed quietly. + * - inline + spend.posture=tokenmax → informational, proceed inline. + * - inline + above floor + TTY → [y/N] prompt. + * - inline + above floor + non-TTY/--json → AUTO-DEFER embeds to capped + * backfill jobs, exit 0 (NEVER exit 2 — the wedged-cron fix). Caller sets + * effectiveNoEmbed and enqueues the backfill. + * + * Output format splits on the EXPLICIT `--json` flag only (absorbs the + * TODOS.md:340 #1784 conflation): JSON envelope iff `--json`, else human text. + */ +async function runInlineCostGate( + engine: BrainEngine, + ctx: CostGateContext, +): Promise { + const { sources, mode, dryRun, jsonOut, yesFlag, full, label } = ctx; + + // Stale backlog: cheap single SQL; fail-open to 0 so a transient DB hiccup + // never blocks the sync. Signature-aware (model/dims swap surfaces here). + let staleChars = 0; + try { + staleChars = await engine.sumStaleChunkChars({ signature: currentEmbeddingSignature() }); + } catch { + staleChars = 0; + } + const staleCostUsd = estimateCostFromChars(staleChars, currentEmbeddingPricePerMTok()); + const embeddingModelName = getEmbeddingModelName(); + const floorUsd = await resolveCostGateFloorUsd(engine); + const posture = await resolveSpendPosture(engine); + + if (mode === 'deferred') { + // Deferred path: print an FYI, NEVER block. The backfill cap is the real + // money gate. + const capUsd = await resolveBackfillCapUsd(engine); + let queuedBackfills = 0; + try { + const r = await engine.executeRaw<{ n: number }>( + `SELECT COUNT(*)::int AS n FROM minion_jobs + WHERE name = 'embed-backfill' + AND status IN ('waiting','active','delayed','waiting-children')`, + ); + queuedBackfills = Number(r[0]?.n) || 0; + } catch { + queuedBackfills = 0; + } + const deferredMsg = + `${label}: embedding deferred to backfill jobs ` + + `(capped $${formatUsdLimit(capUsd)}/source/24h, not charged by this sync). ` + + `Current backlog ~${staleChars.toLocaleString()} chars (~$${staleCostUsd.toFixed(2)} on ` + + `${embeddingModelName}) across ${sources.length} source(s); ` + + `${queuedBackfills} backfill job(s) queued.`; + if (dryRun) { + if (jsonOut) { + console.log(JSON.stringify({ status: 'dry_run', mode, gate: 'dry_run', staleChars, staleCostUsd, capUsd: formatUsdLimit(capUsd), floorUsd: formatUsdLimit(floorUsd), queuedBackfills, model: embeddingModelName })); + } else { + console.log(deferredMsg); + console.log('--dry-run: exit without syncing.'); + } + return { action: 'stop' }; + } + if (jsonOut) { + console.log(JSON.stringify({ status: 'deferred', mode, gate: 'deferred_notice', staleChars, staleCostUsd, capUsd: formatUsdLimit(capUsd), floorUsd: formatUsdLimit(floorUsd), queuedBackfills, model: embeddingModelName })); + } else { + console.log(deferredMsg); + } + return { action: 'proceed', autoDeferEmbeds: false }; + } + + // ── Inline path ─────────────────────────────────────────────── + const inline = estimateInlineNewTokens(sources, String(CHUNKER_VERSION)); + // D7A: `--full` runs `performFullSync` → `runEmbedCore({stale:true})`, which + // sweeps the pre-existing stale backlog INLINE on top of the delta. Price it. + const costUsd = estimateEmbeddingCostUsd(inline.tokens) + (full ? staleCostUsd : 0); + const fullNote = full && staleChars > 0 + ? ` (includes ~${staleChars.toLocaleString()} stale-backlog chars swept by --full)` + : ''; + const staleNote = !full && staleChars > 0 + ? ` (plus ~${staleChars.toLocaleString()} stale-backlog chars pending \`gbrain embed --stale\`)` + : ''; + const previewMsg = + `${label} preview (inline embed): ${inline.changedSources} changed source(s), ` + + `${inline.unchangedSources} unchanged; ${labelEstimate(inline)}, ` + + `est. $${costUsd.toFixed(2)} on ${embeddingModelName}${fullNote}${staleNote}.`; + + if (dryRun) { + if (jsonOut) { + console.log(JSON.stringify({ status: 'dry_run', mode, gate: 'dry_run', newTokens: inline.tokens, estimateKind: inline.estimateKind, staleChars, costUsd, floorUsd: formatUsdLimit(floorUsd), model: embeddingModelName })); + } else { + console.log(previewMsg); + console.log('--dry-run: exit without syncing.'); + } + return { action: 'stop' }; + } + + // --yes bypasses the gate entirely (embed inline, no preview). + if (yesFlag) return { action: 'proceed', autoDeferEmbeds: false }; + + // spend.posture=tokenmax → informational, proceed INLINE (operator declared + // cost isn't the constraint; don't defer). + if (posture === 'tokenmax') { + if (jsonOut) { + console.log(JSON.stringify({ status: 'proceeding', mode, gate: 'posture_tokenmax', newTokens: inline.tokens, estimateKind: inline.estimateKind, costUsd, floorUsd: formatUsdLimit(floorUsd), model: embeddingModelName, hint: SPEND_HINT })); + } else { + console.log(`${previewMsg} spend.posture=tokenmax: proceeding (informational). ${SPEND_HINT}`); + } + return { action: 'proceed', autoDeferEmbeds: false }; + } + + // Link intent: search.mode=tokenmax but spend posture unset → nudge once. + let searchModeHint = ''; + try { + const sm = await engine.getConfig('search.mode'); + if (typeof sm === 'string' && sm.trim().toLowerCase() === 'tokenmax') { + searchModeHint = + ` (search.mode=tokenmax detected — \`gbrain config set spend.posture tokenmax\` ` + + `makes cost gates informational)`; + } + } catch { + /* best-effort */ + } + + if (shouldBlockSync(costUsd, floorUsd, mode, posture)) { + const isTTY = Boolean(process.stdout.isTTY) && Boolean(process.stdin.isTTY); + if (isTTY && !jsonOut) { + // Interactive TTY: prompt [y/N]. + console.log(previewMsg + searchModeHint); + const answer = await promptYesNo('Proceed? [y/N] '); + if (!answer) { + console.log('Cancelled.'); + return { action: 'stop' }; + } + return { action: 'proceed', autoDeferEmbeds: false }; + } + // Non-TTY or --json: AUTO-DEFER embeds to capped backfill jobs. NEVER exit 2 + // (the wedged-cron fix). Format splits on the explicit --json flag only. + if (jsonOut) { + console.log(JSON.stringify({ status: 'auto_deferred', mode, gate: 'auto_deferred_embeds', newTokens: inline.tokens, estimateKind: inline.estimateKind, costUsd, floorUsd: formatUsdLimit(floorUsd), model: embeddingModelName, hint: SPEND_HINT })); + } else { + console.log( + `${previewMsg} Exceeds floor $${formatUsdLimit(floorUsd)} in a non-interactive ` + + `session — importing now, deferring embeds to capped backfill jobs. ` + + `Drain: run the jobs worker or \`gbrain embed --stale\`. Pass --yes to embed inline.\n${SPEND_HINT}`, + ); + } + return { action: 'proceed', autoDeferEmbeds: true }; + } + + // Below floor → proceed without blocking (kills inline-cron noise). + if (jsonOut) { + console.log(JSON.stringify({ status: 'below_floor', mode, gate: 'below_floor', newTokens: inline.tokens, estimateKind: inline.estimateKind, staleChars, costUsd, floorUsd: formatUsdLimit(floorUsd), model: embeddingModelName })); + } else { + console.log(`${previewMsg} Below cost gate floor ($${formatUsdLimit(floorUsd)}), proceeding.`); + } + return { action: 'proceed', autoDeferEmbeds: false }; +} + export interface SyncOpts { repoPath?: string; dryRun?: boolean; @@ -554,19 +928,9 @@ function unique(items: T[]): T[] { return [...new Set(items)]; } -function buildDetachedWorkingTreeManifest(repoPath: string): SyncManifest { - const manifest = buildSyncManifest(git(repoPath, ['diff', '--name-status', '-M', 'HEAD'])); - const untracked = git(repoPath, ['ls-files', '--others', '--exclude-standard']) - .split('\n') - .filter(line => line.length > 0); - - return { - added: unique([...manifest.added, ...untracked]), - modified: unique(manifest.modified), - deleted: unique(manifest.deleted), - renamed: manifest.renamed, - }; -} +// v0.42.42.0 (#2139): `buildDetachedWorkingTreeManifest` relocated to +// `src/core/sync-delta.ts` (re-imported below) so the inline cost estimator +// prices detached sources through the same code the executor imports them with. // v0.18.0 Step 5: source-scoped sync state helpers. When opts.sourceId // is set, read/write the per-source row instead of the global config @@ -1426,29 +1790,28 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise 0) { - const acked = acknowledgeSyncFailures(); - console.log(`Acknowledged ${acked.count} pre-existing failure(s).`); - } - } // v0.18.0 Step 5: --source resolves to a sources(id) row. Falls back // to pre-v0.17 global config (sync.repo_path + sync.last_commit) when @@ -3052,6 +3408,23 @@ See also: if (nudge) process.stderr.write(nudge + '\n'); } + // --skip-failed: acknowledge pre-existing unacked failures BEFORE the sync + // runs, not only ones the current run produces. Without this, the common + // recovery flow — fix the YAML, re-run sync, then run --skip-failed to clear + // the log — fails to clear anything (no NEW failures → the inner ack path in + // performSync is never reached, and "Already up to date." leaves the log). + // + // v0.42.42.0 (#2139, D13C): scoped PER SOURCE. `--all` clears every source's + // open failures; single-source clears only its own (don't ack source B's + // failures when syncing source A). Safe under parallel — the ledger + // serializes writes via `withLedgerLock` and keys rows by `source_id` + // (#1939), which is why the old D15 "no --skip-failed under parallel" + // refusal is lifted below. + if (skipFailed) { + const acked = syncAll ? acknowledgeFailures() : acknowledgeFailures(sourceId); + if (acked.count > 0) console.log(`Acknowledged ${acked.count} pre-existing failure(s).`); + } + // v0.19.0 — `sync --all` iterates all registered sources with a // local_path. Sources are the canonical v0.18.0 abstraction: per-source // last_commit, last_sync_at, config.federated flags. Per-source @@ -3080,132 +3453,21 @@ See also: const { isFederatedV2Enabled } = await import('../core/feature-flags.ts'); const v2Enabled = await isFederatedV2Enabled(engine); - // v0.41.31 cost gate (supersedes the v0.20.0 unconditional gate). Under - // federated_v2 sync DEFERS embedding to per-source embed-backfill jobs - // that carry their own $X/source/24h spend cap, so sync itself spends - // nothing synchronously — the gate is INFORMATIONAL (never exit 2) on - // that path. The blocking ConfirmationRequired gate fires ONLY when embed - // runs INLINE (v2 off, or --serial without --no-embed) AND the estimated - // spend exceeds `sync.cost_gate_min_usd` (default $0.50). Skipped entirely - // when --no-embed is set (user opted out; will run `embed --stale` later). + // v0.42.42.0 (#2139) cost gate — shared `runInlineCostGate`. Under + // federated_v2 + parallel, embedding is DEFERRED to per-source backfill + // jobs (own spend cap) so the gate is FYI-only. Inline mode (v2 off, or + // --serial without --no-embed) gates on the DELTA estimate: below floor + // proceeds; above floor in a non-TTY/--json session AUTO-DEFERS embeds + // (exit 0, never exit 2 — the wedged-cron fix); a TTY prompts. Skipped + // entirely when --no-embed is set. + let autoDeferEmbeds = false; if (!noEmbed) { const mode = willEmbedSynchronously({ v2Enabled, serialFlag, noEmbed }); - // Stale backlog: cheap single SQL; fail-open to 0 so a transient DB - // hiccup never blocks the sync. - let staleChars = 0; - try { - // v0.41.31: signature-aware so a model/dims swap surfaces in the - // backlog estimate (NULL signature grandfathered → not counted). - staleChars = await engine.sumStaleChunkChars({ signature: currentEmbeddingSignature() }); - } catch { - staleChars = 0; - } - const rate = currentEmbeddingPricePerMTok(); - const staleCostUsd = estimateCostFromChars(staleChars, rate); - const embeddingModelName = getEmbeddingModelName(); - const floorUsd = await resolveCostGateFloorUsd(engine); - - if (mode === 'deferred') { - // Deferred path: print an FYI, NEVER exit 2. The backfill cap is the - // real money gate (D1/D4). - const capUsd = await resolveBackfillCapUsd(engine); - // v0.41.31 (TODO-2): surface already-queued backfill jobs so a cron - // operator sees work is enqueued, not lost. Best-effort — minion_jobs - // may not exist on a brain that never ran a worker. - let queuedBackfills = 0; - try { - const r = await engine.executeRaw<{ n: number }>( - `SELECT COUNT(*)::int AS n FROM minion_jobs - WHERE name = 'embed-backfill' - AND status IN ('waiting','active','delayed','waiting-children')`, - ); - queuedBackfills = Number(r[0]?.n) || 0; - } catch { - queuedBackfills = 0; - } - const deferredMsg = - `sync --all: embedding deferred to backfill jobs ` + - `(capped $${capUsd}/source/24h, not charged by this sync). ` + - `Current backlog ~${staleChars.toLocaleString()} chars (~$${staleCostUsd.toFixed(2)} on ` + - `${embeddingModelName}) across ${sources.length} source(s); ` + - `${queuedBackfills} backfill job(s) queued.`; - if (dryRun) { - if (jsonOut) { - console.log(JSON.stringify({ status: 'dry_run', mode, gate: 'dry_run', staleChars, staleCostUsd, capUsd, floorUsd, queuedBackfills, model: embeddingModelName })); - } else { - console.log(deferredMsg); - console.log('--dry-run: exit without syncing.'); - } - return; - } - if (jsonOut) { - console.log(JSON.stringify({ status: 'deferred', mode, gate: 'deferred_notice', staleChars, staleCostUsd, capUsd, floorUsd, queuedBackfills, model: embeddingModelName })); - } else { - console.log(deferredMsg); - } - // fall through to sync — no exit 2. - } else { - // Inline path: sync embeds synchronously with no backfill cap to - // protect it, so the blocking gate applies. The BLOCKING cost is the - // new-content estimate ONLY (full-tree ceiling for changed sources; - // unchanged contribute 0) — that's what this sync actually embeds. - // The pre-existing stale backlog (NULL embeddings + signature drift) - // is NOT swept by sync; `gbrain embed --stale` clears it. So we show - // it informationally but never gate on cost this sync won't incur - // (else a model swap would block the next inline cron — F2). - const currentChunkerVersion = String(CHUNKER_VERSION); - const inline = estimateInlineNewTokens(sources, currentChunkerVersion); - const newCostUsd = estimateEmbeddingCostUsd(inline.tokens); - const costUsd = newCostUsd; - const staleNote = staleChars > 0 - ? ` (plus ~${staleChars.toLocaleString()} stale-backlog chars pending \`gbrain embed --stale\`)` - : ''; - const previewMsg = - `sync --all preview (inline embed): ${inline.changedSources} changed source(s), ` + - `${inline.unchangedSources} unchanged; ~${inline.tokens.toLocaleString()} new tokens, ` + - `est. $${costUsd.toFixed(2)} on ${embeddingModelName}${staleNote}.`; - - if (dryRun) { - if (jsonOut) { - console.log(JSON.stringify({ status: 'dry_run', mode, gate: 'dry_run', newTokens: inline.tokens, staleChars, costUsd, floorUsd, model: embeddingModelName })); - } else { - console.log(previewMsg); - console.log('--dry-run: exit without syncing.'); - } - return; - } - - if (!yesFlag) { - if (shouldBlockSync(costUsd, floorUsd, mode)) { - const isTTY = Boolean(process.stdout.isTTY) && Boolean(process.stdin.isTTY); - if (!isTTY || jsonOut) { - // Agent-facing path: emit structured envelope, exit 2. - const envelope = serializeError(errorFor({ - class: 'ConfirmationRequired', - code: 'cost_preview_requires_yes', - message: previewMsg, - hint: 'Pass --yes to proceed, or --dry-run to see the preview and exit 0.', - })); - console.log(JSON.stringify({ error: envelope, mode, gate: 'confirmation_required', newTokens: inline.tokens, staleChars, costUsd, floorUsd, model: embeddingModelName })); - process.exit(2); - } - // Interactive TTY path: prompt [y/N]. - console.log(previewMsg); - const answer = await promptYesNo('Proceed? [y/N] '); - if (!answer) { - console.log('Cancelled.'); - return; - } - } else { - // Below floor → proceed without blocking (kills inline-cron noise). - if (jsonOut) { - console.log(JSON.stringify({ status: 'below_floor', mode, gate: 'below_floor', newTokens: inline.tokens, staleChars, costUsd, floorUsd, model: embeddingModelName })); - } else { - console.log(`${previewMsg} Below cost gate floor ($${floorUsd.toFixed(2)}), proceeding.`); - } - } - } - } + const gate = await runInlineCostGate(engine, { + sources, mode, dryRun, jsonOut, yesFlag, full, label: 'sync --all', + }); + if (gate.action === 'stop') return; + autoDeferEmbeds = gate.autoDeferEmbeds; } // v0.40.5.0 Federated Sync v2 (master) + v0.40.6.0 layering (this branch): @@ -3266,7 +3528,12 @@ See also: const runOne = async (src: typeof sources[number]): Promise => { const cfg = (src.config || {}) as { strategy?: 'markdown' | 'code' | 'auto' }; // D18: parallel path defers embed; auto-enqueue embed-backfill after. - const effectiveNoEmbed = v2Enabled && !serialFlag && !noEmbed ? true : noEmbed; + // v0.42.42.0 (#2139): `autoDeferEmbeds` (the inline gate tripped in a + // non-TTY session) ALSO forces deferral — global by design (the gate's + // decision unit is the aggregate estimate; deferral strictly dominates + // the exit-2 it replaced for every source). + const effectiveNoEmbed = + (v2Enabled && !serialFlag && !noEmbed ? true : noEmbed) || autoDeferEmbeds; // v0.41.13.0 (T6 / D-V3-3 / D-V4-mech-6) — per-source AbortController. // // When the user passes --timeout, each source gets its OWN @@ -3329,8 +3596,11 @@ See also: // v0.41.13.0 (T7 / D-V3-5): partial excluded — the next clean sync // re-walks the diff and re-decides whether to enqueue embed for // pages whose content actually changed. + // v0.42.42.0 (#2139): `autoDeferEmbeds` enqueues even on the v2-OFF + // legacy path — otherwise the gate's auto-defer would strand + // NULL-embedded chunks with no queued job to embed them. if ( - v2Enabled && + (v2Enabled || autoDeferEmbeds) && !noAutoEmbed && !dryRun && result.status !== 'dry_run' && @@ -3357,18 +3627,13 @@ See also: const parallelEligible = v2Enabled && !serialFlag && engine.kind !== 'pglite' && activeSources.length > 1; - // v0.40.6.0 (D15): refuse --skip-failed / --retry-failed when running - // parallel. sync-failures.jsonl is brain-global; parallel acks race. - if (parallelEligible && (skipFailed || retryFailed)) { - const flag = skipFailed ? '--skip-failed' : '--retry-failed'; - console.error( - `Error: ${flag} is not supported under parallel sync.\n` + - ` (the sync-failures log is brain-global and parallel acks race).\n` + - ` Re-run with --serial for the recovery flow:\n` + - ` gbrain sync --all --serial ${flag}`, - ); - process.exit(1); - } + // v0.42.42.0 (#2139, D13C): the v0.40.6.0 (D15) refusal of --skip-failed / + // --retry-failed under parallel sync is LIFTED. It existed because the + // failure ledger was once brain-global with racing acks; #1939 made the + // ledger per-(source_id, path) and serialized every write through + // `withLedgerLock`, so parallel per-source acks no longer race. Lifting it + // also removes the forcing-function that pushed recovery syncs to --serial + // (and thus armed the inline cost gate) — the root cause behind #2139. // Effective parallelism — surfaced in the --json envelope so consumers // know how the run was actually dispatched. 1 in the serial fallback, @@ -3516,12 +3781,47 @@ See also: signal: composeAbortSignals(singleSourceInterrupt.signal, singleSourceController?.signal), }; + // v0.42.42.0 (#2139, Step 4b): single-source `gbrain sync` gets the SAME + // inline cost gate as `--all`. Previously single-source embedded inline with + // NO gate (only rail: the ≤100-file inline cap). Single-source always embeds + // INLINE (not the parallel-deferred fan-out), so mode is forced 'inline'. + // Skipped on --no-embed, --dry-run (performSync's own dry-run previews and + // spends nothing), and watch. Non-TTY above floor AUTO-DEFERS — so adding + // the gate can never wedge an existing cron; it converts silent ungated + // inline spend into informed inline-or-deferred spend. + let singleSourceAutoDefer = false; + if (!noEmbed && !dryRun && !watch) { + const gateRows = await engine.executeRaw<{ local_path: string | null; config: Record; last_commit: string | null; chunker_version: string | null }>( + `SELECT local_path, config, last_commit, chunker_version FROM sources WHERE id = $1`, + [sourceId], + ); + if (gateRows.length > 0) { + const gateSources = [{ + local_path: gateRows[0].local_path ?? repoPath ?? null, + config: gateRows[0].config ?? {}, + last_commit: gateRows[0].last_commit, + chunker_version: gateRows[0].chunker_version, + }]; + const gate = await runInlineCostGate(engine, { + sources: gateSources, mode: 'inline', dryRun: false, jsonOut, yesFlag, full, label: 'sync', + }); + if (gate.action === 'stop') return; + if (gate.autoDeferEmbeds) { + opts.noEmbed = true; + singleSourceAutoDefer = true; + } + } + } + // Bug 9 — --retry-failed: before running normal sync, clear acknowledgment // flags so the sync picks them up as fresh work. The actual re-attempt // happens inside the regular incremental/full loop because once the commit // pointer is behind the failures, the diff naturally revisits them. if (retryFailed) { - const failures = unacknowledgedSyncFailures(); + // v0.42.42.0 (#2139, D13C): scope the retry count to THIS source — rows + // carry source_id (#1939), so a single-source retry shouldn't report + // another source's failures. + const failures = unacknowledgedSyncFailures().filter(f => f.source_id === sourceId); if (failures.length === 0) { console.log('No unacknowledged sync failures to retry.'); } else { @@ -3564,6 +3864,27 @@ See also: manageGitignore(effectiveRepoPath, engine.kind); } } + // v0.42.42.0 (#2139, Step 4b): the inline gate auto-deferred this run's + // embeds (non-TTY, above floor) — enqueue a capped backfill job so the + // NULL-embedded chunks get embedded out of band instead of being stranded. + if ( + singleSourceAutoDefer && + result.status !== 'dry_run' && + result.status !== 'up_to_date' && + result.status !== 'partial' + ) { + try { + const { submitEmbedBackfill } = await import('../core/embed-backfill-submit.ts'); + const sub = await submitEmbedBackfill(engine, sourceId, { reason: 'sync_autodefer' }); + if (sub.status === 'submitted') { + process.stderr.write(` → embed-backfill job ${sub.jobId} queued (deferred inline embed).\n`); + } else if (sub.status === 'cooldown') { + process.stderr.write(` → embed-backfill skipped (cooldown); run \`gbrain embed --stale\` to drain now.\n`); + } + } catch (e) { + process.stderr.write(` → embed-backfill submission failed: ${e instanceof Error ? e.message : String(e)}\n`); + } + } return; } diff --git a/src/core/config.ts b/src/core/config.ts index 333e4d11a..f62dc6294 100644 --- a/src/core/config.ts +++ b/src/core/config.ts @@ -904,6 +904,15 @@ export const KNOWN_CONFIG_KEYS: readonly string[] = [ // Link resolution (issue #972) 'link_resolution', 'link_resolution.global_basename', + // Spend controls (v0.42.42.0, issue #2139). Previously `--force`-only — the + // operator had to discover these by reading source. Registered so `config + // set` accepts them directly. See docs/operations/spend-controls.md. + 'spend.posture', + 'sync.cost_gate_min_usd', + 'sync.federated_v2', + 'embed.backfill_cooldown_min', + 'embed.backfill_max_usd_per_source_24h', + 'embed.backfill_max_usd', ]; /** diff --git a/src/core/embed-backfill-submit.ts b/src/core/embed-backfill-submit.ts index 940cc941f..1b9b265ab 100644 --- a/src/core/embed-backfill-submit.ts +++ b/src/core/embed-backfill-submit.ts @@ -35,6 +35,7 @@ */ import type { BrainEngine } from './engine.ts'; import { MinionQueue } from './minions/queue.ts'; +import { parseUsdLimit, resolveSpendPosture, type SpendPosture } from './spend-posture.ts'; export const COOLDOWN_CONFIG_KEY = 'embed.backfill_cooldown_min'; export const SPEND_CAP_CONFIG_KEY = 'embed.backfill_max_usd_per_source_24h'; @@ -57,6 +58,13 @@ export interface SubmitEmbedBackfillResult { spend24hUsd?: number; /** Set when status === 'spend_capped'. Active cap. */ spendCapUsd?: number; + /** + * Set true when `spend.posture=tokenmax` waved the job past the 24h spend + * cap (#2139). The spend is still LEDGERED by the per-job BudgetTracker — + * posture removes the ceiling, not the accounting. Cooldown is NOT bypassed + * (it's queue-churn protection, not a spend gate). + */ + spendCapBypassed?: boolean; } export interface SubmitEmbedBackfillOpts { @@ -72,6 +80,8 @@ export interface SubmitEmbedBackfillOpts { nowMs?: number; /** Job priority. Default 5 (lower than autopilot's 0; above default jobs). */ priority?: number; + /** Override the resolved spend posture (tests). Default: read from config. */ + postureOverride?: SpendPosture; } /** @@ -133,9 +143,12 @@ export async function submitEmbedBackfill( const cooldownMin = opts.cooldownMinOverride ?? (await readIntConfig(engine, COOLDOWN_CONFIG_KEY, DEFAULT_COOLDOWN_MIN)); + // v0.42.42.0 (#2139): spend cap honors `off`/`unlimited`/`none` → Infinity. + // `0` still falls back to the default (off semantics ≠ 0). const spendCap = opts.spendCapUsdOverride ?? - (await readIntConfig(engine, SPEND_CAP_CONFIG_KEY, DEFAULT_SPEND_CAP_USD)); + (raw => parseUsdLimit(raw, DEFAULT_SPEND_CAP_USD))(await engine.getConfig(SPEND_CAP_CONFIG_KEY)); + const posture = opts.postureOverride ?? (await resolveSpendPosture(engine)); // ── Source-level cooldown ───────────────────────────────────── // Block re-submission if (a) an embed-backfill is currently active for this @@ -172,9 +185,14 @@ export async function submitEmbedBackfill( } // ── 24h rolling spend cap ───────────────────────────────────── + // v0.42.42.0 (#2139): `spend.posture=tokenmax` waves past the cap (the + // operator declared cost isn't the constraint). The per-job BudgetTracker + // still ledgers the spend — posture removes the ceiling, not the accounting. + // An `off`/`unlimited` cap (Infinity) is likewise never tripped. const spend24hFn = opts.spend24hFn ?? defaultSpend24hForSource; const spend24h = await spend24hFn(engine, sourceId); - if (spend24h >= spendCap) { + const spendCapBypassed = posture === 'tokenmax' && spend24h >= spendCap; + if (spend24h >= spendCap && !spendCapBypassed) { return { status: 'spend_capped', spend24hUsd: spend24h, @@ -194,7 +212,9 @@ export async function submitEmbedBackfill( }, ); - return { status: 'submitted', jobId: job.id }; + return spendCapBypassed + ? { status: 'submitted', jobId: job.id, spendCapBypassed: true, spend24hUsd: spend24h } + : { status: 'submitted', jobId: job.id }; } /** Round timestamp down to the nearest `bucketMs` boundary. */ diff --git a/src/core/embedding.ts b/src/core/embedding.ts index 20515d313..108b854f8 100644 --- a/src/core/embedding.ts +++ b/src/core/embedding.ts @@ -218,16 +218,23 @@ export function willEmbedSynchronously(opts: { } /** - * Pure cost-gate decision. The gate BLOCKS (prompt in TTY, exit 2 envelope - * in non-TTY) only when embed runs inline AND the estimated spend exceeds - * the floor. Deferred mode NEVER blocks — the backfill cap is the real - * money gate, and blocking the cheap markdown import for cost the import - * doesn't synchronously incur is the bug this fix removes. + * Pure cost-gate decision. The gate BLOCKS (prompt in TTY, auto-defer in + * non-TTY) only when embed runs inline AND the estimated spend exceeds the + * floor. Deferred mode NEVER blocks — the backfill cap is the real money gate, + * and blocking the cheap markdown import for cost the import doesn't + * synchronously incur is the bug this fix removes. + * + * v0.42.42.0 (#2139): `spend.posture=tokenmax` makes the gate INFORMATIONAL — + * the operator has declared cost isn't the constraint, so it never blocks + * (the caller prints the estimate and proceeds inline). An `off`/`unlimited` + * floor (Infinity) is likewise never exceeded. */ export function shouldBlockSync( costUsd: number, floorUsd: number, mode: SyncEmbedMode, + posture: 'gated' | 'tokenmax' = 'gated', ): boolean { + if (posture === 'tokenmax') return false; return mode === 'inline' && costUsd > floorUsd; } diff --git a/src/core/git-remote.ts b/src/core/git-remote.ts index dbbc339d1..43f8515eb 100644 --- a/src/core/git-remote.ts +++ b/src/core/git-remote.ts @@ -131,7 +131,7 @@ export interface CloneOpts { export class GitOperationError extends Error { constructor( - public op: 'clone' | 'pull' | 'remote_get_url', + public op: 'clone' | 'pull' | 'fetch' | 'remote_get_url', message: string, public cause?: unknown, ) { @@ -217,6 +217,31 @@ export function pullRepo(repoPath: string, opts: { timeoutMs?: number } = {}): v } } +/** + * Fetch a single remote branch with the SAME SSRF-defensive flags + no-prompt + * env as cloneRepo/pullRepo (GIT_SSRF_FLAGS, --no-recurse-submodules, + * GIT_TERMINAL_PROMPT=0). Used by the sync cost-estimator's fetch-first path + * (#2139) so a cost preview / dry-run never hits a remote through a + * less-protected route than real sync. Throws GitOperationError on failure; + * the estimator catches and falls back to local HEAD. + */ +export function fetchRemote(repoPath: string, branch: string, opts: { timeoutMs?: number } = {}): void { + const args: string[] = ['-C', repoPath, ...GIT_SSRF_FLAGS, 'fetch', ...GIT_SSRF_SUBCOMMAND_FLAGS, 'origin', branch]; + try { + execFileSync('git', args, { + stdio: ['ignore', 'pipe', 'pipe'], + timeout: opts.timeoutMs ?? 30_000, + env: { ...process.env, ...GIT_ENV }, + }); + } catch (e) { + throw new GitOperationError( + 'fetch', + `git fetch failed in ${repoPath}: ${(e as Error).message}`, + e, + ); + } +} + export type RepoState = | 'healthy' | 'missing' diff --git a/src/core/minions/handlers/embed-backfill.ts b/src/core/minions/handlers/embed-backfill.ts index 221d5b676..8fb6d52ac 100644 --- a/src/core/minions/handlers/embed-backfill.ts +++ b/src/core/minions/handlers/embed-backfill.ts @@ -38,6 +38,7 @@ import { embedStaleForSource } from '../../embed-stale.ts'; import { currentEmbeddingSignature } from '../../embedding.ts'; import type { BrainEngine } from '../../engine.ts'; import type { MinionJobContext } from '../types.ts'; +import { parseUsdLimit, usdLimitToCap, resolveSpendPosture } from '../../spend-posture.ts'; const DEFAULT_MAX_USD_PER_JOB = 10; const EMBED_BACKFILL_LOCK_TTL_MIN = 60; @@ -66,12 +67,21 @@ function embedBackfillLockId(sourceId: string): string { return `gbrain-embed-backfill:${sourceId}`; } -/** Read embed.backfill_max_usd config or default. */ -async function readMaxUsd(engine: BrainEngine): Promise { +/** + * Resolve the per-job budget cap (USD) for the BudgetTracker. + * + * v0.42.42.0 (#2139): returns `undefined` = "no cap" (which BudgetTracker + * treats as cap-absent) when the config is `off`/`unlimited`/`none` OR when + * `spend.posture=tokenmax`. NEVER returns Infinity (that would pass through as + * a real ceiling and serialize to `null` in audit rows). Spend is still + * ledgered by the tracker either way — posture removes the ceiling, not the + * accounting. `0`/garbage fall back to the $10 default. + */ +async function readMaxUsd(engine: BrainEngine): Promise { + const posture = await resolveSpendPosture(engine); + if (posture === 'tokenmax') return undefined; const raw = await engine.getConfig('embed.backfill_max_usd'); - if (raw === null || raw === undefined) return DEFAULT_MAX_USD_PER_JOB; - const n = Number(raw); - return Number.isFinite(n) && n > 0 ? n : DEFAULT_MAX_USD_PER_JOB; + return usdLimitToCap(parseUsdLimit(raw, DEFAULT_MAX_USD_PER_JOB)); } /** Validate + extract typed job params. Throws on malformed input. */ diff --git a/src/core/spend-posture.ts b/src/core/spend-posture.ts new file mode 100644 index 000000000..f06d125c3 --- /dev/null +++ b/src/core/spend-posture.ts @@ -0,0 +1,106 @@ +/** + * Spend posture + USD-limit parsing — the single spend-control surface for + * gbrain's cost gates (issue #2139). Two concerns live here: + * + * 1. `spend.posture` (DB-plane config): `'gated'` (default) makes every cost + * gate behave as before; `'tokenmax'` makes them INFORMATIONAL — print the + * estimate, proceed, and keep ledgering spend. The operator who sets + * `tokenmax` has declared "cost is not my constraint." Posture removes the + * CEILING, never the ACCOUNTING (the spend ledger still records every + * dollar). It is deliberately SEPARATE from `search.mode=tokenmax` (which + * governs retrieval payload size, not embedding spend); the gate prints a + * hint linking the two when only the search mode is set. + * + * 2. `parseUsdLimit` / `formatUsdLimit`: first-class `off` / `unlimited` / + * `none` on the USD gate knobs (so operators stop setting sentinel values + * like `100000`). The parse layer represents "no limit" as `Infinity` so + * comparisons stay special-case-free (`cost > Infinity` is never true). + * `formatUsdLimit` renders it as the string `'unlimited'` — NEVER serialize + * raw `Infinity`, because `JSON.stringify(Infinity)` emits `null`, which is + * ambiguous in audit/ledger rows. At the budget-machinery boundary, callers + * convert `Infinity` → `undefined` ("no cap"), which `BudgetTracker` already + * treats as cap-absent. + * + * LEAF module: imports only the engine type so any cost-gate site can pull it + * in without a circular dependency. + */ +import type { BrainEngine } from './engine.ts'; + +export const SPEND_POSTURE_CONFIG_KEY = 'spend.posture'; + +export type SpendPosture = 'gated' | 'tokenmax'; + +/** + * Resolve `spend.posture` from DB-plane config. Fail-open to `'gated'` on a + * missing/unknown value or a config-read error — a posture probe must never + * crash a sync, and an unrecognized value must never silently disable gates. + */ +export async function resolveSpendPosture(engine: BrainEngine): Promise { + try { + const raw = await engine.getConfig(SPEND_POSTURE_CONFIG_KEY); + return normalizeSpendPosture(raw); + } catch { + return 'gated'; + } +} + +/** Pure normalizer (testable without an engine). Anything but `tokenmax` → `gated`. */ +export function normalizeSpendPosture(raw: unknown): SpendPosture { + if (typeof raw === 'string' && raw.trim().toLowerCase() === 'tokenmax') return 'tokenmax'; + return 'gated'; +} + +/** True iff `raw` is a valid `spend.posture` value (for `config set` validation). */ +export function isValidSpendPosture(raw: unknown): boolean { + return typeof raw === 'string' && ['gated', 'tokenmax'].includes(raw.trim().toLowerCase()); +} + +const OFF_TOKENS = new Set(['off', 'unlimited', 'none']); + +/** + * Parse a USD-limit config value. + * - `'off'` / `'unlimited'` / `'none'` (case-insensitive) → `Infinity` (no limit) + * - a finite positive number (or `0` when `allowZero`) → that number + * - anything else (garbage, negative, empty, NaN) → `def` + * + * `allowZero` distinguishes the two knob semantics: + * - `sync.cost_gate_min_usd` uses `allowZero: true` — `0` means "block on any + * nonzero spend" (a real operator choice). `off` is the no-limit escape. + * - the backfill caps reject `0` (fall back to the default); only `off` + * disables them. `off` semantics ≠ `0` (issue #2139). + */ +export function parseUsdLimit( + raw: unknown, + def: number, + opts: { allowZero?: boolean } = {}, +): number { + if (raw === null || raw === undefined) return def; + if (typeof raw === 'string') { + const t = raw.trim().toLowerCase(); + if (t === '') return def; + if (OFF_TOKENS.has(t)) return Infinity; + } + const n = Number(raw); + if (!Number.isFinite(n)) return def; + if (n < 0) return def; + if (n === 0) return opts.allowZero ? 0 : def; + return n; +} + +/** + * Render a USD limit for human/JSON output. `Infinity` → `'unlimited'` (NEVER + * the raw value — `JSON.stringify(Infinity)` is `null`). Finite values are + * returned as-is so callers can `$${formatUsdLimit(x)}` or embed in JSON. + */ +export function formatUsdLimit(n: number): string | number { + return Number.isFinite(n) ? n : 'unlimited'; +} + +/** + * Convert a parsed USD limit to the budget-machinery cap representation: + * `Infinity` → `undefined` ("no cap", which `BudgetTracker` treats as + * cap-absent), finite → the number. Keeps `null` out of ledger rows. + */ +export function usdLimitToCap(n: number): number | undefined { + return Number.isFinite(n) ? n : undefined; +} diff --git a/src/core/sync-delta.ts b/src/core/sync-delta.ts new file mode 100644 index 000000000..9ceb51835 --- /dev/null +++ b/src/core/sync-delta.ts @@ -0,0 +1,151 @@ +/** + * Shared sync-delta machinery — ONE implementation of "what changed since + * last_commit" consumed by BOTH the sync executor (`performSyncInner` in + * `src/commands/sync.ts`) and the inline-embed cost estimator + * (`src/core/sync-cost-estimate.ts`). Before this module the executor diffed + * `last_commit..pin` while the estimator priced the entire tree, so the + * gate's dollar figure had no relationship to what the sync actually embedded + * (issue #2139: a 400x overestimate that wedged the daily cron). Routing both + * through `computeSyncDelta` makes diff/manifest drift between estimate and + * execution structurally impossible. + * + * Shell-injection safe: `execFileSync` with array args (no `/bin/sh -c`), so a + * `sources.local_path` containing shell metacharacters can never escape — same + * posture documented at `git-head.ts:14-19`. + * + * Fail-open ladder (never throws): + * + * computeSyncDelta(repo, from, to) + * │ + * ├─ `git cat-file -t ` throws → { unavailable, anchor_missing } + * │ (bookmark object gc'd after a history rewrite — nothing to diff; + * │ caller falls back to a full reconcile / full-tree ceiling) + * │ + * ├─ `git diff --name-status -M from..to` throws → { unavailable, diff_failed } + * │ (oversized post-rewrite diff exceeds the 30s / 100 MiB budget) + * │ + * └─ ok → { ok, manifest } (+ detached working-tree manifest merged in) + * + * NOTE: a present-but-non-ancestor `from` (force-push, squash, master→main) is + * still diffable — `git diff A..B` is an endpoint-tree comparison and does NOT + * require A to be an ancestor of B (unlike a rev-walk or `A...B` merge-base). + * That is the #1970 property this module preserves. + */ +import { execFileSync } from 'node:child_process'; +import { buildSyncManifest, type SyncManifest } from './sync.ts'; + +/** Runs a git subcommand in `repoPath` and returns trimmed stdout (throws on failure). */ +export type GitRunner = (repoPath: string, args: string[]) => string; + +// Mirrors `git()` + `buildGitInvocation()` in commands/sync.ts: `core.quotepath=false` +// so non-ASCII (CJK) paths arrive as UTF-8; 30s timeout; 100 MiB maxBuffer (a +// 100K-file `--name-status` diff tops out ~10-20 MiB — Node's 1 MiB default +// would ENOBUFS-crash the sync with no log line). +const DEFAULT_GIT_RUNNER: GitRunner = (repoPath, args) => + execFileSync('git', ['-c', 'core.quotepath=false', '-C', repoPath, ...args], { + encoding: 'utf-8', + timeout: 30_000, + maxBuffer: 100 * 1024 * 1024, + }).trim(); + +let _gitRunner: GitRunner = DEFAULT_GIT_RUNNER; + +/** + * Test seam (probe-seam pattern, matches `git-head.ts:_setGitHeadProbeForTests`) + * so tests drive `computeSyncDelta` without mocking child_process or routing + * through `mock.module` (R2-compliant). Pass `null` to restore the default. + */ +export function _setGitRunnerForTests(fn: GitRunner | null): void { + _gitRunner = fn ?? DEFAULT_GIT_RUNNER; +} + +function unique(items: T[]): T[] { + return [...new Set(items)]; +} + +/** + * Working-tree manifest for a DETACHED HEAD (relocated from sync.ts:557 so the + * estimator can price detached sources identically to how the executor imports + * them). On a detached HEAD, sync syncs from the live working tree: tracked + * changes (`git diff --name-status -M HEAD`) PLUS untracked files (`ls-files + * --others --exclude-standard`). Attached HEADs never call this — their + * incremental path imports ONLY the commit diff (untracked/dirty files are not + * imported), which is why the estimator must not price dirty files on an + * attached repo (issue #2139 phantom-cost class). + */ +export function buildDetachedWorkingTreeManifest( + repoPath: string, + run: GitRunner = _gitRunner, +): SyncManifest { + const manifest = buildSyncManifest(run(repoPath, ['diff', '--name-status', '-M', 'HEAD'])); + const untracked = run(repoPath, ['ls-files', '--others', '--exclude-standard']) + .split('\n') + .filter(line => line.length > 0); + return { + added: unique([...manifest.added, ...untracked]), + modified: unique(manifest.modified), + deleted: unique(manifest.deleted), + renamed: manifest.renamed, + }; +} + +export type SyncDeltaResult = + | { status: 'ok'; manifest: SyncManifest } + | { status: 'unavailable'; reason: 'anchor_missing' | 'diff_failed' }; + +export interface ComputeSyncDeltaOpts { + /** + * Pre-computed detached working-tree manifest to merge into the commit diff + * (the executor already builds one for its `up_to_date` gate; pass it to + * avoid a redundant `git diff HEAD` + `ls-files`). When omitted and + * `detached` is true, this module builds it. + */ + detachedManifest?: SyncManifest | null; + /** Build the detached manifest internally (estimator path). Ignored if `detachedManifest` is provided. */ + detached?: boolean; +} + +/** + * The single source of truth for "what changed between two commits in this + * repo." Returns the RAW merged manifest (added/modified/deleted/renamed) — + * callers apply their own `isSyncable` filtering + side effects. + */ +export function computeSyncDelta( + repoPath: string, + fromCommit: string, + toCommit: string, + opts: ComputeSyncDeltaOpts = {}, +): SyncDeltaResult { + const run = _gitRunner; + + // Reachability: a gc'd bookmark object can't be diffed (#1970). + try { + run(repoPath, ['cat-file', '-t', fromCommit]); + } catch { + return { status: 'unavailable', reason: 'anchor_missing' }; + } + + let diffOutput: string; + try { + diffOutput = run(repoPath, ['diff', '--name-status', '-M', `${fromCommit}..${toCommit}`]); + } catch { + return { status: 'unavailable', reason: 'diff_failed' }; + } + + const manifest = buildSyncManifest(diffOutput); + + const detached = + opts.detachedManifest !== undefined && opts.detachedManifest !== null + ? opts.detachedManifest + : opts.detached + ? buildDetachedWorkingTreeManifest(repoPath, run) + : null; + if (detached) { + manifest.added = unique([...manifest.added, ...detached.added]); + manifest.modified = unique([...manifest.modified, ...detached.modified]); + manifest.deleted = unique([...manifest.deleted, ...detached.deleted]); + manifest.renamed = [...manifest.renamed, ...detached.renamed]; + } + + return { status: 'ok', manifest }; +} diff --git a/test/config-set.test.ts b/test/config-set.test.ts index 9857baf46..8d353c838 100644 --- a/test/config-set.test.ts +++ b/test/config-set.test.ts @@ -29,6 +29,15 @@ describe('KNOWN_CONFIG_KEYS', () => { expect(KNOWN_CONFIG_KEYS).toContain('models.tier.subagent'); }); + test('contains the spend-control keys (v0.42.42.0, #2139) — no --force archaeology', () => { + expect(KNOWN_CONFIG_KEYS).toContain('spend.posture'); + expect(KNOWN_CONFIG_KEYS).toContain('sync.cost_gate_min_usd'); + expect(KNOWN_CONFIG_KEYS).toContain('sync.federated_v2'); + expect(KNOWN_CONFIG_KEYS).toContain('embed.backfill_cooldown_min'); + expect(KNOWN_CONFIG_KEYS).toContain('embed.backfill_max_usd_per_source_24h'); + expect(KNOWN_CONFIG_KEYS).toContain('embed.backfill_max_usd'); + }); + test('no duplicate entries', () => { const set = new Set(KNOWN_CONFIG_KEYS); expect(set.size).toBe(KNOWN_CONFIG_KEYS.length); diff --git a/test/embed-backfill-submit.test.ts b/test/embed-backfill-submit.test.ts index 9a19a0d30..3903ceb94 100644 --- a/test/embed-backfill-submit.test.ts +++ b/test/embed-backfill-submit.test.ts @@ -155,6 +155,54 @@ describe('submitEmbedBackfill — 24h spend cap', () => { expect(result.status).toBe('spend_capped'); expect(result.spendCapUsd).toBe(5); }); + + // v0.42.42.0 (#2139): off-switch + tokenmax bypass. + test('cap "off" → submits even at huge spend (Infinity cap never tripped)', async () => { + await engine.setConfig(SPEND_CAP_CONFIG_KEY, 'off'); + const result = await submitEmbedBackfill(engine, 'default', { + reason: 'unit', + spend24hFn: async () => 1e9, + }); + expect(result.status).toBe('submitted'); + }); + + test('0 falls back to the default cap (off semantics ≠ 0)', async () => { + await engine.setConfig(SPEND_CAP_CONFIG_KEY, '0'); + const result = await submitEmbedBackfill(engine, 'default', { + reason: 'unit', + spend24hFn: async () => 25, // == default $25 → capped + }); + expect(result.status).toBe('spend_capped'); + expect(result.spendCapUsd).toBe(25); + }); + + test('spend.posture=tokenmax bypasses the cap, marks spendCapBypassed', async () => { + const result = await submitEmbedBackfill(engine, 'default', { + reason: 'unit', + postureOverride: 'tokenmax', + spendCapUsdOverride: 25, + spend24hFn: async () => 100, // way over cap + }); + expect(result.status).toBe('submitted'); + expect(result.spendCapBypassed).toBe(true); + expect(result.spend24hUsd).toBe(100); + }); + + test('tokenmax does NOT bypass the cooldown (axis split — churn protection stays)', async () => { + const queue = new MinionQueue(engine); + const job = await queue.add('embed-backfill', { sourceId: 'default' }, {}); + await engine.executeRaw( + `UPDATE minion_jobs SET status='completed', finished_at=NOW() - INTERVAL '1 minute' WHERE id=$1`, + [job.id], + ); + const result = await submitEmbedBackfill(engine, 'default', { + reason: 'unit', + postureOverride: 'tokenmax', + cooldownMinOverride: 10, + spend24hFn: async () => 1e9, + }); + expect(result.status).toBe('cooldown'); // posture lifts the cap, NOT the cooldown + }); }); describe('submitEmbedBackfill — source isolation', () => { diff --git a/test/spend-off-switch.test.ts b/test/spend-off-switch.test.ts new file mode 100644 index 000000000..2eb40c1dd --- /dev/null +++ b/test/spend-off-switch.test.ts @@ -0,0 +1,53 @@ +/** + * v0.42.42.0 (#2139) — `--max-usd off` / `--max-cost off` uncapped-switch pins + * across enrich / onboard / reindex (the T6 secondary cost gates). + * + * The enrich arg parser carries the real logic (off → Infinity sentinel, mapped + * to "no BudgetTracker ceiling" in runEnrichCore), so it gets a direct unit test. + * reindex + onboard detect `off` inline at the CLI dispatch and proceed past the + * confirmation / missing-cap refusal; those are pinned as source-level regression + * guards (the codex pre-landing review found all three half-built — these keep + * them from silently regressing without standing up full CLI+gateway harnesses). + */ +import { describe, test, expect } from 'bun:test'; +import { parseArgs } from '../src/commands/enrich.ts'; + +describe('enrich parseArgs — --max-usd off → uncapped (Infinity sentinel)', () => { + test('off / unlimited / none (case-insensitive) → Infinity', () => { + for (const v of ['off', 'OFF', 'unlimited', 'none', 'None']) { + expect(parseArgs(['--max-usd', v]).maxCostUsd).toBe(Infinity); + } + // --max-cost-usd alias too. + expect(parseArgs(['--max-cost-usd', 'off']).maxCostUsd).toBe(Infinity); + }); + test('finite positive number passes through; absent → undefined; garbage → undefined', () => { + expect(parseArgs(['--max-usd', '5']).maxCostUsd).toBe(5); + expect(parseArgs([]).maxCostUsd).toBeUndefined(); + expect(parseArgs(['--max-usd', 'abc']).maxCostUsd).toBeUndefined(); + expect(parseArgs(['--max-usd', '0']).maxCostUsd).toBeUndefined(); // non-positive ignored + }); +}); + +describe('reindex / onboard off-switch dispatch (regression guards)', () => { + test('reindex-code: --max-cost off proceeds past the confirmation gate', async () => { + const src = await Bun.file(new URL('../src/commands/reindex-code.ts', import.meta.url)).text(); + // off sets maxCostOff and the gate proceeds when (tokenmax || maxCostOff). + expect(src).toMatch(/maxCostOff\s*=\s*true/); + expect(src).toMatch(/posture === 'tokenmax' \|\| maxCostOff/); + }); + + test('onboard: --max-usd off lifts the --auto missing-cap refusal', async () => { + const src = await Bun.file(new URL('../src/commands/onboard.ts', import.meta.url)).text(); + expect(src).toMatch(/maxUsdOff\s*=/); + // refusal skipped when (maxUsdOff || tokenmax). + expect(src).toMatch(/maxUsdOff \|\| tokenmax/); + }); + + test('enrich: uncapped path maps the Infinity sentinel to no BudgetTracker ceiling', async () => { + const src = await Bun.file(new URL('../src/commands/enrich.ts', import.meta.url)).text(); + // The sentinel must become `undefined` at the tracker (never raw Infinity, + // which serializes to null in audit rows). + expect(src).toMatch(/opts\.maxCostUsd === Infinity \? undefined/); + expect(src).toMatch(/uncapped \? Infinity : parsed\.maxCostUsd/); + }); +}); diff --git a/test/sync-cost-estimate.test.ts b/test/sync-cost-estimate.test.ts new file mode 100644 index 000000000..cc53947ce --- /dev/null +++ b/test/sync-cost-estimate.test.ts @@ -0,0 +1,140 @@ +/** + * v0.42.42.0 (#2139) — estimateInlineNewTokens ladder coverage. + * + * The inline cost estimator now MIRRORS EXECUTION (delta, not full-tree + * ceiling) so the gate's dollar figure stops being a ~400x phantom on a busy + * brain. Real temp git repos, no PGLite. estimateInlineNewTokens is exported + * from commands/sync.ts; CHUNKER_VERSION is the live chunker version. + */ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { mkdtempSync, writeFileSync, rmSync, mkdirSync } from 'fs'; +import { execSync } from 'child_process'; +import { tmpdir } from 'os'; +import { join } from 'path'; +import { estimateInlineNewTokens } from '../src/commands/sync.ts'; +import { CHUNKER_VERSION } from '../src/core/chunkers/code.ts'; + +const CURRENT = String(CHUNKER_VERSION); +let repo: string; + +function commitAll(msg: string): string { + execSync('git add -A', { cwd: repo, stdio: 'pipe' }); + execSync(`git commit -m "${msg}"`, { cwd: repo, stdio: 'pipe' }); + return execSync('git rev-parse HEAD', { cwd: repo, stdio: 'pipe' }).toString().trim(); +} +function src(over: Partial<{ last_commit: string | null; chunker_version: string | null; config: Record }> = {}) { + return { + local_path: repo, + config: over.config ?? {}, + last_commit: over.last_commit ?? null, + chunker_version: over.chunker_version ?? null, + }; +} + +beforeEach(() => { + repo = mkdtempSync(join(tmpdir(), 'gbrain-est-')); + execSync('git init', { cwd: repo, stdio: 'pipe' }); + execSync('git config user.email "t@t.com"', { cwd: repo, stdio: 'pipe' }); + execSync('git config user.name "T"', { cwd: repo, stdio: 'pipe' }); + mkdirSync(join(repo, 'topics'), { recursive: true }); +}); +afterEach(() => { + if (repo) rmSync(repo, { recursive: true, force: true }); +}); + +describe('estimateInlineNewTokens — ladder', () => { + test('chunker drift → full-tree ceiling even with an empty git delta', () => { + writeFileSync(join(repo, 'topics/a.md'), 'some body content here'); + const head = commitAll('base'); + // git unchanged (last_commit == HEAD) but chunker drifted → must NOT be 0. + const r = estimateInlineNewTokens([src({ last_commit: head, chunker_version: 'STALE-0' })], CURRENT); + expect(r.tokens).toBeGreaterThan(0); + expect(r.estimateKind).toBe('ceiling'); + expect(r.ceilingReasons).toContain('chunker_drift'); + expect(r.changedSources).toBe(1); + }); + + test('[D2A headline] HEAD==last_commit + current chunker + DIRTY tree → 0 (unchanged)', () => { + writeFileSync(join(repo, 'topics/a.md'), 'body'); + const head = commitAll('base'); + // Dirty the tree (untracked scratch + uncommitted edit) — attached sync + // imports nothing, so the estimate must be 0. This is the exact pre-fix + // false-fire shape. + writeFileSync(join(repo, 'scratch.tmp'), 'agent scratch'); + writeFileSync(join(repo, 'topics/a.md'), 'uncommitted edit'); + const r = estimateInlineNewTokens([src({ last_commit: head, chunker_version: CURRENT })], CURRENT); + expect(r.tokens).toBe(0); + expect(r.estimateKind).toBe('unchanged'); + expect(r.unchangedSources).toBe(1); + }); + + test('first sync (last_commit null) → full-tree ceiling', () => { + writeFileSync(join(repo, 'topics/a.md'), 'body content'); + commitAll('base'); + const r = estimateInlineNewTokens([src({ last_commit: null, chunker_version: CURRENT })], CURRENT); + expect(r.tokens).toBeGreaterThan(0); + expect(r.estimateKind).toBe('ceiling'); + expect(r.ceilingReasons).toContain('first_sync'); + }); + + test('delta rung: only changed committed files priced; deletes cost 0', () => { + writeFileSync(join(repo, 'topics/a.md'), 'a'.repeat(400)); + writeFileSync(join(repo, 'topics/b.md'), 'b'.repeat(400)); + const base = commitAll('base'); + writeFileSync(join(repo, 'topics/a.md'), 'a'.repeat(800)); // modify + rmSync(join(repo, 'topics/b.md')); // delete → 0 + commitAll('change'); + + const r = estimateInlineNewTokens([src({ last_commit: base, chunker_version: CURRENT })], CURRENT); + expect(r.estimateKind).toBe('delta'); + expect(r.tokens).toBeGreaterThan(0); // a.md priced + // A pure-delete delta would be 0; here a.md modify keeps it > 0. Sanity: + // the magnitude is delta-scale (one ~800-char file), not full-tree. + }); + + test('delta rung: non-syncable changed file is filtered out (markdown strategy)', () => { + writeFileSync(join(repo, 'topics/a.md'), 'md'); + const base = commitAll('base'); + writeFileSync(join(repo, 'notes.txt'), 'x'.repeat(4000)); // .txt under markdown strategy → not syncable + commitAll('add txt'); + + const r = estimateInlineNewTokens([src({ last_commit: base, chunker_version: CURRENT })], CURRENT); + // No syncable changes → delta priced at 0 tokens (but the source changed). + expect(r.tokens).toBe(0); + expect(r.estimateKind).toBe('delta'); + }); + + test('syncEnabled:false sources are skipped (neither changed nor unchanged)', () => { + writeFileSync(join(repo, 'topics/a.md'), 'body'); + commitAll('base'); + const r = estimateInlineNewTokens([src({ last_commit: null, config: { syncEnabled: false } })], CURRENT); + expect(r.tokens).toBe(0); + expect(r.changedSources).toBe(0); + expect(r.unchangedSources).toBe(0); + }); + + test('mixed: one ceiling source + one unchanged source → estimateKind mixed-or-ceiling, reasons captured', () => { + writeFileSync(join(repo, 'topics/a.md'), 'body'); + const head = commitAll('base'); + const r = estimateInlineNewTokens( + [ + src({ last_commit: null, chunker_version: CURRENT }), // first_sync ceiling + src({ last_commit: head, chunker_version: CURRENT }), // unchanged + ], + CURRENT, + ); + expect(r.ceilingReasons).toContain('first_sync'); + expect(r.unchangedSources).toBe(1); + // hadCeiling true, hadDelta false → 'ceiling' aggregate. + expect(r.estimateKind).toBe('ceiling'); + }); + + test('missing local_path source contributes nothing', () => { + const r = estimateInlineNewTokens( + [{ local_path: null, config: {}, last_commit: null, chunker_version: CURRENT }], + CURRENT, + ); + expect(r.tokens).toBe(0); + expect(r.changedSources).toBe(0); + }); +}); diff --git a/test/sync-cost-gate.serial.test.ts b/test/sync-cost-gate.serial.test.ts index 48d7b057d..1371fd01f 100644 --- a/test/sync-cost-gate.serial.test.ts +++ b/test/sync-cost-gate.serial.test.ts @@ -1,15 +1,17 @@ /** - * v0.41.31 — `gbrain sync --all` cost-gate wiring regressions (PGLite). + * `gbrain sync` cost-gate wiring regressions (PGLite). * - * Pure shouldBlockSync / willEmbedSynchronously logic is pinned in - * test/sync-cost-preview.test.ts. THIS file pins the end-to-end wiring in - * runSync's --all path: + * Pure shouldBlockSync / willEmbedSynchronously / parseUsdLimit logic is pinned + * in test/sync-cost-preview.test.ts. THIS file pins the end-to-end wiring in + * runSync's --all AND single-source paths: * * R-1 (headline): deferred-embed sync --all, non-TTY, with backlog → - * emits gate:'deferred_notice' and NEVER exit 2 (the cron-blocking - * bug this release fixes). - * R-2 (protection): inline-embed sync --all (--serial), non-TTY, above - * floor → still exit 2 with gate:'confirmation_required'. + * emits gate:'deferred_notice' and NEVER exit 2. + * R-2 (v0.42.42.0, #2139): inline sync --all (--serial), non-TTY, above floor + * → AUTO-DEFERS (exit 0, gate:'auto_deferred_embeds') and enqueues an + * embed-backfill job. The exit-2 wedge is gone. + * R-3: chunker drift → full-tree CEILING estimate, auto-defers (not exit 2). + * + posture tokenmax, off-switch, format split (#1784/D3A), single-source gate. * * Serial-quarantined: stubs process.exit + console.log (process-global). */ @@ -21,10 +23,18 @@ import { join } from 'path'; import { PGLiteEngine } from '../src/core/pglite-engine.ts'; import { runSources } from '../src/commands/sources.ts'; import { resetPgliteState } from './helpers/reset-pglite.ts'; -import { configureGateway, resetGateway } from '../src/core/ai/gateway.ts'; +import { configureGateway, resetGateway, __setEmbedTransportForTests } from '../src/core/ai/gateway.ts'; import { CHUNKER_VERSION } from '../src/core/chunkers/code.ts'; import type { ChunkInput } from '../src/core/types.ts'; +/** Offline embed stub so inline-proceed paths (posture tokenmax) don't network. */ +function stubOfflineEmbed(): void { + __setEmbedTransportForTests(async ({ values }: any) => ({ + embeddings: values.map(() => new Array(1536).fill(0)), + usage: { tokens: 0 }, + }) as any); +} + let engine: PGLiteEngine; let repoPath: string; let headSha: string; @@ -64,6 +74,7 @@ beforeEach(async () => { }); afterEach(() => { + __setEmbedTransportForTests(null); resetGateway(); if (repoPath) rmSync(repoPath, { recursive: true, force: true }); }); @@ -115,39 +126,43 @@ describe('v0.41.31 — sync --all cost gate wiring', () => { expect(stdout).toContain('"gate":"deferred_notice"'); }, 60_000); - test('R-2: inline sync --all (--serial) above floor still exit 2', async () => { + test('R-2 (#2139): inline sync --all (--serial) above floor AUTO-DEFERS (exit 0, never exit 2) + enqueues backfill', async () => { await runSources(engine, ['add', 'vault', '--path', repoPath, '--no-federated']); - // Floor 0 → any nonzero inline cost blocks. Source is unsynced - // (last_commit NULL) so estimateInlineNewTokens sees it as changed → - // full-tree tokens > 0 → costUsd > 0 > floor. + // Floor 0 → any nonzero inline cost trips the gate. Source is unsynced + // (last_commit NULL) → first-sync ceiling > 0 > floor. await engine.setConfig('sync.cost_gate_min_usd', '0'); - // --serial forces inline even with v2 on. --json → non-TTY exit-2 path. + // --serial forces inline even with v2 on. --json → non-TTY path → AUTO-DEFER. const { exitCode, stdout } = await runSyncCaptured(['--all', '--serial', '--json', '--no-pull']); - expect(exitCode).toBe(2); - expect(stdout).toContain('"gate":"confirmation_required"'); + expect(exitCode).not.toBe(2); + expect(stdout).toContain('"gate":"auto_deferred_embeds"'); + expect(stdout).not.toContain('"gate":"confirmation_required"'); + // The run PROCEEDED to import (the wedge is gone) — embeds were deferred, + // not blocked. (The embed-backfill enqueue wiring + its graceful + // missing-table tolerance is pinned in embed-backfill-submit.test.ts; the + // minion_jobs table isn't provisioned in this gate-wiring harness.) + expect(stdout).toContain('"sync_status":"first_sync"'); }, 60_000); - test('R-3: inline, git-unchanged source but STALE chunker_version still estimates (not $0)', async () => { - // The unchanged-source short-circuit requires HEAD==last_commit AND clean - // tree AND chunker_version == current. Here git is unchanged but the - // chunker drifted, so the source must NOT be treated as 0 — sync would - // re-chunk + re-embed everything. floor=0 so any nonzero cost blocks. + test('R-3 (#2139): chunker drift → full-tree CEILING estimate, auto-defers (not exit 2)', async () => { + // git unchanged (HEAD==last_commit) but chunker drifted → the source must + // NOT price $0 (sync would re-chunk + re-embed everything). The estimate is + // the full-tree ceiling; the gate auto-defers rather than wedging. await runSources(engine, ['add', 'vault', '--path', repoPath, '--no-federated']); await engine.executeRaw(`UPDATE sources SET last_commit = $1, chunker_version = $2 WHERE id = 'vault'`, [headSha, 'STALE-0']); await engine.setConfig('sync.cost_gate_min_usd', '0'); const { exitCode, stdout } = await runSyncCaptured(['--all', '--serial', '--json', '--no-pull']); - expect(exitCode).toBe(2); - expect(stdout).toContain('"gate":"confirmation_required"'); + expect(exitCode).not.toBe(2); + expect(stdout).toContain('"gate":"auto_deferred_embeds"'); + expect(stdout).toContain('"estimateKind":"ceiling"'); }, 60_000); - test('R-3 control: inline, git-unchanged + CURRENT chunker_version short-circuits to $0 (no exit 2)', async () => { - // Same setup but chunker_version matches current → the source IS unchanged - // → contributes 0 new-content tokens → below floor → proceeds (no block). - // Proves the short-circuit fires when (and only when) everything matches. + test('R-3 control: git-unchanged + CURRENT chunker → $0 estimate, below floor (no auto-defer)', async () => { + // Mirrors the executor's up_to_date predicate: HEAD==last_commit AND chunker + // matches → 0 new tokens → below floor → proceeds without deferring. await runSources(engine, ['add', 'vault', '--path', repoPath, '--no-federated']); await engine.executeRaw(`UPDATE sources SET last_commit = $1, chunker_version = $2 WHERE id = 'vault'`, [headSha, String(CHUNKER_VERSION)]); await engine.setConfig('sync.cost_gate_min_usd', '0'); @@ -155,6 +170,78 @@ describe('v0.41.31 — sync --all cost gate wiring', () => { const { exitCode, stdout } = await runSyncCaptured(['--all', '--serial', '--json', '--no-pull']); expect(exitCode).not.toBe(2); - expect(stdout).not.toContain('"gate":"confirmation_required"'); + expect(stdout).not.toContain('"gate":"auto_deferred_embeds"'); + expect(stdout).toContain('"estimateKind":"unchanged"'); + }, 60_000); + + test('headline regression: HEAD==last_commit + DIRTY untracked file → $0, no gate (the false-fire)', async () => { + // The exact pre-fix false-fire: a busy brain's working tree is never + // git-clean, but the commits are caught up. The OLD estimator priced the + // whole tree (158M-token phantom); the new one mirrors execution → $0. + await runSources(engine, ['add', 'vault', '--path', repoPath, '--no-federated']); + await engine.executeRaw(`UPDATE sources SET last_commit = $1, chunker_version = $2 WHERE id = 'vault'`, [headSha, String(CHUNKER_VERSION)]); + // Dirty the tree with an untracked non-syncable scratch file (agents/crons + // write constantly) — attached-HEAD sync never imports it. + writeFileSync(join(repoPath, 'scratch.tmp'), 'uncommitted agent scratch'); + writeFileSync(join(repoPath, 'topics/foo.md'), 'uncommitted edit, not staged'); + await engine.setConfig('sync.cost_gate_min_usd', '0'); + + const { exitCode, stdout } = await runSyncCaptured(['--all', '--serial', '--json', '--no-pull']); + + expect(exitCode).not.toBe(2); + expect(stdout).not.toContain('"gate":"auto_deferred_embeds"'); + expect(stdout).toContain('"estimateKind":"unchanged"'); + }, 60_000); + + test('spend.posture=tokenmax → proceeds inline, gate:posture_tokenmax (informational)', async () => { + stubOfflineEmbed(); // inline embed proceeds — keep it off the network. + await runSources(engine, ['add', 'vault', '--path', repoPath, '--no-federated']); + await engine.setConfig('sync.cost_gate_min_usd', '0'); + await engine.setConfig('spend.posture', 'tokenmax'); + + const { exitCode, stdout } = await runSyncCaptured(['--all', '--serial', '--json', '--no-pull']); + + expect(exitCode).not.toBe(2); + expect(stdout).toContain('"gate":"posture_tokenmax"'); + expect(stdout).not.toContain('"gate":"auto_deferred_embeds"'); + }, 60_000); + + test('sync.cost_gate_min_usd=off → floor renders "unlimited", never blocks', async () => { + stubOfflineEmbed(); + await runSources(engine, ['add', 'vault', '--path', repoPath, '--no-federated']); + await engine.setConfig('sync.cost_gate_min_usd', 'off'); + + const { exitCode, stdout } = await runSyncCaptured(['--all', '--serial', '--json', '--no-pull']); + + expect(exitCode).not.toBe(2); + expect(stdout).toContain('"floorUsd":"unlimited"'); + expect(stdout).not.toContain('"gate":"auto_deferred_embeds"'); + }, 60_000); + + test('format split (#1784/D3A): non-TTY WITHOUT --json emits human text, no JSON envelope', async () => { + await runSources(engine, ['add', 'vault', '--path', repoPath, '--no-federated']); + await engine.setConfig('sync.cost_gate_min_usd', '0'); + + // No --json: above floor in a non-TTY session → human auto-defer text. + const { exitCode, stdout } = await runSyncCaptured(['--all', '--serial', '--no-pull']); + + expect(exitCode).not.toBe(2); + expect(stdout).not.toContain('"gate":'); // no JSON envelope without --json + expect(stdout.toLowerCase()).toContain('deferring embeds'); + expect(stdout).toContain('spend.posture'); // self-describing hint present + }, 60_000); + + test('single-source sync gets the same gate (auto-defers above floor, exit 0)', async () => { + await runSources(engine, ['add', 'vault', '--path', repoPath, '--no-federated']); + await engine.setConfig('sync.cost_gate_min_usd', '0'); + + // Single-source (no --all): unsynced → ceiling > 0 → non-TTY auto-defer. + const { exitCode, stdout } = await runSyncCaptured(['--source', 'vault', '--json', '--no-pull']); + + expect(exitCode).not.toBe(2); + expect(stdout).toContain('"gate":"auto_deferred_embeds"'); + // The gate now exists on the single-source path (was ungated before + // #2139) and proceeds to import rather than blocking. + expect(stdout.toLowerCase()).toContain('imported'); }, 60_000); }); diff --git a/test/sync-cost-preview.test.ts b/test/sync-cost-preview.test.ts index 52d4dba6b..79ef708dc 100644 --- a/test/sync-cost-preview.test.ts +++ b/test/sync-cost-preview.test.ts @@ -23,6 +23,13 @@ import { import { lookupEmbeddingPrice } from '../src/core/embedding-pricing.ts'; import { resetGateway } from '../src/core/ai/gateway.ts'; import { estimateTokens } from '../src/core/chunkers/code.ts'; +import { + parseUsdLimit, + formatUsdLimit, + usdLimitToCap, + normalizeSpendPosture, + isValidSpendPosture, +} from '../src/core/spend-posture.ts'; describe('Layer 8 D1 — embedding cost model', () => { // The estimateEmbeddingCostUsd tests assert the gateway-UNCONFIGURED OpenAI @@ -127,6 +134,73 @@ describe('v0.41.31 — shouldBlockSync (cost-gate decision)', () => { expect(shouldBlockSync(0.0001, 0, 'inline')).toBe(true); expect(shouldBlockSync(0, 0, 'inline')).toBe(false); }); + + // v0.42.42.0 (#2139): posture + Infinity-floor behavior. + test('spend.posture=tokenmax never blocks, even above floor', () => { + expect(shouldBlockSync(999, 0.5, 'inline', 'tokenmax')).toBe(false); + expect(shouldBlockSync(999, 0, 'inline', 'tokenmax')).toBe(false); + }); + test('default posture (gated) preserves the legacy decision', () => { + expect(shouldBlockSync(0.51, 0.5, 'inline', 'gated')).toBe(true); + expect(shouldBlockSync(0.03, 0.5, 'inline', 'gated')).toBe(false); + }); + test('off/unlimited floor (Infinity) is never exceeded → never blocks', () => { + expect(shouldBlockSync(999, Infinity, 'inline')).toBe(false); + expect(shouldBlockSync(1e9, Infinity, 'inline', 'gated')).toBe(false); + }); +}); + +describe('v0.42.42.0 (#2139) — spend-posture USD-limit parsing', () => { + test('off / unlimited / none (case-insensitive) → Infinity', () => { + for (const v of ['off', 'OFF', 'unlimited', 'Unlimited', 'none', 'NONE', ' off ']) { + expect(parseUsdLimit(v, 25)).toBe(Infinity); + } + }); + test('finite positive numbers pass through', () => { + expect(parseUsdLimit('5', 25)).toBe(5); + expect(parseUsdLimit(0.5, 25)).toBe(0.5); + expect(parseUsdLimit('100000', 25)).toBe(100000); + }); + test('0 falls back to default unless allowZero', () => { + expect(parseUsdLimit('0', 25)).toBe(25); // backfill cap: off ≠ 0 + expect(parseUsdLimit('0', 0.5, { allowZero: true })).toBe(0); // floor: 0 = block-on-any + }); + test('garbage / negative / empty / null → default', () => { + expect(parseUsdLimit('abc', 25)).toBe(25); + expect(parseUsdLimit('-3', 25)).toBe(25); + expect(parseUsdLimit('', 25)).toBe(25); + expect(parseUsdLimit(null, 25)).toBe(25); + expect(parseUsdLimit(undefined, 0.5)).toBe(0.5); + }); + test('formatUsdLimit: Infinity → "unlimited" (never the JSON.stringify=null trap), finite passthrough', () => { + expect(formatUsdLimit(Infinity)).toBe('unlimited'); + expect(formatUsdLimit(5)).toBe(5); + expect(formatUsdLimit(0)).toBe(0); + // The trap this guards: raw Infinity serializes to null. + expect(JSON.stringify({ cap: Infinity })).toBe('{"cap":null}'); + expect(JSON.stringify({ cap: formatUsdLimit(Infinity) })).toBe('{"cap":"unlimited"}'); + }); + test('usdLimitToCap: Infinity → undefined (no cap), finite passthrough', () => { + expect(usdLimitToCap(Infinity)).toBeUndefined(); + expect(usdLimitToCap(10)).toBe(10); + }); + test('normalizeSpendPosture: only tokenmax is tokenmax; everything else gated', () => { + expect(normalizeSpendPosture('tokenmax')).toBe('tokenmax'); + expect(normalizeSpendPosture('TokenMax')).toBe('tokenmax'); + expect(normalizeSpendPosture('gated')).toBe('gated'); + expect(normalizeSpendPosture('max')).toBe('gated'); + expect(normalizeSpendPosture('')).toBe('gated'); + expect(normalizeSpendPosture(null)).toBe('gated'); + expect(normalizeSpendPosture(42)).toBe('gated'); + }); + test('isValidSpendPosture accepts gated/tokenmax (case-insensitive), rejects the rest', () => { + expect(isValidSpendPosture('gated')).toBe(true); + expect(isValidSpendPosture('tokenmax')).toBe(true); + expect(isValidSpendPosture('TokenMax')).toBe(true); // normalized lowercase + expect(isValidSpendPosture('max')).toBe(false); + expect(isValidSpendPosture('')).toBe(false); + expect(isValidSpendPosture(7)).toBe(false); + }); }); describe('Layer 8 D1 — estimateTokens (exported from chunkers/code.ts)', () => { diff --git a/test/sync-delta.test.ts b/test/sync-delta.test.ts new file mode 100644 index 000000000..20f1ffff8 --- /dev/null +++ b/test/sync-delta.test.ts @@ -0,0 +1,154 @@ +/** + * v0.42.42.0 (#2139) — computeSyncDelta unit coverage. + * + * The shared diff/manifest helper that BOTH the sync executor and the inline + * cost estimator route through (so the gate's dollar figure can't drift from + * what the sync imports). Real temp git repos; no PGLite, no env writes + * (R1/R2-clean). The git-runner seam drives the unavailable branches. + */ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { mkdtempSync, writeFileSync, rmSync, mkdirSync, renameSync } from 'fs'; +import { execSync } from 'child_process'; +import { tmpdir } from 'os'; +import { join } from 'path'; +import { + computeSyncDelta, + buildDetachedWorkingTreeManifest, + _setGitRunnerForTests, +} from '../src/core/sync-delta.ts'; + +let repo: string; + +function git(args: string): string { + return execSync(`git ${args}`, { cwd: repo, stdio: 'pipe' }).toString().trim(); +} +function commitAll(msg: string): string { + execSync('git add -A', { cwd: repo, stdio: 'pipe' }); + execSync(`git commit -m "${msg}"`, { cwd: repo, stdio: 'pipe' }); + return git('rev-parse HEAD'); +} + +beforeEach(() => { + repo = mkdtempSync(join(tmpdir(), 'gbrain-delta-')); + execSync('git init', { cwd: repo, stdio: 'pipe' }); + execSync('git config user.email "t@t.com"', { cwd: repo, stdio: 'pipe' }); + execSync('git config user.name "T"', { cwd: repo, stdio: 'pipe' }); + mkdirSync(join(repo, 'topics'), { recursive: true }); +}); + +afterEach(() => { + _setGitRunnerForTests(null); + if (repo) rmSync(repo, { recursive: true, force: true }); +}); + +describe('computeSyncDelta — commit diff', () => { + test('A/M/D classified; only committed changes in the manifest', () => { + writeFileSync(join(repo, 'topics/a.md'), 'a'); + writeFileSync(join(repo, 'topics/b.md'), 'b'); + const base = commitAll('base'); + writeFileSync(join(repo, 'topics/a.md'), 'a-edited'); // modify + writeFileSync(join(repo, 'topics/c.md'), 'c'); // add + rmSync(join(repo, 'topics/b.md')); // delete + const head = commitAll('change'); + + const r = computeSyncDelta(repo, base, head); + expect(r.status).toBe('ok'); + if (r.status !== 'ok') return; + expect(r.manifest.modified).toContain('topics/a.md'); + expect(r.manifest.added).toContain('topics/c.md'); + expect(r.manifest.deleted).toContain('topics/b.md'); + }); + + test('rename → destination path on the renamed list', () => { + writeFileSync(join(repo, 'topics/old.md'), 'x'.repeat(200)); + const base = commitAll('base'); + renameSync(join(repo, 'topics/old.md'), join(repo, 'topics/new.md')); + const head = commitAll('rename'); + + const r = computeSyncDelta(repo, base, head); + expect(r.status).toBe('ok'); + if (r.status !== 'ok') return; + expect(r.manifest.renamed.map(x => x.to)).toContain('topics/new.md'); + }); + + test('[D2A] attached HEAD: dirty tracked + untracked files are NOT in the manifest', () => { + writeFileSync(join(repo, 'topics/a.md'), 'a'); + const base = commitAll('base'); + const head = git('rev-parse HEAD'); // HEAD == base, no new commits + // Dirty the tree: an uncommitted edit + an untracked scratch file. + writeFileSync(join(repo, 'topics/a.md'), 'uncommitted edit'); + writeFileSync(join(repo, 'scratch.tmp'), 'untracked'); + + const r = computeSyncDelta(repo, base, head); // not detached → commit diff only + expect(r.status).toBe('ok'); + if (r.status !== 'ok') return; + expect(r.manifest.added).toHaveLength(0); + expect(r.manifest.modified).toHaveLength(0); + expect(r.manifest.deleted).toHaveLength(0); + }); +}); + +describe('computeSyncDelta — detached HEAD merges the working-tree manifest', () => { + test('detached + working-tree changes → merged into the manifest', () => { + writeFileSync(join(repo, 'topics/a.md'), 'a'); + const base = commitAll('base'); + // Detach HEAD and dirty the tree. + execSync(`git checkout --detach ${base}`, { cwd: repo, stdio: 'pipe' }); + writeFileSync(join(repo, 'topics/a.md'), 'detached edit'); // tracked modify + writeFileSync(join(repo, 'topics/new.md'), 'new'); // untracked add + + const r = computeSyncDelta(repo, base, base, { detached: true }); + expect(r.status).toBe('ok'); + if (r.status !== 'ok') return; + expect(r.manifest.modified).toContain('topics/a.md'); + expect(r.manifest.added).toContain('topics/new.md'); // untracked picked up on detached + }); + + test('buildDetachedWorkingTreeManifest: clean detached tree → empty manifest', () => { + writeFileSync(join(repo, 'topics/a.md'), 'a'); + const base = commitAll('base'); + execSync(`git checkout --detach ${base}`, { cwd: repo, stdio: 'pipe' }); + const m = buildDetachedWorkingTreeManifest(repo); + expect(m.added).toHaveLength(0); + expect(m.modified).toHaveLength(0); + }); +}); + +describe('computeSyncDelta — fail-open ladder', () => { + test('bogus anchor SHA → unavailable: anchor_missing', () => { + writeFileSync(join(repo, 'topics/a.md'), 'a'); + const head = commitAll('base'); + const r = computeSyncDelta(repo, 'deadbeefdeadbeefdeadbeefdeadbeefdeadbeef', head); + expect(r.status).toBe('unavailable'); + if (r.status === 'unavailable') expect(r.reason).toBe('anchor_missing'); + }); + + test('non-ancestor anchor still diffs (the #1970 property)', () => { + // git diff A..B is endpoint-tree, no ancestry requirement. + writeFileSync(join(repo, 'topics/a.md'), 'a'); + const base = commitAll('base'); + // Rewrite history: amend creates a new commit not descended from `base`, + // but `base` is still on disk (reflog) → diffable. + writeFileSync(join(repo, 'topics/a.md'), 'rewritten'); + execSync('git add -A && git commit --amend -m rewritten', { cwd: repo, stdio: 'pipe' }); + const head = git('rev-parse HEAD'); + expect(head).not.toBe(base); + + const r = computeSyncDelta(repo, base, head); + expect(r.status).toBe('ok'); // orphaned-but-present anchor is still diffable + }); + + test('injected git failure on the diff → unavailable: diff_failed', () => { + writeFileSync(join(repo, 'topics/a.md'), 'a'); + const base = commitAll('base'); + const head = git('rev-parse HEAD'); + _setGitRunnerForTests((_repo, args) => { + if (args[0] === 'cat-file') return 'commit'; // anchor reachable + if (args[0] === 'diff') throw new Error('simulated oversized diff / timeout'); + return ''; + }); + const r = computeSyncDelta(repo, base, head); + expect(r.status).toBe('unavailable'); + if (r.status === 'unavailable') expect(r.reason).toBe('diff_failed'); + }); +}); diff --git a/test/sync-failures.test.ts b/test/sync-failures.test.ts index 63ccb6a29..b1b762fc8 100644 --- a/test/sync-failures.test.ts +++ b/test/sync-failures.test.ts @@ -170,10 +170,10 @@ describe('Bug 9 — sync.ts CLI flag wiring', () => { // performSync's inner ack path only fires when failedFiles.length > 0 // in the current run. This test pins the up-front ack at the top of // runSync so the flag means "ack whatever is currently flagged". + // v0.42.42.0 (#2139, D13C): the pre-ack is now scoped PER SOURCE — `--all` + // acks every source, single-source acks only its own. const source = await Bun.file(new URL('../src/commands/sync.ts', import.meta.url)).text(); - // Ensure the up-front check exists before the syncAll / performSync - // dispatch, gated on skipFailed. - expect(source).toMatch(/if \(skipFailed\) \{[\s\S]*?unacknowledgedSyncFailures\(\)[\s\S]*?acknowledgeSyncFailures\(\)/); + expect(source).toMatch(/if \(skipFailed\) \{[\s\S]*?syncAll \? acknowledgeFailures\(\) : acknowledgeFailures\(sourceId\)/); }); test('acknowledgeSyncFailures clears stale failures end-to-end', async () => {