diff --git a/CHANGELOG.md b/CHANGELOG.md index 8b2b3cf30..25af01a5f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,127 @@ All notable changes to GBrain will be documented in this file. +## [0.41.4.0] - 2026-05-24 + +**Run your embedder, your reranker, and your dream judge on your own hardware. Windows users can pipe into `gbrain` like everyone else.** Three community PRs land together in one wave that takes gbrain's "local AI" story from "kinda" to "first-class." If you point gbrain at a llama.cpp embedding server, an Ollama box, or a self-hosted llama.cpp running Qwen3-Reranker, the doctor / autopilot / budget / models surfaces all treat them as real providers. The dream-cycle significance judge stops being hardcoded to Anthropic — point it at DeepSeek or any other registered chat provider via one config line. And `gbrain put my-slug` finally reads stdin correctly on Windows. + +Everything in this wave came from the community. Closed PRs #1325, #1326, #1329, #1349, #1365, #1366 cherry-picked or reworked here. + +### How to turn it on + +Nothing to flip. The local-provider path activates the moment you set a local provider as your embedding/reranker model: + +```bash +# Local embedding via llama.cpp: +gbrain config set embedding_model llama-server:nomic-embed-text + +# Local reranker via llama.cpp in --reranking mode (separate port from embeddings): +llama-server --model qwen3-reranker-4b-q4_k_m.gguf --alias qwen3-reranker-4b --reranking --port 8081 +gbrain config set provider_base_urls.llama-server-reranker http://localhost:8081/v1 +gbrain config set search.reranker.model llama-server-reranker:qwen3-reranker-4b +gbrain config set search.reranker.enabled true + +# Dream judge via DeepSeek (was Anthropic-only): +gbrain config set models.dream.synthesize_verdict deepseek:deepseek-chat +# DEEPSEEK_API_KEY env var is picked up automatically by the deepseek recipe +``` + +`gbrain models doctor` now probes embedding reachability the same way it probes reranker. Fresh installs see the local-providers-are-priced-at-$0 contract automatically. + +### What you'd see in a concrete example + +| Scenario | Before | After | +|---|---|---| +| `gbrain doctor --remediation-plan` on an Ollama embedding brain | "blocked: missing embedding API key" (incorrect) | no false block — local provider recognized | +| `gbrain reindex --max-cost 0.01` on a local embed/rerank | `BudgetExhausted(no_pricing)` hard-fail | runs (priced at $0 — electricity, not tokens) | +| `gbrain models doctor` with local llama-server-embeddings | only config probe ran; dead server invisible until first embed | reachability probe fires; dead server flagged immediately | +| `gbrain models doctor` with local llama-server-reranker | probe used recipe's 30s default | now matches `search.reranker.timeout_ms` if you set it (probe lies were possible either direction) | +| `gbrain dream --phase synthesize` on a DeepSeek-configured brain | "no ANTHROPIC_API_KEY for significance judge" — phase no-op | routes through gateway → DeepSeek → emits verdicts | +| `echo content \| gbrain put slug` on Windows | `ENOENT: /dev/stdin` | reads stdin via fd 0 (canonical cross-platform pattern) | +| Reading the PR queue with a long recipe name (`llama-server-reranker`, 21 chars) | "PROVIDER" column overflowed → row started with no space delimiter; downstream parser broke | dynamic-width column accommodates any recipe id | + +### What's safe to know about + +- **Voyage/Google brains relying on `gbrain config set voyage_api_key` (no env var):** `gbrain doctor --remediation-plan` is now strict about which provider's key it checks. If you set the key via `gbrain config set` (not env), the remediation planner sees it as not-yet-configured because those config keys aren't threaded to the gateway yet. Workaround: set the env var (`VOYAGE_API_KEY=...`, `GOOGLE_GENERATIVE_AI_API_KEY=...`). This is honest about a real gap — the previous behavior silently passed because of a wide-net fallback that accepted any OpenAI/ZE key. +- **Dream synthesize verdict model change:** if you had `dream.synthesize.verdict_model` set (deprecated key), it'll keep working via the legacy fallback. The canonical key is now `models.dream.synthesize_verdict`. +- **The new CI guard `scripts/check-gateway-routed-no-direct-anthropic.sh`** prevents `synthesize.ts` and `think/index.ts` from regressing back to `new Anthropic()`. Type-only imports stay allowed (the adapter needs `Anthropic.Message` as a type). + +### Itemized changes + +**Local providers as first-class (was: hosted-only assumed everywhere):** + +- New `embeddingProviderConfigured(embeddingModel, resolveKey)` helper in `src/core/brain-score-recommendations.ts` reads the recipe registry: empty `auth_env.required` (ollama, llama-server) returns true with no key; hosted providers check their OWN required key. Replaces the prefix ladder in `doctor.ts` AND the parallel copy in `autopilot.ts`. Exported `HOSTED_EMBED_KEY_CONFIG` map (env-var → config field) lets both producers build the same closure once. +- `RecommendationContext.hasEmbeddingApiKey` renamed to `embeddingProviderConfigured` (the field never meant "API key"). Blocker reason broadened from "missing embedding API key" to "embedding provider not configured" — covers both missing hosted key AND missing local recipe. +- `FREE_LOCAL_EMBED_PROVIDERS = {ollama, llama-server}` joins existing `FREE_LOCAL_RERANK_PROVIDERS` in `src/core/budget/budget-tracker.ts`. `--max-cost`-bounded embed/reindex jobs no longer hard-fail TX2 on local providers. `lmstudio` deliberately excluded (no recipe); `litellm` excluded (proxy can front paid). +- New `probeEmbeddingReachability()` in `src/commands/models.ts` mirrors `probeRerankerReachability` — a 1-input embed with 5s abort timeout, new `embedding_reachability` touchpoint, gated on the zero-network config probe returning `ok` first. + +**Local reranker via llama.cpp (was: ZeroEntropy-hosted only):** + +- New recipe `llama-server-reranker` at `src/core/ai/recipes/llama-server-reranker.ts`. Distinct from `llama-server` (embedding) because llama.cpp's `--reranking` and `--embeddings` flags are mutually exclusive at launch — one process per mode, two recipes, two base URLs. Default port 8081 vs 8080. +- `RerankerTouchpoint.path?: string` + `default_timeout_ms?: number` in `src/core/ai/types.ts`. Absent on ZE's recipe → behavior there unchanged. New recipe declares `path: '/rerank'` (concat with `/v1` base URL → `…/v1/rerank`) and `default_timeout_ms: 30000` (CPU-cold-start headroom). +- `src/core/search/mode.ts` reranker-timeout precedence: per-call > config > recipe `default_timeout_ms` > mode bundle. Closes the dead-default-timeout class. +- `src/cli.ts` env passthrough: `LLAMA_SERVER_RERANKER_BASE_URL` → `provider_base_urls.llama-server-reranker`. Sibling of `LLAMA_SERVER_BASE_URL`. +- `resolveLiveRerankerModel(engine)` + `resolveLiveRerankerTimeoutMs(engine)` in `src/commands/models.ts` so probe and live search read the SAME config (closes file-plane / DB-plane divergence). Probe's `getRerankerModel()` was reading `GBrainConfig.reranker_model` — a file-plane field nothing writes; live search read the DB plane via `loadSearchModeConfig`. + +**Dream cycle significance judge — gateway-routed (was: hardcoded `new Anthropic()`):** + +- `src/core/cycle/synthesize.ts` `makeJudgeClient(verdictModel)` replaces `makeHaikuClient()`. Mirrors `tryBuildGatewayClient` in `src/core/think/index.ts:579-637` (the v0.35.5.0 #952 pattern). Construction-time probe returns `null` on missing key OR unknown provider; the verdict loop wraps the gateway.chat call in try/catch for AIConfigError so mid-run provider failures surface as per-transcript `worth=false, reasons=['gateway error: ...']` instead of crashing the phase. +- Strict gateway migration — NO env-var sniffing. Canonical config key `models.dream.synthesize_verdict` (per `PER_TASK_KEYS` in `src/core/model-config.ts`). Deprecated `dream.synthesize.verdict_model` still resolves via the v0.28 model-resolution chain for legacy configs. +- `JudgeClient` interface signature preserved verbatim for test-seam stability. + +**Cross-platform stdin:** + +- `src/cli.ts:511` swaps `readFileSync('/dev/stdin', 'utf-8')` → `readFileSync(0, 'utf-8')`. Canonical Node cross-platform stdin idiom — works on Unix and Windows alike. PR #1366's 16-line try/catch fallback was a wronger fix to the same problem; the one-line change is what shipped. + +**Doc improvement (the value PR #1365 was reaching for):** + +- `src/core/operations.ts` `put_page` description extended with: "For large content on Windows (pipe-buffer limit ~45KB) or any file-as-input workflow, use `gbrain capture --file PATH --slug SLUG` — capture reads the file as a Buffer with a binary-NUL guard and adds provenance write-through (v0.39.3.0)." Surfaces via `gbrain put --help`. + +**CI guard (locks the bug class shut):** + +- New `scripts/check-gateway-routed-no-direct-anthropic.sh` greps `src/core/cycle/synthesize.ts` and `src/core/think/index.ts` for direct `new Anthropic()` constructor calls AND for value-shaped imports of `@anthropic-ai/sdk`. Type-only imports stay allowed. Clause-level parsing handles every TypeScript import shape (default, named, namespace, mixed-type-value, dynamic). Wired into `bun run verify` and `bun run check:all`. + +**Tests + regressions:** + +- `test/cycle/synthesize-gateway-adapter.test.ts` (NEW, 11 cases): A1-A9 unit + R3 parsed-verdict semantic parity + R3 corollary (unparseable-output fallback). +- `test/cycle/regression-pr-wave-r1-r2-r4.test.ts` (NEW): R1 (`get_page` accepts calls without `content`), R2 (`put_page` `content` stays `required: true`), R4 (cross-platform stdin behavior + source-grep). The closed PR #1365 would have broken R1 and R2; pinned forever. +- `test/e2e/dream-synthesize-pglite.test.ts` extended: new "gateway-adapter mid-run AIConfigError catch" case + reason-text update + `withoutAnthropicKey` hardened to override `GBRAIN_HOME` (closes a hermeticity hole the gateway rework opened). +- `src/commands/providers.ts` `formatRecipeTable` column width is now dynamic (`max(14, longest_id + 1)`) so `llama-server-reranker` (21 chars) doesn't overflow the historical 14-char column. The existing "each recipe appears at most once" test now passes naturally. + +**Process:** + +- 107 + 64 + 11 + 7 + 5 new test cases land across the wave. Final wave-affected count: 263 cases across 15 files, all green. +- `llms.txt` + `llms-full.txt` regenerated at end of wave via `bun run build:llms` (matched against the CHANGELOG voice rules). +- CLAUDE.md annotated with the gateway-adapter rework + CI guard so future contributors find the contract via the canonical doc. + +**Contributors:** PR #1325 by @tobbecokta, #1326 + #1329 by @kohai-ut, #1349 by @justemu, #1365 + #1366 by @ecat2010. Co-authored credits preserved in commit history. + +## To take advantage of v0.41.4.0 + +`gbrain upgrade` should do this automatically. No schema migration ships in this release. + +1. **If you run local embeddings (ollama, llama-server):** + ```bash + gbrain doctor --remediation-plan --json | jq '[.. | objects | select(.reason? == "embedding provider not configured")]' + ``` + Expect `[]` for a configured local-embeddings brain. If you see a block, your model string doesn't match a registered recipe — `gbrain models` lists the active resolution. + +2. **If you run the dream cycle on a non-Anthropic provider:** + ```bash + gbrain config set models.dream.synthesize_verdict deepseek:deepseek-chat + gbrain dream --phase synthesize --dry-run --json | jq '.details.verdicts' + ``` + Pre-rework all verdicts would show `worth: false, reasons: ['no ANTHROPIC_API_KEY ...']`. Post-rework you'll see real Haiku-shape verdicts from your configured provider. + +3. **If you're on Windows:** + ```bash + echo "test content" | gbrain put windows-test-slug + ``` + Should write the page; pre-fix this would throw `ENOENT: /dev/stdin`. + +4. **If any step fails or the numbers look wrong,** please file an issue: + https://github.com/garrytan/gbrain/issues with the output of `gbrain doctor` and which step broke. This feedback loop is how the gbrain maintainers find fragile upgrade paths. + ## [0.41.3.0] - 2026-05-24 **Pre-register Claude and ChatGPT clients without `--enable-dcr` — the SECURITY.md-recommended setup actually works now.** diff --git a/CLAUDE.md b/CLAUDE.md index 52d77e4df..2191e98c5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -115,6 +115,7 @@ strict behavior when unset. - `src/core/ai/types.ts` — provider/recipe types. **v0.28.7 (#680):** `EmbeddingTouchpoint` extended with optional `chars_per_token` (default 4 chars/token, matching OpenAI tiktoken on English) and `safety_factor` (default 0.8, budget-utilization ceiling). Both consulted only when `max_batch_tokens` is also set. Voyage declares `chars_per_token=1` + `safety_factor=0.5` to handle dense payloads (CJK/JSON/base64) that overshoot tiktoken. The pre-split budget is `max_batch_tokens × safety_factor / chars_per_token`. **v0.28.11 (#719):** `EmbeddingTouchpoint.multimodal_models?: string[]` model-level allow-list for recipes that mix text-only + multimodal models under one touchpoint (Voyage's 12 models share `supports_multimodal: true` but only `voyage-multimodal-3` accepts `/multimodalembeddings`). When omitted, recipe-level `supports_multimodal` is sufficient. `AIGatewayConfig.embedding_multimodal_model?: string` lets `embedMultimodal()` route to a different model than `embedding_model` — brains using OpenAI for text can use Voyage for images without flipping the primary embedding pipeline. **v0.37.6.0 (#1210):** new `Recipe.default_headers?: Record` (static) and `Recipe.resolveDefaultHeaders?(env)` (env-templated) seam for per-recipe headers that ride alongside auth on every openai-compat touchpoint. Mutually exclusive (declaring both throws `AIConfigError` at gateway-configure time); keys conflicting with the resolved auth header (`Authorization`, the resolver's custom header) are rejected at `applyResolveAuth` call time so defaults cannot accidentally shadow auth. Used by OpenRouter for the `HTTP-Referer` + `X-OpenRouter-Title` + `X-Title` attribution triple; usable by any future recipe (Together/Groq) that wants attribution. - `src/core/ai/gateway.ts` — unified seam for every AI call. **v0.36.3.0:** `embedQuery(text, opts?)` and `isAvailable(touchpoint, modelOverride?)` accept a model override so the resolved-column path can embed via the column's provider (Voyage / ZeroEntropy / OpenAI) instead of the global default. The hybrid path passes `{embeddingModel: resolved.provider, dimensions: resolved.dimensions}`; the gateway resolves the matching recipe and routes through its `instantiateEmbedding()` branch. `isAvailable('embedding', 'voyage:voyage-3-large')` checks the override's recipe (not the default) so hybrid skips vector search only when the active column's provider is actually down — fixes the CDX-4 bug where a healthy Voyage column would skip vector retrieval because OpenAI happened to be unconfigured. **v0.35.0.0:** ZeroEntropy support lands. New `zeroEntropyCompatFetch` shim (sibling to `voyageCompatFetch`) handles ZE's non-OpenAI-compatible wire shape — rewrites the request URL from `/embeddings` to `/models/embed`, injects `input_type` (default `'document'`; `'query'` when threaded via `providerOptions.openaiCompatible.input_type`) and explicit `encoding_format: 'float'`, and rewrites the response from `{results: [{embedding}], usage: {total_bytes, total_tokens}}` to `{data: [{embedding, index}], usage: {prompt_tokens, total_tokens}}` so the SDK's openai-compatible Zod schema validates (Voyage's shim hit the same `prompt_tokens` requirement at `:655`). Layer 1 (Content-Length) + Layer 2 (per-embedding) OOM caps via a new tagged `ZeroEntropyResponseTooLargeError` class (kept separate from `VoyageResponseTooLargeError` because `test/voyage-response-cap.test.ts` does structural source-text greps pinning the Voyage name — class unification is a deferred cleanup). Wired in `instantiateEmbedding()` via the same `recipe.id === 'zeroentropyai'` branch pattern Voyage uses. New `gateway.rerank()` native HTTP path (no AI-SDK reranking abstraction): resolves the configured reranker model via `getRerankerModel()`, posts to `${recipe.base_url}/models/rerank` with bearer auth, returns `RerankResult[]` sorted by relevance score. `RerankError.reason` classifier: `auth | rate_limit | network | timeout | payload_too_large | unknown`. 5s default timeout (search hot path). Pre-flight payload guard rejects bodies over `recipe.touchpoints.reranker.max_payload_bytes` with `reason: 'payload_too_large'` so callers can fail-open without an HTTP call. `_rerankTransport` test seam mirrors `_embedTransport`. New `gateway.embedQuery(text)` companion threads `inputType: 'query'` through `dimsProviderOptions()` (now 4-arg). `getRerankerModel()` accessor + `isAvailable('reranker')` branch added. `configureGateway` + `reconfigureGatewayWithEngine` thread `reranker_model` through the same path as embedding/expansion/chat. `applyResolveAuth` + `defaultResolveAuth` widen touchpoint param to include `'reranker'`. **v0.34.1.0 (#875):** new `embedMultimodalOpenAICompat()` routes recipes with `implementation: 'openai-compatible'` (LiteLLM, Anyscale, vLLM, Gemini multimodal via proxy) through the standard `/embeddings` endpoint with content arrays carrying `image_url` entries. The pre-existing Voyage `/multimodalembeddings` path is unchanged; the gateway selects by recipe `implementation` tag. Runtime dimension validation throws `AIConfigError` (with model id + observed + expected) before the vector reaches storage when the provider returns a width that doesn't match the recipe's `default_dims` or the brain's `embedding_dimensions` config — no more cryptic `vector dimension mismatch` at INSERT time. Pinned by 11 cases in `test/openai-compat-multimodal.test.ts`. **v0.28.7 (#680):** module-scoped `_embedTransport` defaulting to AI SDK `embedMany`, with `__setEmbedTransportForTests(fn)` test seam so tests drive the public `embed()` function with a stubbed transport instead of probing private helpers. `splitByTokenBudget` and `isTokenLimitError` are now exported `@internal` — pure functions reused directly by the test file. Module-level `_shrinkState: Map` halves the recipe's effective `safety_factor` on token-limit miss (floor 0.05) and heals back ×1.5 toward the ceiling after `SHRINK_HEAL_AFTER=10` consecutive successes. `configureGateway()` walks every registered recipe at construction time and emits a once-per-process stderr warning for any embedding touchpoint missing `max_batch_tokens` (excluding the canonical OpenAI fast-path recipe). `resetGateway()` clears `_shrinkState`, the warned-set, and restores the real transport. ASCII flow diagram embedded in the `embed()` JSDoc covers the routing decision, recursion + halving, and shrinkState lifecycle. **v0.28.11 (#719):** `embedMultimodal()` reads `cfg.embedding_multimodal_model` first (falls back to `cfg.embedding_model` for single-model setups). After the existing recipe-level `supports_multimodal` fast-fail, validates the resolved model against `touchpoint.multimodal_models` when declared — closes the Voyage-text-only-model-into-multimodal-endpoint footgun before any HTTP call (Codex F1 from PR review). New `getMultimodalModel()` accessor mirrors `getEmbeddingModel` / `getChatModel` so doctor and integration tests can read the gateway state. **v0.33.1.1 (#962, Codex P3 follow-up):** new exported `VoyageResponseTooLargeError` tagged class at the top of the file. `voyageCompatFetch`'s two OOM-defense caps (Layer 1 Content-Length check at `:595`, Layer 2 per-embedding base64 cap at `:619`) now throw `VoyageResponseTooLargeError` instead of a generic `Error`. The inbound response-rewriter's surrounding try/catch (which intentionally swallows parse failures so misshaped Voyage responses fall through to the SDK's JSON parser) checks `instanceof VoyageResponseTooLargeError` and rethrows. Pre-fix, the Layer 2 throw was silently swallowed and the oversized response returned to the AI SDK anyway — Layer 2 was theatrical. Source-shape regression assertion in `test/voyage-response-cap.test.ts` pins the `instanceof ⇒ throw err` line. - `src/core/ai/recipes/zeroentropyai.ts` (v0.35.0.0) — ZeroEntropy openai-compatible recipe declaring BOTH `embedding` (`zembed-1`, 7 Matryoshka dims: 2560/1280/640/320/160/80/40) AND `reranker` (`zerank-2` flagship + `zerank-1` + `zerank-1-small`, 5MB payload cap) touchpoints. `implementation: 'openai-compatible'` (NOT the misspelled `'openai-compat'` the original plan draft had — pinned by F1 regression in `test/ai/zeroentropy-recipe.test.ts`). `base_url_default: 'https://api.zeroentropy.dev/v1'` already ends with `/v1`, so the `zeroEntropyCompatFetch` URL rewrite `/embeddings → /models/embed` produces `…/v1/models/embed` (NOT `…/v1/v1/…` — pinned by F2 regression). `chars_per_token: 1` + `safety_factor: 0.5` match Voyage's dense-content hedge. +- `src/core/ai/recipes/llama-server-reranker.ts` (v0.40.6.1) — sibling of `llama-server` (the embedding recipe) for llama.cpp running in `--reranking` mode. Distinct recipe rather than dual-touchpoint extension because `--reranking` and `--embeddings` are mutually exclusive at server-launch time, so the two backends need independent base URLs (default 8081 here vs 8080 there). Declares `reranker` touchpoint with `models: []` (user-provided id matching the `--alias` the user launched llama-server with), `path: '/rerank'` (leaf-only; consumes the new `RerankerTouchpoint.path` override on the touchpoint; gateway concatenates with `base_url_default` which already ends in `/v1`, producing `…/v1/rerank` — original draft of this recipe set `path: '/v1/rerank'` which doubled the prefix, caught by post-impl `/codex review` and corrected before merge), `default_timeout_ms: 30_000` (consumed by `src/core/search/mode.ts`'s reranker timeout resolution chain — gives CPU-only first-call warmup headroom; the 5s mode-bundle default would fail-open silently as `timeout`), `cost_per_1m_tokens_usd: 0` (recognized by `FREE_LOCAL_RERANK_PROVIDERS` in `src/core/budget/budget-tracker.ts` so `--max-cost` callers don't TX2 hard-fail on local rerank). Setup hint emphasizes `--alias` because llama-server's `/v1/models` defaults model id to the gguf file path without it, which makes provider:model config strings ugly. Covers Qwen3-Reranker via llama.cpp AND self-hosted ZE weights via llama.cpp — same recipe, different `--model` at launch (quality parity with ZE-hosted NOT guaranteed; user pins their own eval). Pinned by `test/ai/recipe-llama-server-reranker.test.ts`. Voyage / Cohere / vLLM rerankers stay out of scope — different wire shapes need adapter hooks designed in a follow-up plan. Same wave (v0.40.6.1) adds: `path?: string` + `default_timeout_ms?: number` on `RerankerTouchpoint` in `src/core/ai/types.ts`; consumed by the URL build at `src/core/ai/gateway.ts:rerank()` and by mode-resolution at `src/core/search/mode.ts:resolveSearchMode` (precedence: per-call > config-key > recipe touchpoint default > mode bundle); `LLAMA_SERVER_RERANKER_BASE_URL` env passthrough in `src/cli.ts:buildGatewayConfig`; `FREE_LOCAL_RERANK_PROVIDERS` set in `src/core/budget/budget-tracker.ts:lookupPricing` (rerank-kind-only zero-pricing for the local provider prefix); doctor-fix at `src/commands/models.ts:probeRerankerConfig` to read `search.reranker.model` via `loadSearchModeConfig` + `resolveSearchMode` (closes file-plane / DB-plane divergence where doctor said "not configured" while live search was actively reranking — pre-fix bug was `getRerankerModel()` reading `GBrainConfig.reranker_model`, a file-plane field nothing writes); `probeRerankerReachability` upgraded to read the recipe's `default_timeout_ms` so CPU-only cold-start doesn't false-fail. - `src/core/ai/recipes/openrouter.ts` (v0.37.6.0, #1210) — OpenRouter openai-compatible recipe: single key, many providers via `openrouter:/` strings. `base_url_default: 'https://openrouter.ai/api/v1'`. Embedding touchpoint: `openai/text-embedding-3-small` at 1536 dims with Matryoshka `dims_options: [512, 768, 1024, 1536]` (native breakpoints from Weaviate's MRL analysis); `max_batch_tokens: 300_000` = OpenAI's aggregate-per-request token cap (NOT per-input — Codex caught the semantic in pre-merge review). Chat touchpoint declares 8 curated entry points (gpt-5.2, gpt-5.2-chat, gpt-5.5, claude-haiku-4.5, claude-sonnet-4.6, claude-opus-4.7, gemini-3-flash-preview, deepseek-chat) but openai-compat tier accepts any model ID; deliberately no `max_context_tokens` because OR's catalog spans 128K to 1M+. `supports_subagent_loop: false` is INFORMATIONAL — the real gate is `isAnthropicProvider()` in `src/core/model-config.ts` which hard-pins gbrain's subagent infra to Anthropic-direct. The recipe declares `resolveDefaultHeaders(env)` (the env-templated variant of the new `default_headers` seam) returning OR's three attribution headers: `HTTP-Referer` (required for OR to create an app-attribution entry), `X-OpenRouter-Title` (current preferred name per OR docs), `X-Title` (documented back-compat alias). Defaults to `https://gbrain.ai` / `gbrain`; forks (downstream agent deployments) override via `OPENROUTER_REFERER` / `OPENROUTER_TITLE` env vars so their traffic gets their attribution on OR's leaderboard. Smoke-tested by `test/ai/recipe-openrouter.test.ts` (11 cases including the D5 shape-test regression guard: every model in the chat list matches `^[a-z0-9-]+\/[a-z0-9._-]+$`, catching typos and malformed IDs without pinning the catalog's churn). - `src/core/rerank-audit.ts` (v0.35.0.0) — failure-only JSONL audit at `~/.gbrain/audit/rerank-failures-YYYY-Www.jsonl` (ISO-week rotation, mirrors `src/core/audit-slug-fallback.ts`). Exports `logRerankFailure({reason, model, query_hash, doc_count, error_summary})` + `readRecentRerankFailures(days)`. **Deliberately no `logRerankSuccess`** (CDX2-F22 in plan review): writing once per tokenmax search is hot-path I/O churn AND success events leak query volume + timing into a local audit file. `gbrain doctor`'s `reranker_health` check reads `search.reranker.enabled` first so "no events in window" is interpreted correctly (disabled → ok; enabled → ok). Query text is SHA-256-prefix-hashed (8 hex chars) for privacy. `GBRAIN_AUDIT_DIR` env override honored via the shared `resolveAuditDir()`. - `src/core/search/embedding-column.ts` (v0.36.3.0) — single source of truth for "which `content_chunks.*` column does this query rank against?" Pure functions, no engine I/O: `loadRegistry(cfg)` walks the `embedding_columns` config (DB plane, JSON map keyed by column name with `{provider, dimensions, type}` entries), seeds the OpenAI `embedding` builtin when unset, validates everything before it lands (column-name regex, type ∈ `vector | halfvec`, dims in [1, 8192], provider format) using `Object.create(null)` + `Object.hasOwn` so a key like `constructor` rejects instead of resolving to `Object.prototype.constructor`. `resolveColumn(registry, override?, cfg)` is the boundary call: returns a `ResolvedColumn` descriptor (frozen `{name, provider, dimensions, type}`) honoring per-call override → `search_embedding_column` config → `'embedding'` default. Throws `UnknownEmbeddingColumnError` with the list of registered names on miss. `isCacheSafe(resolved, cfg)` compares the full embedding SPACE (provider + dimensions + name) against cfg's default — a user who repointed the `embedding` builtin at Voyage doesn't accidentally serve OpenAI-shaped cache rows. `validateResolvedColumn(descriptor)` re-validates hand-rolled descriptors that bypass the registry (internal-SDK passthrough path) so the SQL-injection escape hatch through the descriptor field is closed. Consumed by `hybridSearch` (resolves once at the boundary, threads a `ResolvedColumn` into per-engine `searchVector` instead of a raw string), `gateway.embedQuery(text, {embeddingModel, dimensions})` (resolved column's provider drives the query-time embed call), `cosineReScore` (engine pulls vectors from the active column, not the hardcoded `embedding`), and the `query` MCP op (per-call `embedding_column` param). 538 lines of pure resolver, 511 unit cases in `test/search/embedding-column.test.ts` covering the 16 codex-flagged corners (prototype-pollution, descriptor passthrough, env-only Postgres install, empty-brain coverage gate, cache-space comparison). @@ -248,7 +249,8 @@ strict behavior when unset. - `src/core/import-checkpoint.ts` (v0.34.2.0) — `loadCheckpoint(brainDir)`, `saveCheckpoint(brainDir, completed)`, `resumeFilter(files, completed, brainDir)`, `clearCheckpoint()`, plus the `ImportCheckpoint` type. Path-set checkpoint format (`{schema_version, brainDir, completed: string[]}`) replaces the v0.33.x positional `{processedIndex: N}` format. Atomic write via `.tmp` + `rename()` so a mid-write crash never leaves a partial JSON. `loadCheckpoint` returns `null` on: missing file, malformed JSON, brainDir mismatch (you ran import against a different brain), and the old positional format (logged to stderr before being discarded). `resumeFilter` returns `{toProcess, skippedCount}` — pure, no I/O, deterministic. `clearCheckpoint` is no-op-on-missing for clean-exit cleanup. Honors `GBRAIN_HOME` via `gbrainPath()` so test isolation via `withEnv({GBRAIN_HOME: tmpdir})` works without monkey-patching the fs layer. Best-effort persistence — `saveCheckpoint` logs warnings on write errors but never throws, so import keeps making progress even if disk is full. - `src/core/sort-newest-first.ts` (v0.34.2.0) — single source of truth for the descending-lex sort that `gbrain import` and `gbrain sync` both apply. Mutates in place (Array.prototype.sort semantics), returns the same array reference for fluent chaining. Empty/single-element inputs short-circuit. Future ordering changes flip one line in this helper instead of touching two CLI commands. Pinned by `test/sort-newest-first.test.ts` (5 hermetic cases: descending order, mixed prefixes, empty input, single-element input, in-place-mutation contract). - `src/core/cycle.ts` — v0.17 brain maintenance cycle primitive (extended to **9 phases in v0.29**). `runCycle(engine: BrainEngine | null, opts: CycleOpts): Promise` composes phases in semantically-driven order: **lint → backlinks → sync → synthesize → extract → patterns → recompute_emotional_weight → embed → orphans**. v0.29 adds the `recompute_emotional_weight` phase between patterns and embed; it sees the union of `syncPagesAffected` + `synthesizeWrittenSlugs` for incremental mode, or all pages when neither anchor is set (full backfill via `gbrain dream --phase recompute_emotional_weight`). v0.29 also extends `CycleReport.totals` with `pages_emotional_weight_recomputed` (additive, schema_version stays "1"). v0.23's `synthesize` phase runs after sync (cross-references see fresh brain) and before extract (auto-link materializes its writes); `patterns` runs after extract so it reads a fresh graph (codex finding #7 — subagent put_page sets `ctx.remote=true` and skips auto-link/timeline by default; extract is the canonical materialization). Three callers: `gbrain dream` CLI, `gbrain autopilot` daemon's inline path, and the Minions `autopilot-cycle` handler. Coordination via `gbrain_cycle_locks` DB table + `~/.gbrain/cycle.lock` file lock with PID-liveness for PGLite. `CycleReport.schema_version: "1"` is stable; totals additively grew in v0.23 (`transcripts_processed`, `synth_pages_written`, `patterns_written`). `yieldBetweenPhases` runs between phases. **v0.23 added `yieldDuringPhase`** for in-phase keepalive — synthesize/patterns call it during long waits to renew the cycle-lock TTL. Engine nullable; lock-skip on read-only phase selections. v0.22.1 (#403): `CycleOpts.signal?: AbortSignal` propagates the worker's abort signal; `checkAborted()` fires between every phase. v0.22.1 (#417): `runPhaseSync` returns `pagesAffected` via `SyncPhaseResult`; `runCycle` captures it and threads to `runPhaseExtract` as the 4th arg. v0.22.1 (Codex F2): `runPhaseSync` takes `willRunExtractPhase: boolean` and sets `noExtract: phases.includes('extract')` so `gbrain dream --phase sync` doesn't silently lose extraction. v0.22.5 (#475): `resolveSourceForDir(engine, brainDir)` threads `sourceId` to `performSync()` so sync reads the per-source `sources.last_commit` anchor instead of the drift-prone global `config.sync.last_commit` key. -- `src/core/cycle/synthesize.ts` (v0.23) — Synthesize phase: conversation-transcript-to-brain pipeline. Reads from `dream.synthesize.session_corpus_dir`, runs cheap Haiku verdict (cached in `dream_verdicts`), then fans out one Sonnet subagent per worth-processing transcript with `allowed_slug_prefixes` (sourced from `skills/_brain-filing-rules.json` `dream_synthesize_paths.globs`). Orchestrator collects slugs from `subagent_tool_executions` (NOT `pages.updated_at` — codex finding #2) and reverse-renders DB → markdown via `serializeMarkdown`. Cooldown via `dream.synthesize.last_completion_ts`, written ONLY on success. Idempotency key `dream:synth::`. Auto-commit deferred to v1.1 (codex #5). `--dry-run` runs Haiku, skips Sonnet (codex #8). Subagent never gets fs-write access. **v0.23.2:** `renderPageToMarkdown` (now exported) stamps `dream_generated: true` and `dream_cycle_date` into every reverse-write's frontmatter; `writeSummaryPage` does the same on the dream-cycle summary index. The marker is the explicit identity surface checked by `isDreamOutput` in `transcript-discovery.ts` — replaces the v0.23.1 content-prefix heuristic that could miss real output (`serializeMarkdown` doesn't embed slugs in body) and false-positive on user transcripts citing brain pages. `judgeSignificance` and `JudgeClient` are exported; `judgeSignificance` accepts a `verdictModel` parameter (default `claude-haiku-4-5-20251001`) loaded from `dream.synthesize.verdict_model` via `loadSynthConfig`. **v0.30.2:** model-aware chunker `splitTranscriptByBudget(content, contentHash, maxChars)` splits oversized transcripts at paragraph boundaries (`## Topic:` → `---` → `\n` ladder) using a deterministic offset seeded from the first 32 bits of `contentHash` so retries chunk identically. Per-chunk char budget computed from `MODEL_CONTEXT_TOKENS[resolvedModel] × 0.9 × 3.5 chars/token`; non-Anthropic ids fall back to a 180K-token safe default with a once-per-process stderr warning. Operator overrides: `dream.synthesize.max_prompt_tokens` (floor 100K, wins when set) and `dream.synthesize.max_chunks_per_transcript` (default 24). Per-chunk idempotency keys `dream:synth:::cof`; single-chunk transcripts preserve the legacy `dream:synth::` key byte-for-byte (D8 lookup), so existing brains skip with `already_synthesized_legacy_single_chunk` instead of re-spending Sonnet on upgrade. `collectChildPutPageSlugs` raw-fetches every (job_id, slug) pair (not `SELECT DISTINCT`) and rewrites bare-hash6 slugs to `-c` for chunked children (D6 — orchestrator-side, zero Sonnet trust). Cap-hit skips don't write to `dream_verdicts`, so raising the cap on next run re-attempts cleanly. D7 scope: bounds INITIAL prompt size only; tool-loop turn-N accumulation is caught by the v0.30.2 terminal-error classification in `subagent.ts`, not bounded ahead of time. +- `src/core/cycle/synthesize.ts` (v0.23) — Synthesize phase: conversation-transcript-to-brain pipeline. Reads from `dream.synthesize.session_corpus_dir`, runs cheap Haiku verdict (cached in `dream_verdicts`), then fans out one Sonnet subagent per worth-processing transcript with `allowed_slug_prefixes` (sourced from `skills/_brain-filing-rules.json` `dream_synthesize_paths.globs`). Orchestrator collects slugs from `subagent_tool_executions` (NOT `pages.updated_at` — codex finding #2) and reverse-renders DB → markdown via `serializeMarkdown`. Cooldown via `dream.synthesize.last_completion_ts`, written ONLY on success. Idempotency key `dream:synth::`. Auto-commit deferred to v1.1 (codex #5). `--dry-run` runs Haiku, skips Sonnet (codex #8). Subagent never gets fs-write access. **v0.23.2:** `renderPageToMarkdown` (now exported) stamps `dream_generated: true` and `dream_cycle_date` into every reverse-write's frontmatter; `writeSummaryPage` does the same on the dream-cycle summary index. The marker is the explicit identity surface checked by `isDreamOutput` in `transcript-discovery.ts` — replaces the v0.23.1 content-prefix heuristic that could miss real output (`serializeMarkdown` doesn't embed slugs in body) and false-positive on user transcripts citing brain pages. `judgeSignificance` and `JudgeClient` are exported; `judgeSignificance` accepts a `verdictModel` parameter (default `claude-haiku-4-5-20251001`) loaded from `dream.synthesize.verdict_model` via `loadSynthConfig`. **v0.30.2:** model-aware chunker `splitTranscriptByBudget(content, contentHash, maxChars)` splits oversized transcripts at paragraph boundaries (`## Topic:` → `---` → `\n` ladder) using a deterministic offset seeded from the first 32 bits of `contentHash` so retries chunk identically. Per-chunk char budget computed from `MODEL_CONTEXT_TOKENS[resolvedModel] × 0.9 × 3.5 chars/token`; non-Anthropic ids fall back to a 180K-token safe default with a once-per-process stderr warning. Operator overrides: `dream.synthesize.max_prompt_tokens` (floor 100K, wins when set) and `dream.synthesize.max_chunks_per_transcript` (default 24). Per-chunk idempotency keys `dream:synth:::cof`; single-chunk transcripts preserve the legacy `dream:synth::` key byte-for-byte (D8 lookup), so existing brains skip with `already_synthesized_legacy_single_chunk` instead of re-spending Sonnet on upgrade. `collectChildPutPageSlugs` raw-fetches every (job_id, slug) pair (not `SELECT DISTINCT`) and rewrites bare-hash6 slugs to `-c` for chunked children (D6 — orchestrator-side, zero Sonnet trust). Cap-hit skips don't write to `dream_verdicts`, so raising the cap on next run re-attempts cleanly. D7 scope: bounds INITIAL prompt size only; tool-loop turn-N accumulation is caught by the v0.30.2 terminal-error classification in `subagent.ts`, not bounded ahead of time. **v0.41+ community-PR-wave (reworked PR #1349):** `makeHaikuClient()` replaced by gateway-routed `makeJudgeClient(verdictModel)` exported helper — mirrors `tryBuildGatewayClient` in `src/core/think/index.ts:579-637` (the v0.35.5.0 #952 pattern). Construction-time provider/key probe returns `null` on a clear miss (unknown provider id via `resolveRecipe` AIConfigError, or Anthropic provider with no key via `hasAnthropicKey()` matching the same env + config-file pattern think uses). Verdict loop wraps `judgeSignificance` in try/catch for `AIConfigError` so mid-run provider failures (revoked key, recipe misconfig) surface as per-transcript `worth=false, reasons=['gateway error: ...']` instead of crashing the phase. Canonical config key `models.dream.synthesize_verdict` (per `PER_TASK_KEYS` in `src/core/model-config.ts`); contributor's parallel `DEEPSEEK_API_KEY` fetch adapter dropped in favor of recipe-registered providers (DeepSeek, OpenRouter, Voyage, Ollama, llama-server, ...). `JudgeClient` interface signature preserved verbatim for test-seam stability. CI guard at `scripts/check-gateway-routed-no-direct-anthropic.sh` prevents future contributors from re-introducing `new Anthropic()` here or in `think/index.ts`. Pinned by 13 cases in `test/cycle/synthesize-gateway-adapter.test.ts` (A1-A9 unit + R3 parsed-verdict semantic parity + R3 corollary) and 1 E2E case in `test/e2e/dream-synthesize-pglite.test.ts` ('gateway-adapter mid-run AIConfigError catch') plus 4 R1-R4 critical regression pins in `test/cycle/regression-pr-wave-r1-r2-r4.test.ts`. +- `scripts/check-gateway-routed-no-direct-anthropic.sh` (v0.41+ community-PR-wave) — CI guard that fails the build if `src/core/cycle/synthesize.ts` or `src/core/think/index.ts` reintroduces a runtime `new Anthropic()` constructor call or a value-shaped `import Anthropic from '@anthropic-ai/sdk'` import. Type-only imports (`import type Anthropic from '@anthropic-ai/sdk'`) stay allowed for adapter types. Comment lines (`//` or ` *` prefixes) are excluded so historical JSDoc references don't false-fire. Mirrors `scripts/check-jsonb-pattern.sh`. Wired into `bun run verify` and `bun run check:all`. Extend `GUARDED_FILES` in the script when migrating another file off direct SDK construction. - `src/core/cycle/patterns.ts` (v0.23) — Patterns phase: cross-session theme detection over reflections within `dream.patterns.lookback_days` (default 30). Names a pattern only when ≥`dream.patterns.min_evidence` (default 3) reflections support it. Single Sonnet subagent; same allow-list path as synthesize. Runs AFTER `extract` so the graph is fresh. - `src/core/cycle/extract-facts.ts` (v0.32.2, extended v0.35.6.0) — extract_facts cycle phase. v0.32.2 contract: fence is canonical; per-page wipe (`deleteFactsForPage`) + reinsert from `parseFactsFence` + `extractFactsFromFenceText` + `engine.insertFacts`. Empty-fence guard refuses when v0.31 legacy rows (`row_num IS NULL AND entity_slug IS NOT NULL`) pend the v0_32_2 backfill (status: warn, hint: `gbrain apply-migrations --yes`). **v0.35.6.0** adds a phantom-redirect pre-pass that runs AFTER the legacy-row guard, BEFORE the main reconcile loop. When `opts.brainDir` is set, `runPhantomRedirectPass(engine, brainDir, sourceId, dryRun)` walks unprefixed-slug pages capped by `GBRAIN_PHANTOM_REDIRECT_LIMIT` (default 50). The pass returns `touched_canonicals` — canonical slugs whose disk fence was merged with phantom rows; `runExtractFacts` UNIONs them into the main reconcile slug set so canonical's DB facts derive from the merged fence in the same cycle (round-14 scenario-B fix: phantom had only-on-disk fence, no DB facts). `ExtractFactsResult` gains six phantom fields: `phantomsScanned`, `phantomsRedirected`, `phantomsAmbiguous`, `phantomsSkippedDrift`, `phantomsLockBusy`, `phantomsMorePending`. Three of those bubble to `CycleReport.totals` (`phantoms_redirected`, `phantoms_ambiguous`, `phantoms_skipped_drift`). - `src/core/entities/resolve.ts` (v0.30+, extended v0.35.6.0) — Free-form entity name → canonical slug resolution. `resolveEntitySlug(engine, source_id, raw)`: exact slug → fuzzy (pg_trgm @ 0.4 threshold) → bare-name prefix expansion (`people/-%` then `companies/-%` using correlated-subquery `connection_count` for tiebreaker) → deterministic `slugify` fallback. **v0.35.6.0** exports two new helpers for the phantom-redirect pass: `resolvePhantomCanonical(engine, sourceId, phantomSlug)` — variant that SKIPS the exact-slug step (codex #1: phantom slug `'alice'` exact-matches itself, would make the redirect handler a no-op); returns the canonical only when result is non-null AND contains `/`. `findPrefixCandidates(engine, sourceId, token)` — standalone SQL query returning ALL candidates across `PREFIX_EXPANSION_DIRS` (currently hardcoded `['people', 'companies']`) using `slug LIKE ANY($N::text[])` over patterns `dir/token` + `dir/token-%`; cap of 10 ordered by `connection_count DESC, slug ASC`. NOT a wrapper around `tryPrefixExpansion` because that path returns per-dir top-1 and suppresses ambiguity by design (codex #11). Pinned by `test/phantom-redirect.test.ts` resolvePhantomCanonical describe (3 cases) + findPrefixCandidates describe (6 cases including multi-dir ambiguity and the `people/aliceberg`-doesn't-match-`alice` false-positive guard). diff --git a/README.md b/README.md index 228a074e3..867240ac2 100644 --- a/README.md +++ b/README.md @@ -250,6 +250,7 @@ Data flowing into the brain. Each integration is a recipe — markdown + setup h - **Voice**: Phone calls create brain pages via Twilio + OpenAI Realtime (or DIY STT+LLM+TTS). Setup recipe: [`recipes/twilio-voice-brain.md`](recipes/twilio-voice-brain.md). - **Email + calendar**: webhook handlers that route to brain signals. [`docs/integrations/meeting-webhooks.md`](docs/integrations/meeting-webhooks.md). - **Embedding providers**: 16 recipes covering OpenAI (default fallback), OpenRouter, Voyage, ZeroEntropy (default), Google Gemini, Azure OpenAI, MiniMax, Alibaba DashScope, Zhipu, Ollama (local), llama.cpp llama-server (local), LiteLLM proxy. Pricing matrix + decision tree in [`docs/integrations/embedding-providers.md`](docs/integrations/embedding-providers.md). +- **Rerankers**: ZeroEntropy `zerank-2` hosted (default in `tokenmax` mode) plus the v0.40.6.1 `llama-server-reranker` recipe for fully-local cross-encoder rerank via llama.cpp — runs Qwen3-Reranker or self-hosted ZeroEntropy weights against the same `gateway.rerank()` seam. Setup walkthrough in [`docs/ai-providers/llama-server-reranker.md`](docs/ai-providers/llama-server-reranker.md). - **Credential gateway**: vault-aware secret distribution. [`docs/integrations/credential-gateway.md`](docs/integrations/credential-gateway.md). - **MCP clients**: every major MCP client is supported. [`docs/mcp/`](docs/mcp/) per-client setup. diff --git a/TODOS.md b/TODOS.md index db53520f6..38d6aadd8 100644 --- a/TODOS.md +++ b/TODOS.md @@ -1,5 +1,41 @@ # TODOS +## community-pr-wave follow-ups (filed during ship) + +- [ ] **`FREE_LOCAL_*_PROVIDERS` zero-pricing bypassable via redirected + BASE_URL env vars.** An operator who sets `LLAMA_SERVER_BASE_URL=https://paid-api.com/v1` + routes `llama-server:foo` requests to a paid proxy, but the budget + tracker still zero-prices them because the provider-prefix match in + `FREE_LOCAL_EMBED_PROVIDERS` / `FREE_LOCAL_RERANK_PROVIDERS` doesn't + see the resolved URL. The bypass is real but requires operator + misconfiguration (paid-API behind a "local" recipe alias) — same + trust posture as the rest of the BASE_URL env vars. + + Fix shape (couples with the unification TODO already filed for v0.41+): + move the freeness decision from provider-prefix lookup to the gateway's + embed/rerank call sites where the resolved URL is known, or detect + non-loopback `provider_base_urls` and refuse zero-pricing in that case. + + Surfaced by codex Pass-9 adversarial review; pre-existing for the rerank + case in v0.40.7.1, broadened to embed by v0.40.8.0. Tracked here so the + unification PR closes both at once. + +- [ ] **`probeEmbeddingReachability` should honor recipe `default_timeout_ms` + for embed touchpoint.** The reranker probe was just fixed in PR #1326 to + read `recipe.touchpoints.reranker.default_timeout_ms` so Qwen3-Reranker-4B + has CPU cold-start headroom. The embedding probe hardcodes 5000ms + (`src/commands/models.ts:467`) and the JSDoc admits "the 5s timeout may + trip on the very first probe — re-run if so." A local llama-server embed + endpoint hits the identical CPU cold-start curve. + + Fix: add optional `default_timeout_ms?: number` to `EmbeddingTouchpoint` + in `src/core/ai/types.ts` (sibling to the rerank field), thread through + `probeEmbeddingReachability` using the same `recipe.touchpoints.embedding.default_timeout_ms ?? 5000` + pattern that the reranker probe uses. Add a regression test in + `test/models-doctor-embed.test.ts` pinning the precedence chain. + + Surfaced by the community-PR-wave pre-landing review (informational, no + blocker on the wave itself — workaround is "re-run the probe"). ## v0.41.3 security/MCP fix wave follow-ups (filed during ship of `garrytan/security-mcp-fix-wave`) Source: codex outside-voice review on the v0.41.3 wave (D7) identified @@ -514,6 +550,60 @@ at plan time and got carved out: alongside the staleness number so doctor surfaces the coverage gap inline. Implementation: reuse `buildSyncStatusReport` from `src/commands/sync.ts`, +## v0.40.6.1 llama-server-reranker follow-ups (v0.40.7+) + +Filed from the /ship Claude adversarial subagent review against this PR. None are +exploitable today; they harden the new local-reranker surface against future +contributor traps. + +- [ ] **P1: SSRF scheme validation sweep for all 6 openai-compat `_BASE_URL` env vars.** + `src/cli.ts:1483-1487` accepts `LLAMA_SERVER_BASE_URL`, `LLAMA_SERVER_RERANKER_BASE_URL`, + `OLLAMA_BASE_URL`, `LMSTUDIO_BASE_URL`, `LITELLM_BASE_URL`, `OPENROUTER_BASE_URL` with + zero scheme validation. A `file://` or `gopher://` value silently becomes the + recipe's base URL. Pre-existing pattern; this wave adds one more env var to the gap + without expanding the class. Fix: add a `validateOpenAICompatBaseURL(url)` helper + (assert `http(s):` scheme + reuse `src/core/ssrf-validate.ts` private-IP checks + for the non-localhost case), apply to all 6 envs at the `buildGatewayConfig` site. + ~20 LOC + 6 test cases. Should be its own focused PR. + +- [ ] **P2: Document `FREE_LOCAL_RERANK_PROVIDERS` invariant.** `src/core/budget/budget-tracker.ts:lookupPricing` + returns `{input:0, output:0}` for any model id under the `llama-server-reranker:` + provider on the rerank kind. The contract relies on all callers going through + `gateway.rerank()`'s `assertTouchpoint`-with-extended-models check (which validates + the model exists before pricing fires). Theoretical bypass: a future caller that + reserves directly against BudgetTracker with `kind: 'rerank'` and an arbitrary + `llama-server-reranker:` model id gets free pricing. Fix: code comment + documenting the invariant, OR move the freeness check to gateway.rerank() where + the validation already runs. + +- [ ] **P2: Recipe path-concat sanity check at gateway-init.** `src/core/ai/gateway.ts:rerank()` + concatenates `${compat.baseURL.replace(/\/$/, '')}${tp.path ?? '/models/rerank'}`. + A future recipe with `path: 'rerank'` (no leading slash) produces `…/v1rerank`; + a future recipe with `path: '/v1/rerank'` when `base_url_default` already ends + in `/v1` reintroduces the codex-caught doubling bug. Fix: at `configureGateway` + time, assert `tp.path` (when set) starts with `/` and warn-log when the recipe + pattern looks doubling-prone. Surface at init, not first-rerank. + +- [ ] **P3: Debug-log on malformed `search.reranker.model`.** `src/core/search/mode.ts:lookupRerankerRecipeDefaultTimeout` + silently returns undefined when `getRecipe(providerId)` misses (typos, malformed + strings). Fail-open is correct for timeouts (5000ms is a safe bundle default), + but the user-facing UX is "config was set, nothing changed" with no signal. + Fix: stderr-log once when `modelStr` is non-empty but the provider id doesn't + resolve, gated by `GBRAIN_DEBUG=1`. + +- [ ] **P3: Narrow `resolveLiveRerankerModel` catch.** `src/commands/models.ts:resolveLiveRerankerModel` + has a blanket `try/catch` around `loadSearchModeConfig` + `resolveSearchMode` + that falls back to `getRerankerModel()`. Real errors (schema-version mismatch, + malformed config JSON, engine connectivity blip) get hidden behind a misleading + "not configured" doctor verdict. Fix: narrow the catch to specific shapes OR + emit `GBRAIN_DEBUG=1` stderr warning before falling back. + +- [ ] **P3: Validate `modelStr` shape before allocating probe timeout.** + `src/commands/models.ts:probeRerankerReachability` resolves the recipe + sets + `probeTimeoutMs = 30000` before checking that `modelStr` has a non-empty model + half. Result: `llama-server-reranker:` (trailing colon, empty model) waits 30s + before failing at `assertTouchpoint`. Fix: regex-validate `modelStr` shape + (`^[a-z][a-z0-9-]*:[a-zA-Z0-9_.-]+$`) before timeout allocation. ## v0.40.1.0 Track D follow-ups (v0.41+) diff --git a/VERSION b/VERSION index fc2e80cfb..eaecdf9b7 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.41.3.0 \ No newline at end of file +0.41.4.0 diff --git a/docs/ai-providers/llama-server-reranker.md b/docs/ai-providers/llama-server-reranker.md new file mode 100644 index 000000000..2cf5428a2 --- /dev/null +++ b/docs/ai-providers/llama-server-reranker.md @@ -0,0 +1,161 @@ +# llama-server reranker (local) — Qwen3-Reranker, self-hosted ZE, any ZE-wire-shape provider + +[`llama-server`](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md) +is the HTTP wrapper that ships with llama.cpp. With `--reranking`, it +exposes an OpenAI-style `POST /v1/rerank` endpoint that returns +`{results: [{index, relevance_score}]}` — exactly the wire shape gbrain +already drives for ZeroEntropy's hosted reranker. The +`llama-server-reranker` recipe (added in v0.40.6.1) routes +`gateway.rerank()` at your local llama.cpp instance instead of ZE. + +Two flavors of "local" this recipe covers: + +- **Qwen3-Reranker** (0.6B / 4B / 8B) — open-weight cross-encoder; pull + the GGUF from HuggingFace and serve. +- **Self-hosted ZeroEntropy** (`zerank-2`, `zerank-1-small`) — the + weights are on HuggingFace too. GGUF-convert them and serve them the + same way. **Quality is not guaranteed to match ZE-hosted:** GGUF + conversion + quantization + pooling/rank metadata + tokenizer special + tokens all affect scores. If you self-host ZE for production + retrieval, pin your own brain-relevant eval ( + [docs/eval-bench.md](../eval-bench.md)) as a regression guard. + +This recipe is the path override + recipe shape. Any provider whose +request/response wire matches ZE/llama.cpp can use it by just pointing +at a different base URL. Providers whose wire shape differs (Voyage uses +`top_k` not `top_n`, returns `data[]` not `results[]`) need a separate +recipe with adapter hooks — that lands in a follow-up plan. + +## Setup + +### 1. Build llama.cpp (or download a release) + +```bash +# Clone and build (CPU only; add `-DGGML_CUDA=ON` for GPU) +git clone https://github.com/ggml-org/llama.cpp.git +cd llama.cpp +cmake -B build +cmake --build build --config Release -j +``` + +Pin a specific commit when you ship — `llama-server`'s path aliases +(`/rerank`, `/v1/rerank`, `/reranking`, `/v1/reranking`) have shifted +across releases. The recipe sends to `/v1/rerank`. + +### 2. Pull a reranker GGUF + +For Qwen3-Reranker-4B (quantized Q4_K_M is the sweet spot for CPU): + +```bash +# Pick a quant level — Q4_K_M is the usual CPU sweet spot. +huggingface-cli download \ + Qwen/Qwen3-Reranker-4B-GGUF qwen3-reranker-4b-q4_k_m.gguf \ + --local-dir ./models +``` + +For self-hosted ZeroEntropy weights, find a community GGUF conversion +or convert from the HuggingFace weights yourself (out of scope of this +doc — see llama.cpp's `convert_hf_to_gguf.py`). + +### 3. Launch llama-server with --reranking AND --alias + +```bash +./build/bin/llama-server \ + --model ./models/qwen3-reranker-4b-q4_k_m.gguf \ + --alias qwen3-reranker-4b \ + --reranking \ + --port 8081 +``` + +The `--alias` matters: without it, llama-server's `/v1/models` (and the +`model` field rerank requests echo) defaults to the full gguf file +path, which makes the gbrain config string ugly and brittle. With +`--alias qwen3-reranker-4b`, your config string is short and stable. + +`--reranking` and `--embeddings` are mutually exclusive at server +launch. If you also run a local embedder via the +[`llama-server`](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md) +recipe, run two separate llama-server processes on two different ports +(typically 8080 for embeddings, 8081 for reranking — gbrain's defaults +match that convention). + +### 4. Wire gbrain at your server + +```bash +# Point gbrain at the llama.cpp host (skip if running locally on default port) +gbrain config set provider_base_urls.llama-server-reranker http://your-host:8081/v1 + +# Tell search to use this reranker +gbrain config set search.reranker.model llama-server-reranker:qwen3-reranker-4b +gbrain config set search.reranker.enabled true +``` + +The `qwen3-reranker-4b` after the colon is your `--alias` value from +step 3. Any string works as long as it matches your server's alias. + +Env vars work too as an alternative to the config set above: + +```bash +export LLAMA_SERVER_RERANKER_BASE_URL=http://your-host:8081/v1 +# Optional: if you front llama-server with nginx + bearer auth +export LLAMA_SERVER_RERANKER_API_KEY=your-bearer-token +``` + +### 5. Verify + +```bash +gbrain models doctor +# Expect: ✔ reranker_config llama-server-reranker:qwen3-reranker-4b ok +# ✔ reranker_config llama-server-reranker:qwen3-reranker-4b ok (reachability) + +gbrain search "some query" --json | jq '.[].rerank_score' +# Expect: rerank_score on every row +``` + +If `gbrain models doctor` reports the reachability probe as `network` +status, two common causes: + +1. The server is reachable but in embedding mode, not reranking mode. + `--reranking` and `--embeddings` are mutually exclusive at launch + — relaunch the right one. +2. The recipe path doesn't match what your llama.cpp version serves. + This recipe sends `/v1/rerank`; older llama.cpp installs may only + serve `/rerank`. Pin to a recent llama.cpp commit. + +## Cold-start headroom + +CPU-only first-call warmup on a 4B reranker can take 8-15 seconds. The +recipe declares `default_timeout_ms: 30000` so the first call after a +server restart doesn't fail-open silently. That value flows through +search-mode resolution unless you override it: + +```bash +# Tighten or loosen per-search timeout (overrides recipe default): +gbrain config set search.reranker.timeout_ms 60000 +``` + +Per-call overrides in `SearchOpts.reranker_timeout_ms` still win for +any single call. + +## Budget caps + local rerank + +The recipe declares `cost_per_1m_tokens_usd: 0` and registers under +`FREE_LOCAL_RERANK_PROVIDERS` in the budget tracker, so +`--max-cost`-bounded callers (autopilot loops, batch jobs) do NOT +hard-fail when configured for local rerank. Local rerank costs +electricity, not API tokens. + +```bash +GBRAIN_MAX_USD=0.01 gbrain search "..." --reranker llama-server-reranker:qwen3-reranker-4b +# Works: rerank fires, recorded at $0, cumulative cap untouched. +``` + +## Fail-open contract preserved + +`applyReranker` in `src/core/search/rerank.ts` still has the +fail-open posture: any error class (network, timeout, malformed +response) logs to `~/.gbrain/audit/rerank-failures-*.jsonl` and +returns the original RRF order unchanged. Search reliability beats +reranker quality. If your llama.cpp host goes down, your searches keep +working — they just stop ranking against the cross-encoder until you +restart the server. diff --git a/docs/integrations/embedding-providers.md b/docs/integrations/embedding-providers.md index ef0949740..36d981a80 100644 --- a/docs/integrations/embedding-providers.md +++ b/docs/integrations/embedding-providers.md @@ -63,7 +63,8 @@ The doctor distinguishes two repair paths: - **Cost-sensitive, English-only**: Ollama (free, local) or Voyage (paid, best quality per dollar). - **Quality-first**: Voyage `voyage-4-large` (1024-2048 dims, ~3-4× more dense tokens than OpenAI tiktoken). - **Code-heavy brain (gstack per-worktree, source repos)**: Voyage `voyage-code-3` (1024 default; supports 256/512/1024/2048). Tuned on programming languages. Voyage publishes head-to-head numbers showing it outperforms their general flagships on code retrieval ([voyageai.com/blog](https://voyageai.com/blog)). For gstack's per-worktree pglite-backed code brain, this is the right default — see Topology 3 in `docs/architecture/topologies.md`. -- **Reranking pair**: Voyage (their reranker `rerank-2.5` pairs cleanly with Voyage embeddings). +- **Reranking pair**: ZeroEntropy `zerank-2` is the hosted default in `tokenmax` mode (see [`docs/ai-providers/zeroentropy.md`](../ai-providers/zeroentropy.md)). Voyage `rerank-2.5` pairs cleanly with Voyage embeddings. +- **Local reranking (no API spend)**: `llama-server-reranker` recipe (v0.40.6.1) — point gbrain at your own `llama-server --reranking` instance running Qwen3-Reranker or self-hosted ZeroEntropy weights. Same `gateway.rerank()` seam, $0 per call. Walkthrough in [`docs/ai-providers/llama-server-reranker.md`](../ai-providers/llama-server-reranker.md). - **One key for many hosted models**: OpenRouter. Set `OPENROUTER_API_KEY` and use `openrouter:/` for chat against GPT-5.2, Claude 4.x, Gemini 3, DeepSeek, and dozens more without juggling per-provider keys. Embedding catalog includes OpenAI, Google, Qwen, BGE-M3. - **Enterprise compliance**: Azure OpenAI (data residency + private endpoints) or self-hosted via llama-server / Ollama. - **China region**: DashScope (Alibaba) or Zhipu (BigModel). DashScope's international endpoint at `dashscope-intl.aliyuncs.com`; override `provider_base_urls.dashscope` for the China endpoint. diff --git a/llms-full.txt b/llms-full.txt index 05c16ec80..65d7ce8da 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -257,6 +257,7 @@ strict behavior when unset. - `src/core/ai/types.ts` — provider/recipe types. **v0.28.7 (#680):** `EmbeddingTouchpoint` extended with optional `chars_per_token` (default 4 chars/token, matching OpenAI tiktoken on English) and `safety_factor` (default 0.8, budget-utilization ceiling). Both consulted only when `max_batch_tokens` is also set. Voyage declares `chars_per_token=1` + `safety_factor=0.5` to handle dense payloads (CJK/JSON/base64) that overshoot tiktoken. The pre-split budget is `max_batch_tokens × safety_factor / chars_per_token`. **v0.28.11 (#719):** `EmbeddingTouchpoint.multimodal_models?: string[]` model-level allow-list for recipes that mix text-only + multimodal models under one touchpoint (Voyage's 12 models share `supports_multimodal: true` but only `voyage-multimodal-3` accepts `/multimodalembeddings`). When omitted, recipe-level `supports_multimodal` is sufficient. `AIGatewayConfig.embedding_multimodal_model?: string` lets `embedMultimodal()` route to a different model than `embedding_model` — brains using OpenAI for text can use Voyage for images without flipping the primary embedding pipeline. **v0.37.6.0 (#1210):** new `Recipe.default_headers?: Record` (static) and `Recipe.resolveDefaultHeaders?(env)` (env-templated) seam for per-recipe headers that ride alongside auth on every openai-compat touchpoint. Mutually exclusive (declaring both throws `AIConfigError` at gateway-configure time); keys conflicting with the resolved auth header (`Authorization`, the resolver's custom header) are rejected at `applyResolveAuth` call time so defaults cannot accidentally shadow auth. Used by OpenRouter for the `HTTP-Referer` + `X-OpenRouter-Title` + `X-Title` attribution triple; usable by any future recipe (Together/Groq) that wants attribution. - `src/core/ai/gateway.ts` — unified seam for every AI call. **v0.36.3.0:** `embedQuery(text, opts?)` and `isAvailable(touchpoint, modelOverride?)` accept a model override so the resolved-column path can embed via the column's provider (Voyage / ZeroEntropy / OpenAI) instead of the global default. The hybrid path passes `{embeddingModel: resolved.provider, dimensions: resolved.dimensions}`; the gateway resolves the matching recipe and routes through its `instantiateEmbedding()` branch. `isAvailable('embedding', 'voyage:voyage-3-large')` checks the override's recipe (not the default) so hybrid skips vector search only when the active column's provider is actually down — fixes the CDX-4 bug where a healthy Voyage column would skip vector retrieval because OpenAI happened to be unconfigured. **v0.35.0.0:** ZeroEntropy support lands. New `zeroEntropyCompatFetch` shim (sibling to `voyageCompatFetch`) handles ZE's non-OpenAI-compatible wire shape — rewrites the request URL from `/embeddings` to `/models/embed`, injects `input_type` (default `'document'`; `'query'` when threaded via `providerOptions.openaiCompatible.input_type`) and explicit `encoding_format: 'float'`, and rewrites the response from `{results: [{embedding}], usage: {total_bytes, total_tokens}}` to `{data: [{embedding, index}], usage: {prompt_tokens, total_tokens}}` so the SDK's openai-compatible Zod schema validates (Voyage's shim hit the same `prompt_tokens` requirement at `:655`). Layer 1 (Content-Length) + Layer 2 (per-embedding) OOM caps via a new tagged `ZeroEntropyResponseTooLargeError` class (kept separate from `VoyageResponseTooLargeError` because `test/voyage-response-cap.test.ts` does structural source-text greps pinning the Voyage name — class unification is a deferred cleanup). Wired in `instantiateEmbedding()` via the same `recipe.id === 'zeroentropyai'` branch pattern Voyage uses. New `gateway.rerank()` native HTTP path (no AI-SDK reranking abstraction): resolves the configured reranker model via `getRerankerModel()`, posts to `${recipe.base_url}/models/rerank` with bearer auth, returns `RerankResult[]` sorted by relevance score. `RerankError.reason` classifier: `auth | rate_limit | network | timeout | payload_too_large | unknown`. 5s default timeout (search hot path). Pre-flight payload guard rejects bodies over `recipe.touchpoints.reranker.max_payload_bytes` with `reason: 'payload_too_large'` so callers can fail-open without an HTTP call. `_rerankTransport` test seam mirrors `_embedTransport`. New `gateway.embedQuery(text)` companion threads `inputType: 'query'` through `dimsProviderOptions()` (now 4-arg). `getRerankerModel()` accessor + `isAvailable('reranker')` branch added. `configureGateway` + `reconfigureGatewayWithEngine` thread `reranker_model` through the same path as embedding/expansion/chat. `applyResolveAuth` + `defaultResolveAuth` widen touchpoint param to include `'reranker'`. **v0.34.1.0 (#875):** new `embedMultimodalOpenAICompat()` routes recipes with `implementation: 'openai-compatible'` (LiteLLM, Anyscale, vLLM, Gemini multimodal via proxy) through the standard `/embeddings` endpoint with content arrays carrying `image_url` entries. The pre-existing Voyage `/multimodalembeddings` path is unchanged; the gateway selects by recipe `implementation` tag. Runtime dimension validation throws `AIConfigError` (with model id + observed + expected) before the vector reaches storage when the provider returns a width that doesn't match the recipe's `default_dims` or the brain's `embedding_dimensions` config — no more cryptic `vector dimension mismatch` at INSERT time. Pinned by 11 cases in `test/openai-compat-multimodal.test.ts`. **v0.28.7 (#680):** module-scoped `_embedTransport` defaulting to AI SDK `embedMany`, with `__setEmbedTransportForTests(fn)` test seam so tests drive the public `embed()` function with a stubbed transport instead of probing private helpers. `splitByTokenBudget` and `isTokenLimitError` are now exported `@internal` — pure functions reused directly by the test file. Module-level `_shrinkState: Map` halves the recipe's effective `safety_factor` on token-limit miss (floor 0.05) and heals back ×1.5 toward the ceiling after `SHRINK_HEAL_AFTER=10` consecutive successes. `configureGateway()` walks every registered recipe at construction time and emits a once-per-process stderr warning for any embedding touchpoint missing `max_batch_tokens` (excluding the canonical OpenAI fast-path recipe). `resetGateway()` clears `_shrinkState`, the warned-set, and restores the real transport. ASCII flow diagram embedded in the `embed()` JSDoc covers the routing decision, recursion + halving, and shrinkState lifecycle. **v0.28.11 (#719):** `embedMultimodal()` reads `cfg.embedding_multimodal_model` first (falls back to `cfg.embedding_model` for single-model setups). After the existing recipe-level `supports_multimodal` fast-fail, validates the resolved model against `touchpoint.multimodal_models` when declared — closes the Voyage-text-only-model-into-multimodal-endpoint footgun before any HTTP call (Codex F1 from PR review). New `getMultimodalModel()` accessor mirrors `getEmbeddingModel` / `getChatModel` so doctor and integration tests can read the gateway state. **v0.33.1.1 (#962, Codex P3 follow-up):** new exported `VoyageResponseTooLargeError` tagged class at the top of the file. `voyageCompatFetch`'s two OOM-defense caps (Layer 1 Content-Length check at `:595`, Layer 2 per-embedding base64 cap at `:619`) now throw `VoyageResponseTooLargeError` instead of a generic `Error`. The inbound response-rewriter's surrounding try/catch (which intentionally swallows parse failures so misshaped Voyage responses fall through to the SDK's JSON parser) checks `instanceof VoyageResponseTooLargeError` and rethrows. Pre-fix, the Layer 2 throw was silently swallowed and the oversized response returned to the AI SDK anyway — Layer 2 was theatrical. Source-shape regression assertion in `test/voyage-response-cap.test.ts` pins the `instanceof ⇒ throw err` line. - `src/core/ai/recipes/zeroentropyai.ts` (v0.35.0.0) — ZeroEntropy openai-compatible recipe declaring BOTH `embedding` (`zembed-1`, 7 Matryoshka dims: 2560/1280/640/320/160/80/40) AND `reranker` (`zerank-2` flagship + `zerank-1` + `zerank-1-small`, 5MB payload cap) touchpoints. `implementation: 'openai-compatible'` (NOT the misspelled `'openai-compat'` the original plan draft had — pinned by F1 regression in `test/ai/zeroentropy-recipe.test.ts`). `base_url_default: 'https://api.zeroentropy.dev/v1'` already ends with `/v1`, so the `zeroEntropyCompatFetch` URL rewrite `/embeddings → /models/embed` produces `…/v1/models/embed` (NOT `…/v1/v1/…` — pinned by F2 regression). `chars_per_token: 1` + `safety_factor: 0.5` match Voyage's dense-content hedge. +- `src/core/ai/recipes/llama-server-reranker.ts` (v0.40.6.1) — sibling of `llama-server` (the embedding recipe) for llama.cpp running in `--reranking` mode. Distinct recipe rather than dual-touchpoint extension because `--reranking` and `--embeddings` are mutually exclusive at server-launch time, so the two backends need independent base URLs (default 8081 here vs 8080 there). Declares `reranker` touchpoint with `models: []` (user-provided id matching the `--alias` the user launched llama-server with), `path: '/rerank'` (leaf-only; consumes the new `RerankerTouchpoint.path` override on the touchpoint; gateway concatenates with `base_url_default` which already ends in `/v1`, producing `…/v1/rerank` — original draft of this recipe set `path: '/v1/rerank'` which doubled the prefix, caught by post-impl `/codex review` and corrected before merge), `default_timeout_ms: 30_000` (consumed by `src/core/search/mode.ts`'s reranker timeout resolution chain — gives CPU-only first-call warmup headroom; the 5s mode-bundle default would fail-open silently as `timeout`), `cost_per_1m_tokens_usd: 0` (recognized by `FREE_LOCAL_RERANK_PROVIDERS` in `src/core/budget/budget-tracker.ts` so `--max-cost` callers don't TX2 hard-fail on local rerank). Setup hint emphasizes `--alias` because llama-server's `/v1/models` defaults model id to the gguf file path without it, which makes provider:model config strings ugly. Covers Qwen3-Reranker via llama.cpp AND self-hosted ZE weights via llama.cpp — same recipe, different `--model` at launch (quality parity with ZE-hosted NOT guaranteed; user pins their own eval). Pinned by `test/ai/recipe-llama-server-reranker.test.ts`. Voyage / Cohere / vLLM rerankers stay out of scope — different wire shapes need adapter hooks designed in a follow-up plan. Same wave (v0.40.6.1) adds: `path?: string` + `default_timeout_ms?: number` on `RerankerTouchpoint` in `src/core/ai/types.ts`; consumed by the URL build at `src/core/ai/gateway.ts:rerank()` and by mode-resolution at `src/core/search/mode.ts:resolveSearchMode` (precedence: per-call > config-key > recipe touchpoint default > mode bundle); `LLAMA_SERVER_RERANKER_BASE_URL` env passthrough in `src/cli.ts:buildGatewayConfig`; `FREE_LOCAL_RERANK_PROVIDERS` set in `src/core/budget/budget-tracker.ts:lookupPricing` (rerank-kind-only zero-pricing for the local provider prefix); doctor-fix at `src/commands/models.ts:probeRerankerConfig` to read `search.reranker.model` via `loadSearchModeConfig` + `resolveSearchMode` (closes file-plane / DB-plane divergence where doctor said "not configured" while live search was actively reranking — pre-fix bug was `getRerankerModel()` reading `GBrainConfig.reranker_model`, a file-plane field nothing writes); `probeRerankerReachability` upgraded to read the recipe's `default_timeout_ms` so CPU-only cold-start doesn't false-fail. - `src/core/ai/recipes/openrouter.ts` (v0.37.6.0, #1210) — OpenRouter openai-compatible recipe: single key, many providers via `openrouter:/` strings. `base_url_default: 'https://openrouter.ai/api/v1'`. Embedding touchpoint: `openai/text-embedding-3-small` at 1536 dims with Matryoshka `dims_options: [512, 768, 1024, 1536]` (native breakpoints from Weaviate's MRL analysis); `max_batch_tokens: 300_000` = OpenAI's aggregate-per-request token cap (NOT per-input — Codex caught the semantic in pre-merge review). Chat touchpoint declares 8 curated entry points (gpt-5.2, gpt-5.2-chat, gpt-5.5, claude-haiku-4.5, claude-sonnet-4.6, claude-opus-4.7, gemini-3-flash-preview, deepseek-chat) but openai-compat tier accepts any model ID; deliberately no `max_context_tokens` because OR's catalog spans 128K to 1M+. `supports_subagent_loop: false` is INFORMATIONAL — the real gate is `isAnthropicProvider()` in `src/core/model-config.ts` which hard-pins gbrain's subagent infra to Anthropic-direct. The recipe declares `resolveDefaultHeaders(env)` (the env-templated variant of the new `default_headers` seam) returning OR's three attribution headers: `HTTP-Referer` (required for OR to create an app-attribution entry), `X-OpenRouter-Title` (current preferred name per OR docs), `X-Title` (documented back-compat alias). Defaults to `https://gbrain.ai` / `gbrain`; forks (downstream agent deployments) override via `OPENROUTER_REFERER` / `OPENROUTER_TITLE` env vars so their traffic gets their attribution on OR's leaderboard. Smoke-tested by `test/ai/recipe-openrouter.test.ts` (11 cases including the D5 shape-test regression guard: every model in the chat list matches `^[a-z0-9-]+\/[a-z0-9._-]+$`, catching typos and malformed IDs without pinning the catalog's churn). - `src/core/rerank-audit.ts` (v0.35.0.0) — failure-only JSONL audit at `~/.gbrain/audit/rerank-failures-YYYY-Www.jsonl` (ISO-week rotation, mirrors `src/core/audit-slug-fallback.ts`). Exports `logRerankFailure({reason, model, query_hash, doc_count, error_summary})` + `readRecentRerankFailures(days)`. **Deliberately no `logRerankSuccess`** (CDX2-F22 in plan review): writing once per tokenmax search is hot-path I/O churn AND success events leak query volume + timing into a local audit file. `gbrain doctor`'s `reranker_health` check reads `search.reranker.enabled` first so "no events in window" is interpreted correctly (disabled → ok; enabled → ok). Query text is SHA-256-prefix-hashed (8 hex chars) for privacy. `GBRAIN_AUDIT_DIR` env override honored via the shared `resolveAuditDir()`. - `src/core/search/embedding-column.ts` (v0.36.3.0) — single source of truth for "which `content_chunks.*` column does this query rank against?" Pure functions, no engine I/O: `loadRegistry(cfg)` walks the `embedding_columns` config (DB plane, JSON map keyed by column name with `{provider, dimensions, type}` entries), seeds the OpenAI `embedding` builtin when unset, validates everything before it lands (column-name regex, type ∈ `vector | halfvec`, dims in [1, 8192], provider format) using `Object.create(null)` + `Object.hasOwn` so a key like `constructor` rejects instead of resolving to `Object.prototype.constructor`. `resolveColumn(registry, override?, cfg)` is the boundary call: returns a `ResolvedColumn` descriptor (frozen `{name, provider, dimensions, type}`) honoring per-call override → `search_embedding_column` config → `'embedding'` default. Throws `UnknownEmbeddingColumnError` with the list of registered names on miss. `isCacheSafe(resolved, cfg)` compares the full embedding SPACE (provider + dimensions + name) against cfg's default — a user who repointed the `embedding` builtin at Voyage doesn't accidentally serve OpenAI-shaped cache rows. `validateResolvedColumn(descriptor)` re-validates hand-rolled descriptors that bypass the registry (internal-SDK passthrough path) so the SQL-injection escape hatch through the descriptor field is closed. Consumed by `hybridSearch` (resolves once at the boundary, threads a `ResolvedColumn` into per-engine `searchVector` instead of a raw string), `gateway.embedQuery(text, {embeddingModel, dimensions})` (resolved column's provider drives the query-time embed call), `cosineReScore` (engine pulls vectors from the active column, not the hardcoded `embedding`), and the `query` MCP op (per-call `embedding_column` param). 538 lines of pure resolver, 511 unit cases in `test/search/embedding-column.test.ts` covering the 16 codex-flagged corners (prototype-pollution, descriptor passthrough, env-only Postgres install, empty-brain coverage gate, cache-space comparison). @@ -390,7 +391,8 @@ strict behavior when unset. - `src/core/import-checkpoint.ts` (v0.34.2.0) — `loadCheckpoint(brainDir)`, `saveCheckpoint(brainDir, completed)`, `resumeFilter(files, completed, brainDir)`, `clearCheckpoint()`, plus the `ImportCheckpoint` type. Path-set checkpoint format (`{schema_version, brainDir, completed: string[]}`) replaces the v0.33.x positional `{processedIndex: N}` format. Atomic write via `.tmp` + `rename()` so a mid-write crash never leaves a partial JSON. `loadCheckpoint` returns `null` on: missing file, malformed JSON, brainDir mismatch (you ran import against a different brain), and the old positional format (logged to stderr before being discarded). `resumeFilter` returns `{toProcess, skippedCount}` — pure, no I/O, deterministic. `clearCheckpoint` is no-op-on-missing for clean-exit cleanup. Honors `GBRAIN_HOME` via `gbrainPath()` so test isolation via `withEnv({GBRAIN_HOME: tmpdir})` works without monkey-patching the fs layer. Best-effort persistence — `saveCheckpoint` logs warnings on write errors but never throws, so import keeps making progress even if disk is full. - `src/core/sort-newest-first.ts` (v0.34.2.0) — single source of truth for the descending-lex sort that `gbrain import` and `gbrain sync` both apply. Mutates in place (Array.prototype.sort semantics), returns the same array reference for fluent chaining. Empty/single-element inputs short-circuit. Future ordering changes flip one line in this helper instead of touching two CLI commands. Pinned by `test/sort-newest-first.test.ts` (5 hermetic cases: descending order, mixed prefixes, empty input, single-element input, in-place-mutation contract). - `src/core/cycle.ts` — v0.17 brain maintenance cycle primitive (extended to **9 phases in v0.29**). `runCycle(engine: BrainEngine | null, opts: CycleOpts): Promise` composes phases in semantically-driven order: **lint → backlinks → sync → synthesize → extract → patterns → recompute_emotional_weight → embed → orphans**. v0.29 adds the `recompute_emotional_weight` phase between patterns and embed; it sees the union of `syncPagesAffected` + `synthesizeWrittenSlugs` for incremental mode, or all pages when neither anchor is set (full backfill via `gbrain dream --phase recompute_emotional_weight`). v0.29 also extends `CycleReport.totals` with `pages_emotional_weight_recomputed` (additive, schema_version stays "1"). v0.23's `synthesize` phase runs after sync (cross-references see fresh brain) and before extract (auto-link materializes its writes); `patterns` runs after extract so it reads a fresh graph (codex finding #7 — subagent put_page sets `ctx.remote=true` and skips auto-link/timeline by default; extract is the canonical materialization). Three callers: `gbrain dream` CLI, `gbrain autopilot` daemon's inline path, and the Minions `autopilot-cycle` handler. Coordination via `gbrain_cycle_locks` DB table + `~/.gbrain/cycle.lock` file lock with PID-liveness for PGLite. `CycleReport.schema_version: "1"` is stable; totals additively grew in v0.23 (`transcripts_processed`, `synth_pages_written`, `patterns_written`). `yieldBetweenPhases` runs between phases. **v0.23 added `yieldDuringPhase`** for in-phase keepalive — synthesize/patterns call it during long waits to renew the cycle-lock TTL. Engine nullable; lock-skip on read-only phase selections. v0.22.1 (#403): `CycleOpts.signal?: AbortSignal` propagates the worker's abort signal; `checkAborted()` fires between every phase. v0.22.1 (#417): `runPhaseSync` returns `pagesAffected` via `SyncPhaseResult`; `runCycle` captures it and threads to `runPhaseExtract` as the 4th arg. v0.22.1 (Codex F2): `runPhaseSync` takes `willRunExtractPhase: boolean` and sets `noExtract: phases.includes('extract')` so `gbrain dream --phase sync` doesn't silently lose extraction. v0.22.5 (#475): `resolveSourceForDir(engine, brainDir)` threads `sourceId` to `performSync()` so sync reads the per-source `sources.last_commit` anchor instead of the drift-prone global `config.sync.last_commit` key. -- `src/core/cycle/synthesize.ts` (v0.23) — Synthesize phase: conversation-transcript-to-brain pipeline. Reads from `dream.synthesize.session_corpus_dir`, runs cheap Haiku verdict (cached in `dream_verdicts`), then fans out one Sonnet subagent per worth-processing transcript with `allowed_slug_prefixes` (sourced from `skills/_brain-filing-rules.json` `dream_synthesize_paths.globs`). Orchestrator collects slugs from `subagent_tool_executions` (NOT `pages.updated_at` — codex finding #2) and reverse-renders DB → markdown via `serializeMarkdown`. Cooldown via `dream.synthesize.last_completion_ts`, written ONLY on success. Idempotency key `dream:synth::`. Auto-commit deferred to v1.1 (codex #5). `--dry-run` runs Haiku, skips Sonnet (codex #8). Subagent never gets fs-write access. **v0.23.2:** `renderPageToMarkdown` (now exported) stamps `dream_generated: true` and `dream_cycle_date` into every reverse-write's frontmatter; `writeSummaryPage` does the same on the dream-cycle summary index. The marker is the explicit identity surface checked by `isDreamOutput` in `transcript-discovery.ts` — replaces the v0.23.1 content-prefix heuristic that could miss real output (`serializeMarkdown` doesn't embed slugs in body) and false-positive on user transcripts citing brain pages. `judgeSignificance` and `JudgeClient` are exported; `judgeSignificance` accepts a `verdictModel` parameter (default `claude-haiku-4-5-20251001`) loaded from `dream.synthesize.verdict_model` via `loadSynthConfig`. **v0.30.2:** model-aware chunker `splitTranscriptByBudget(content, contentHash, maxChars)` splits oversized transcripts at paragraph boundaries (`## Topic:` → `---` → `\n` ladder) using a deterministic offset seeded from the first 32 bits of `contentHash` so retries chunk identically. Per-chunk char budget computed from `MODEL_CONTEXT_TOKENS[resolvedModel] × 0.9 × 3.5 chars/token`; non-Anthropic ids fall back to a 180K-token safe default with a once-per-process stderr warning. Operator overrides: `dream.synthesize.max_prompt_tokens` (floor 100K, wins when set) and `dream.synthesize.max_chunks_per_transcript` (default 24). Per-chunk idempotency keys `dream:synth:::cof`; single-chunk transcripts preserve the legacy `dream:synth::` key byte-for-byte (D8 lookup), so existing brains skip with `already_synthesized_legacy_single_chunk` instead of re-spending Sonnet on upgrade. `collectChildPutPageSlugs` raw-fetches every (job_id, slug) pair (not `SELECT DISTINCT`) and rewrites bare-hash6 slugs to `-c` for chunked children (D6 — orchestrator-side, zero Sonnet trust). Cap-hit skips don't write to `dream_verdicts`, so raising the cap on next run re-attempts cleanly. D7 scope: bounds INITIAL prompt size only; tool-loop turn-N accumulation is caught by the v0.30.2 terminal-error classification in `subagent.ts`, not bounded ahead of time. +- `src/core/cycle/synthesize.ts` (v0.23) — Synthesize phase: conversation-transcript-to-brain pipeline. Reads from `dream.synthesize.session_corpus_dir`, runs cheap Haiku verdict (cached in `dream_verdicts`), then fans out one Sonnet subagent per worth-processing transcript with `allowed_slug_prefixes` (sourced from `skills/_brain-filing-rules.json` `dream_synthesize_paths.globs`). Orchestrator collects slugs from `subagent_tool_executions` (NOT `pages.updated_at` — codex finding #2) and reverse-renders DB → markdown via `serializeMarkdown`. Cooldown via `dream.synthesize.last_completion_ts`, written ONLY on success. Idempotency key `dream:synth::`. Auto-commit deferred to v1.1 (codex #5). `--dry-run` runs Haiku, skips Sonnet (codex #8). Subagent never gets fs-write access. **v0.23.2:** `renderPageToMarkdown` (now exported) stamps `dream_generated: true` and `dream_cycle_date` into every reverse-write's frontmatter; `writeSummaryPage` does the same on the dream-cycle summary index. The marker is the explicit identity surface checked by `isDreamOutput` in `transcript-discovery.ts` — replaces the v0.23.1 content-prefix heuristic that could miss real output (`serializeMarkdown` doesn't embed slugs in body) and false-positive on user transcripts citing brain pages. `judgeSignificance` and `JudgeClient` are exported; `judgeSignificance` accepts a `verdictModel` parameter (default `claude-haiku-4-5-20251001`) loaded from `dream.synthesize.verdict_model` via `loadSynthConfig`. **v0.30.2:** model-aware chunker `splitTranscriptByBudget(content, contentHash, maxChars)` splits oversized transcripts at paragraph boundaries (`## Topic:` → `---` → `\n` ladder) using a deterministic offset seeded from the first 32 bits of `contentHash` so retries chunk identically. Per-chunk char budget computed from `MODEL_CONTEXT_TOKENS[resolvedModel] × 0.9 × 3.5 chars/token`; non-Anthropic ids fall back to a 180K-token safe default with a once-per-process stderr warning. Operator overrides: `dream.synthesize.max_prompt_tokens` (floor 100K, wins when set) and `dream.synthesize.max_chunks_per_transcript` (default 24). Per-chunk idempotency keys `dream:synth:::cof`; single-chunk transcripts preserve the legacy `dream:synth::` key byte-for-byte (D8 lookup), so existing brains skip with `already_synthesized_legacy_single_chunk` instead of re-spending Sonnet on upgrade. `collectChildPutPageSlugs` raw-fetches every (job_id, slug) pair (not `SELECT DISTINCT`) and rewrites bare-hash6 slugs to `-c` for chunked children (D6 — orchestrator-side, zero Sonnet trust). Cap-hit skips don't write to `dream_verdicts`, so raising the cap on next run re-attempts cleanly. D7 scope: bounds INITIAL prompt size only; tool-loop turn-N accumulation is caught by the v0.30.2 terminal-error classification in `subagent.ts`, not bounded ahead of time. **v0.41+ community-PR-wave (reworked PR #1349):** `makeHaikuClient()` replaced by gateway-routed `makeJudgeClient(verdictModel)` exported helper — mirrors `tryBuildGatewayClient` in `src/core/think/index.ts:579-637` (the v0.35.5.0 #952 pattern). Construction-time provider/key probe returns `null` on a clear miss (unknown provider id via `resolveRecipe` AIConfigError, or Anthropic provider with no key via `hasAnthropicKey()` matching the same env + config-file pattern think uses). Verdict loop wraps `judgeSignificance` in try/catch for `AIConfigError` so mid-run provider failures (revoked key, recipe misconfig) surface as per-transcript `worth=false, reasons=['gateway error: ...']` instead of crashing the phase. Canonical config key `models.dream.synthesize_verdict` (per `PER_TASK_KEYS` in `src/core/model-config.ts`); contributor's parallel `DEEPSEEK_API_KEY` fetch adapter dropped in favor of recipe-registered providers (DeepSeek, OpenRouter, Voyage, Ollama, llama-server, ...). `JudgeClient` interface signature preserved verbatim for test-seam stability. CI guard at `scripts/check-gateway-routed-no-direct-anthropic.sh` prevents future contributors from re-introducing `new Anthropic()` here or in `think/index.ts`. Pinned by 13 cases in `test/cycle/synthesize-gateway-adapter.test.ts` (A1-A9 unit + R3 parsed-verdict semantic parity + R3 corollary) and 1 E2E case in `test/e2e/dream-synthesize-pglite.test.ts` ('gateway-adapter mid-run AIConfigError catch') plus 4 R1-R4 critical regression pins in `test/cycle/regression-pr-wave-r1-r2-r4.test.ts`. +- `scripts/check-gateway-routed-no-direct-anthropic.sh` (v0.41+ community-PR-wave) — CI guard that fails the build if `src/core/cycle/synthesize.ts` or `src/core/think/index.ts` reintroduces a runtime `new Anthropic()` constructor call or a value-shaped `import Anthropic from '@anthropic-ai/sdk'` import. Type-only imports (`import type Anthropic from '@anthropic-ai/sdk'`) stay allowed for adapter types. Comment lines (`//` or ` *` prefixes) are excluded so historical JSDoc references don't false-fire. Mirrors `scripts/check-jsonb-pattern.sh`. Wired into `bun run verify` and `bun run check:all`. Extend `GUARDED_FILES` in the script when migrating another file off direct SDK construction. - `src/core/cycle/patterns.ts` (v0.23) — Patterns phase: cross-session theme detection over reflections within `dream.patterns.lookback_days` (default 30). Names a pattern only when ≥`dream.patterns.min_evidence` (default 3) reflections support it. Single Sonnet subagent; same allow-list path as synthesize. Runs AFTER `extract` so the graph is fresh. - `src/core/cycle/extract-facts.ts` (v0.32.2, extended v0.35.6.0) — extract_facts cycle phase. v0.32.2 contract: fence is canonical; per-page wipe (`deleteFactsForPage`) + reinsert from `parseFactsFence` + `extractFactsFromFenceText` + `engine.insertFacts`. Empty-fence guard refuses when v0.31 legacy rows (`row_num IS NULL AND entity_slug IS NOT NULL`) pend the v0_32_2 backfill (status: warn, hint: `gbrain apply-migrations --yes`). **v0.35.6.0** adds a phantom-redirect pre-pass that runs AFTER the legacy-row guard, BEFORE the main reconcile loop. When `opts.brainDir` is set, `runPhantomRedirectPass(engine, brainDir, sourceId, dryRun)` walks unprefixed-slug pages capped by `GBRAIN_PHANTOM_REDIRECT_LIMIT` (default 50). The pass returns `touched_canonicals` — canonical slugs whose disk fence was merged with phantom rows; `runExtractFacts` UNIONs them into the main reconcile slug set so canonical's DB facts derive from the merged fence in the same cycle (round-14 scenario-B fix: phantom had only-on-disk fence, no DB facts). `ExtractFactsResult` gains six phantom fields: `phantomsScanned`, `phantomsRedirected`, `phantomsAmbiguous`, `phantomsSkippedDrift`, `phantomsLockBusy`, `phantomsMorePending`. Three of those bubble to `CycleReport.totals` (`phantoms_redirected`, `phantoms_ambiguous`, `phantoms_skipped_drift`). - `src/core/entities/resolve.ts` (v0.30+, extended v0.35.6.0) — Free-form entity name → canonical slug resolution. `resolveEntitySlug(engine, source_id, raw)`: exact slug → fuzzy (pg_trgm @ 0.4 threshold) → bare-name prefix expansion (`people/-%` then `companies/-%` using correlated-subquery `connection_count` for tiebreaker) → deterministic `slugify` fallback. **v0.35.6.0** exports two new helpers for the phantom-redirect pass: `resolvePhantomCanonical(engine, sourceId, phantomSlug)` — variant that SKIPS the exact-slug step (codex #1: phantom slug `'alice'` exact-matches itself, would make the redirect handler a no-op); returns the canonical only when result is non-null AND contains `/`. `findPrefixCandidates(engine, sourceId, token)` — standalone SQL query returning ALL candidates across `PREFIX_EXPANSION_DIRS` (currently hardcoded `['people', 'companies']`) using `slug LIKE ANY($N::text[])` over patterns `dir/token` + `dir/token-%`; cap of 10 ordered by `connection_count DESC, slug ASC`. NOT a wrapper around `tryPrefixExpansion` because that path returns per-dir top-1 and suppresses ambiguity by design (codex #11). Pinned by `test/phantom-redirect.test.ts` resolvePhantomCanonical describe (3 cases) + findPrefixCandidates describe (6 cases including multi-dir ambiguity and the `people/aliceberg`-doesn't-match-`alice` false-positive guard). @@ -2773,6 +2775,7 @@ Data flowing into the brain. Each integration is a recipe — markdown + setup h - **Voice**: Phone calls create brain pages via Twilio + OpenAI Realtime (or DIY STT+LLM+TTS). Setup recipe: [`recipes/twilio-voice-brain.md`](recipes/twilio-voice-brain.md). - **Email + calendar**: webhook handlers that route to brain signals. [`docs/integrations/meeting-webhooks.md`](docs/integrations/meeting-webhooks.md). - **Embedding providers**: 16 recipes covering OpenAI (default fallback), OpenRouter, Voyage, ZeroEntropy (default), Google Gemini, Azure OpenAI, MiniMax, Alibaba DashScope, Zhipu, Ollama (local), llama.cpp llama-server (local), LiteLLM proxy. Pricing matrix + decision tree in [`docs/integrations/embedding-providers.md`](docs/integrations/embedding-providers.md). +- **Rerankers**: ZeroEntropy `zerank-2` hosted (default in `tokenmax` mode) plus the v0.40.6.1 `llama-server-reranker` recipe for fully-local cross-encoder rerank via llama.cpp — runs Qwen3-Reranker or self-hosted ZeroEntropy weights against the same `gateway.rerank()` seam. Setup walkthrough in [`docs/ai-providers/llama-server-reranker.md`](docs/ai-providers/llama-server-reranker.md). - **Credential gateway**: vault-aware secret distribution. [`docs/integrations/credential-gateway.md`](docs/integrations/credential-gateway.md). - **MCP clients**: every major MCP client is supported. [`docs/mcp/`](docs/mcp/) per-client setup. @@ -4629,6 +4632,8 @@ built-in server is the recommended path. --- +# AI providers + # Debugging ## docs/GBRAIN_VERIFY.md diff --git a/llms.txt b/llms.txt index 993ee9a46..b904a0ad6 100644 --- a/llms.txt +++ b/llms.txt @@ -24,6 +24,11 @@ Repo: https://github.com/garrytan/gbrain - [docs/guides/quiet-hours.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/guides/quiet-hours.md): Notification hold + timezone-aware delivery. - [docs/mcp/DEPLOY.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/mcp/DEPLOY.md): MCP server deployment. +## AI providers + +- [docs/ai-providers/zeroentropy.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/ai-providers/zeroentropy.md): ZeroEntropy zembed-1 embedding + zerank-2 reranker (hosted): API key, embedding switch, reranker config. +- [docs/ai-providers/llama-server-reranker.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/ai-providers/llama-server-reranker.md): Local reranker via llama.cpp --reranking: Qwen3-Reranker or self-hosted ZE weights, --alias setup, gbrain config keys, cold-start timeout, budget-cap interaction. + ## Debugging - [docs/GBRAIN_VERIFY.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/GBRAIN_VERIFY.md): 7-check post-setup verification. Start here when something feels off. diff --git a/package.json b/package.json index 92fa3466c..b1542dfcb 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gbrain", - "version": "0.41.3.0", + "version": "0.41.4.0", "description": "Postgres-native personal knowledge brain with hybrid RAG search", "type": "module", "main": "src/core/index.ts", @@ -40,14 +40,15 @@ "build:pglite-snapshot": "bun run scripts/build-pglite-snapshot.ts", "test": "bash scripts/run-unit-parallel.sh", "test:full": "bun run verify && bash scripts/run-unit-parallel.sh && bun run test:slow && ([ -n \"$DATABASE_URL\" ] && bash scripts/run-e2e.sh || echo '[test:full] skipped E2E (no DATABASE_URL); run docker-compose -f docker-compose.ci.yml up + bun run test:e2e to include' 1>&2)", - "verify": "bun run check:privacy && bun run check:proposal-pii && bun run check:test-names && bun run check:jsonb && bun run check:source-id-projection && bun run check:source-config-leak && bun run check:progress && bun run check:test-isolation && bun run check:wasm && bun run check:admin-build && bun run check:admin-scope-drift && bun run check:cli-exec && bun run check:system-of-record && bun run check:eval-glossary && bun run check:no-pii-agent-voice && bun run check:synthetic-corpus-privacy && bun run check:skill-brain-first && bun run check:fuzz-purity && bun run check:operations-filter-bypass && bun run typecheck", + "verify": "bun run check:privacy && bun run check:proposal-pii && bun run check:test-names && bun run check:jsonb && bun run check:source-id-projection && bun run check:source-config-leak && bun run check:progress && bun run check:test-isolation && bun run check:wasm && bun run check:admin-build && bun run check:admin-scope-drift && bun run check:cli-exec && bun run check:system-of-record && bun run check:eval-glossary && bun run check:no-pii-agent-voice && bun run check:synthetic-corpus-privacy && bun run check:skill-brain-first && bun run check:fuzz-purity && bun run check:operations-filter-bypass && bun run check:gateway-routed && bun run typecheck", "check:source-config-leak": "scripts/check-source-config-leak.sh", "check:no-pii-agent-voice": "scripts/check-no-pii-in-agent-voice.sh", "check:synthetic-corpus-privacy": "scripts/check-synthetic-corpus-privacy.sh", "check:system-of-record": "scripts/check-system-of-record.sh", "check:admin-scope-drift": "scripts/check-admin-scope-drift.sh", "check:cli-exec": "scripts/check-cli-executable.sh", - "check:all": "scripts/check-privacy.sh && scripts/check-proposal-pii.sh && scripts/check-test-real-names.sh && scripts/check-jsonb-pattern.sh && scripts/check-source-id-projection.sh && scripts/check-source-config-leak.sh && scripts/check-progress-to-stdout.sh && scripts/check-no-legacy-getconnection.sh && scripts/check-test-isolation.sh && scripts/check-trailing-newline.sh && scripts/check-wasm-embedded.sh && scripts/check-exports-count.sh && scripts/check-admin-build.sh && scripts/check-admin-scope-drift.sh && scripts/check-cli-executable.sh && scripts/check-skill-brain-first.sh && scripts/check-operations-filter-bypass.sh", + "check:all": "scripts/check-privacy.sh && scripts/check-proposal-pii.sh && scripts/check-test-real-names.sh && scripts/check-jsonb-pattern.sh && scripts/check-source-id-projection.sh && scripts/check-source-config-leak.sh && scripts/check-progress-to-stdout.sh && scripts/check-no-legacy-getconnection.sh && scripts/check-test-isolation.sh && scripts/check-trailing-newline.sh && scripts/check-wasm-embedded.sh && scripts/check-exports-count.sh && scripts/check-admin-build.sh && scripts/check-admin-scope-drift.sh && scripts/check-cli-executable.sh && scripts/check-skill-brain-first.sh && scripts/check-operations-filter-bypass.sh && scripts/check-gateway-routed-no-direct-anthropic.sh", + "check:gateway-routed": "scripts/check-gateway-routed-no-direct-anthropic.sh", "check:skill-brain-first": "scripts/check-skill-brain-first.sh", "check:wasm": "scripts/check-wasm-embedded.sh", "check:newlines": "scripts/check-trailing-newline.sh", diff --git a/scripts/check-gateway-routed-no-direct-anthropic.sh b/scripts/check-gateway-routed-no-direct-anthropic.sh new file mode 100755 index 000000000..4801fddec --- /dev/null +++ b/scripts/check-gateway-routed-no-direct-anthropic.sh @@ -0,0 +1,133 @@ +#!/usr/bin/env bash +# CI guard: fail if gateway-routed source files reintroduce direct Anthropic +# SDK instantiation (`new Anthropic()` / `import Anthropic from '@anthropic-ai/sdk'` +# as a runtime constructor, NOT a type-only import). +# +# Why this exists: v0.35.5.0 migrated src/core/think/index.ts from `new Anthropic()` +# to a gateway.chat() adapter (closed #952). v0.41+ wave did the same for +# src/core/cycle/synthesize.ts (T5 in the community PR wave). Both files +# now route through src/core/ai/gateway.ts so any provider with a registered +# recipe (Anthropic, DeepSeek, OpenRouter, Voyage, Ollama, llama-server, ...) +# is reachable via `models.dream.synthesize_verdict` / chat model config. +# +# Without this guard, a future contributor adding `import Anthropic from +# '@anthropic-ai/sdk'` and `new Anthropic()` to either file silently re-opens +# the same provider-lock-in bug class. The symptom is "my DeepSeek config +# isn't being used by dream synthesize" — invisible until first user report. +# +# Mirrors the pattern of scripts/check-jsonb-pattern.sh. +# +# Usage: scripts/check-gateway-routed-no-direct-anthropic.sh +# Exit: 0 when clean, 1 when a guarded file imports the SDK as a runtime value. + +set -euo pipefail + +ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)" +cd "$ROOT" + +# Files whose contract is "ALL chat calls route through gateway.chat()". +# Extend this list when migrating another file off direct SDK construction. +GUARDED_FILES=( + "src/core/cycle/synthesize.ts" + "src/core/think/index.ts" +) + +FAILED=0 + +for f in "${GUARDED_FILES[@]}"; do + if [ ! -f "$f" ]; then + # File was renamed or removed. Don't fail loud — flag and continue. + echo "WARN: guarded file missing: $f (rename/remove? update GUARDED_FILES in $(basename "$0"))" + continue + fi + + # Match `new Anthropic(...)` — the runtime constructor call. Both `new Anthropic()` + # and `new Anthropic({apiKey: '...'})` shapes are caught. + # Exclude single-line `//` and block `*` comment lines so historical references + # in JSDoc / explanatory comments don't false-fire. Code AND code-in-template + # literals still hit (those don't start with `//` or ` *`). + if grep -En 'new\s+Anthropic\s*\(' "$f" 2>/dev/null | grep -vE '^[0-9]+:\s*(//|\*)' | grep .; then + echo + echo "ERROR: $f reintroduced direct Anthropic SDK construction (\`new Anthropic()\`)." + echo " This file's contract is to route all chat calls through gateway.chat()." + echo " Use the adapter pattern from src/core/think/index.ts:tryBuildGatewayClient" + echo " or src/core/cycle/synthesize.ts:makeJudgeClient." + FAILED=1 + fi + + # Match any value-shaped (NOT type-only) import of the SDK. The type-only forms + # `import type Anthropic from '@anthropic-ai/sdk'` AND + # `import { type Foo } from '@anthropic-ai/sdk'` (all-type-clauses-only) are allowed + # for typing the adapter's Anthropic.Message return shape. Covers: + # import Anthropic from '@anthropic-ai/sdk' (default) + # import { Anthropic } from '@anthropic-ai/sdk' (named) + # import Anthropic, { Other } from '@anthropic-ai/sdk' (default + named) + # import { Anthropic as A } from '@anthropic-ai/sdk' (named-renamed) + # import { type Msg, Anthropic } from '@anthropic-ai/sdk' (mixed type + value) + # import * as Anthropic from '@anthropic-ai/sdk' (namespace) + # Strategy: catch every line ending in `from '@anthropic-ai/sdk'`, strip + # comment lines, strip top-level `import type ...` (the only allowed shape), + # then check whether any remaining line contains a value identifier OUTSIDE + # type-prefixed clauses. We handle the mixed-import case by inspecting the + # specifier list: if any specifier is not `type Foo`, the import is value-shaped. + while IFS= read -r line; do + [ -z "$line" ] && continue + # Strip the leading "line-number:" prefix grep adds. + body="${line#*:}" + # Top-level `import type ...` is allowed — entire import is type-only. + if printf '%s' "$body" | grep -qE '^\s*import\s+type\s'; then continue; fi + # Comment line. + if printf '%s' "$body" | grep -qE '^\s*(//|\*)'; then continue; fi + # Extract the specifiers list (between `import` and `from`), if present. + # If the list contains any specifier NOT prefixed with `type ` (or there's + # no brace list at all — default/namespace import), it's a value import. + # POSIX character classes for cross-shell portability (macOS BSD sed + # doesn't support `\s` even in extended-regex mode). + specifiers=$(printf '%s' "$body" | sed -nE 's/^[[:space:]]*import[[:space:]]+\{([^}]*)\}[[:space:]]+from.*/\1/p') + if [ -n "$specifiers" ]; then + # Brace list present. Allow only if EVERY non-empty specifier is type-prefixed. + # Use a temp file instead of `while | exit 1` (subshell trap). + tmpflag=$(mktemp -t gateway-guard-XXXX) + echo 0 > "$tmpflag" + printf '%s' "$specifiers" | tr ',' '\n' | while IFS= read -r spec; do + spec=$(printf '%s' "$spec" | sed -E 's/^[[:space:]]+|[[:space:]]+$//g') + [ -z "$spec" ] && continue + if ! printf '%s' "$spec" | grep -qE '^type\s'; then echo 1 > "$tmpflag"; fi + done + has_value=$(cat "$tmpflag") + rm -f "$tmpflag" + if [ "$has_value" = "1" ]; then + echo "$line" + echo + echo "ERROR: $f imports @anthropic-ai/sdk with a value-shaped specifier." + echo " Use \`import type ...\` for all clauses, or route runtime" + echo " chat calls through src/core/ai/gateway.ts." + FAILED=1 + fi + else + # No brace list — default, namespace, or bare import — always value-shaped. + echo "$line" + echo + echo "ERROR: $f imports @anthropic-ai/sdk as a runtime value." + echo " Use \`import type Anthropic from '@anthropic-ai/sdk'\` for type-only" + echo " references to Anthropic.Message / Anthropic.MessageCreateParamsNonStreaming." + echo " Route runtime chat calls through src/core/ai/gateway.ts." + FAILED=1 + fi + done < <(grep -En "from\s+['\"]@anthropic-ai/sdk['\"]" "$f" 2>/dev/null) + # Dynamic import — also a value-shaped reference. + if grep -En "import\s*\(\s*['\"]@anthropic-ai/sdk['\"]" "$f" 2>/dev/null \ + | grep -vE '^[0-9]+:\s*(//|\*)' | grep .; then + echo + echo "ERROR: $f dynamically imports @anthropic-ai/sdk." + echo " Route runtime chat calls through src/core/ai/gateway.ts." + FAILED=1 + fi +done + +if [ "$FAILED" -eq 1 ]; then + exit 1 +fi + +echo "OK: gateway-routed files have no direct Anthropic SDK construction" +echo " (guarded: ${GUARDED_FILES[*]})" diff --git a/scripts/llms-config.ts b/scripts/llms-config.ts index 45c37da6c..de8553f3d 100644 --- a/scripts/llms-config.ts +++ b/scripts/llms-config.ts @@ -126,6 +126,27 @@ export const SECTIONS: DocSection[] = [ }, ], }, + { + heading: "AI providers", + entries: [ + { + title: "docs/ai-providers/zeroentropy.md", + description: + "ZeroEntropy zembed-1 embedding + zerank-2 reranker (hosted): API key, embedding switch, reranker config.", + path: "docs/ai-providers/zeroentropy.md", + // Setup walkthrough — discoverable in the index, not inlined in the + // single-fetch bundle (keeps llms-full.txt under FULL_SIZE_BUDGET). + includeInFull: false, + }, + { + title: "docs/ai-providers/llama-server-reranker.md", + description: + "Local reranker via llama.cpp --reranking: Qwen3-Reranker or self-hosted ZE weights, --alias setup, gbrain config keys, cold-start timeout, budget-cap interaction.", + path: "docs/ai-providers/llama-server-reranker.md", + includeInFull: false, + }, + ], + }, { heading: "Debugging", entries: [ diff --git a/src/cli.ts b/src/cli.ts index 8dcd86b77..f53099b91 100755 --- a/src/cli.ts +++ b/src/cli.ts @@ -508,7 +508,7 @@ export function parseOpArgs(op: Operation, args: string[]): Record MAX_STDIN) { console.error(`Error: stdin content exceeds ${MAX_STDIN} bytes. Split into smaller inputs.`); @@ -1503,6 +1503,11 @@ export function buildGatewayConfig(c: GBrainConfig): AIGatewayConfig { // OLLAMA_BASE_URL. Caller-provided cfg.provider_base_urls wins. const envBaseUrls: Record = {}; if (process.env.LLAMA_SERVER_BASE_URL) envBaseUrls['llama-server'] = process.env.LLAMA_SERVER_BASE_URL; + // v0.40.6.1: sibling recipe for llama-server in reranking mode. Separate + // env var because --reranking and --embeddings are mutually exclusive at + // server launch — users running both will have two llama-server processes + // on different ports. + if (process.env.LLAMA_SERVER_RERANKER_BASE_URL) envBaseUrls['llama-server-reranker'] = process.env.LLAMA_SERVER_RERANKER_BASE_URL; if (process.env.OLLAMA_BASE_URL) envBaseUrls['ollama'] = process.env.OLLAMA_BASE_URL; if (process.env.LMSTUDIO_BASE_URL) envBaseUrls['lmstudio'] = process.env.LMSTUDIO_BASE_URL; if (process.env.LITELLM_BASE_URL) envBaseUrls['litellm'] = process.env.LITELLM_BASE_URL; diff --git a/src/commands/autopilot.ts b/src/commands/autopilot.ts index 0c2c2b66e..c6c21def2 100644 --- a/src/commands/autopilot.ts +++ b/src/commands/autopilot.ts @@ -427,7 +427,7 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) { // poll-only deployments. try { const { MinionQueue } = await import('../core/minions/queue.ts'); - const { computeRecommendations } = await import('../core/brain-score-recommendations.ts'); + const { computeRecommendations, embeddingProviderConfigured, HOSTED_EMBED_KEY_CONFIG } = await import('../core/brain-score-recommendations.ts'); const queue = new MinionQueue(engine); const slotMs = Math.floor(Date.now() / (baseInterval * 1000)) * baseInterval * 1000; const slot = new Date(slotMs).toISOString(); @@ -487,9 +487,28 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) { // Cheap path: engine.getHealth() is a single SQL count query. const health = await engine.getHealth(); const score = health.brain_score; + // v0.40.x: recipe-aware embedding-provider check shared with doctor.ts. + // Resolve the configured model (gateway → DB fallback), then pre-await + // the handful of hosted-key config values so the resolveKey closure + // passed to embeddingProviderConfigured() can stay synchronous. + let embeddingModel: string | undefined; + try { + const gw = await import('../core/ai/gateway.ts'); + embeddingModel = gw.getEmbeddingModel(); + } catch { + embeddingModel = (await engine.getConfig('embedding_model')) ?? undefined; + } + const embedKeyCfg: Record = {}; + for (const field of Object.values(HOSTED_EMBED_KEY_CONFIG)) { + embedKeyCfg[field] = await engine.getConfig(field); + } const ctx = { repoPath, - hasEmbeddingApiKey: !!(process.env.OPENAI_API_KEY || await engine.getConfig('openai_api_key')), + embeddingModel, + embeddingProviderConfigured: embeddingProviderConfigured(embeddingModel, (envVar) => { + const cfgField = HOSTED_EMBED_KEY_CONFIG[envVar]; + return !!(process.env[envVar] || (cfgField ? embedKeyCfg[cfgField] : undefined)); + }), hasChatApiKey: !!(process.env.ANTHROPIC_API_KEY || await engine.getConfig('anthropic_api_key')), }; const plan = computeRecommendations(health, ctx).filter((r) => r.status === 'remediable'); diff --git a/src/commands/doctor.ts b/src/commands/doctor.ts index f723fbd6b..73d0e5996 100644 --- a/src/commands/doctor.ts +++ b/src/commands/doctor.ts @@ -5528,31 +5528,27 @@ async function loadRecommendationContext(engine: BrainEngine) { embeddingModel = dbModel ?? undefined; embeddingDimensions = dbDims ? Number(dbDims) : undefined; } - // Provider-aware key check. The active embedding provider determines - // which key matters. Pre-fix this was OpenAI-only, so a ZE brain with - // OPENAI_API_KEY set looked "healthy" even though no key reached ZE. + // v0.40.x: recipe-aware provider check, shared with autopilot.ts via + // embeddingProviderConfigured(). Local providers (ollama, llama-server — + // empty auth_env.required) need no hosted key; hosted providers check + // their OWN required key (so a Voyage brain is judged by VOYAGE_API_KEY, + // not by whether an OpenAI/ZE key happens to exist — the pre-fix wart). + // fileCfg loads synchronously, so the resolveKey closure is sync. const { loadConfigFileOnly } = await import('../core/config.ts'); const fileCfg = loadConfigFileOnly(); - let hasEmbeddingApiKey = false; - if (embeddingModel?.startsWith('openai:')) { - hasEmbeddingApiKey = !!(process.env.OPENAI_API_KEY || fileCfg?.openai_api_key); - } else if (embeddingModel?.startsWith('zeroentropyai:')) { - hasEmbeddingApiKey = !!(process.env.ZEROENTROPY_API_KEY || fileCfg?.zeroentropy_api_key); - } else { - // Voyage / generic openai-compatible / unknown provider — fall back - // to "any key present" as the legacy hint. - hasEmbeddingApiKey = !!( - process.env.OPENAI_API_KEY || - process.env.ZEROENTROPY_API_KEY || - fileCfg?.openai_api_key || - fileCfg?.zeroentropy_api_key - ); - } + const { embeddingProviderConfigured, HOSTED_EMBED_KEY_CONFIG } = await import( + '../core/brain-score-recommendations.ts' + ); + const embeddingConfigured = embeddingProviderConfigured(embeddingModel, (envVar) => { + const cfgField = HOSTED_EMBED_KEY_CONFIG[envVar]; + const fromCfg = cfgField ? (fileCfg as Record | null)?.[cfgField] : undefined; + return !!(process.env[envVar] || fromCfg); + }); return { repoPath: repoPath ?? undefined, embeddingModel, embeddingDimensions, - hasEmbeddingApiKey, + embeddingProviderConfigured: embeddingConfigured, hasChatApiKey: !!(process.env.ANTHROPIC_API_KEY || fileCfg?.anthropic_api_key), }; } diff --git a/src/commands/models.ts b/src/commands/models.ts index ca5a447e7..8b444370b 100644 --- a/src/commands/models.ts +++ b/src/commands/models.ts @@ -151,7 +151,7 @@ function formatText(report: ModelsReport): string { } } lines.push(''); - lines.push('Tip: probe reachability with `gbrain models doctor` (opt-in; spends ~1 token per model).'); + lines.push('Tip: probe reachability with `gbrain models doctor` (opt-in; spends a minimal request per configured chat/embed/rerank surface).'); return lines.join('\n'); } @@ -161,7 +161,7 @@ type ProbeStatus = 'ok' | 'model_not_found' | 'auth' | 'rate_limit' | 'network' interface ProbeResult { model: string; - touchpoint: 'chat' | 'expansion' | 'embedding_config' | 'reranker_config'; + touchpoint: 'chat' | 'expansion' | 'embedding_config' | 'embedding_reachability' | 'reranker_config'; status: ProbeStatus; message: string; elapsed_ms: number; @@ -269,6 +269,54 @@ async function probeEmbeddingConfig(): Promise { } } +/** + * v0.40.6.1: resolve the reranker model the same way live search does, so + * doctor doesn't drift from the live path. Pre-v0.40.6.1 the probe read + * `getRerankerModel()` from the gateway, which is fed from + * `GBrainConfig.reranker_model` — a file-plane field nothing currently + * writes. Meanwhile live search resolves `search.reranker.model` via + * `resolveSearchMode()` (per-call > config-key > recipe > bundle default). + * The two paths could disagree silently: doctor says "not configured" + * while every `gbrain search` call is using a mode default. This helper + * walks the same chain live search does so doctor's verdict matches. + * + * Falls back to `getRerankerModel()` (gateway value) when the engine path + * fails, so doctor stays useful in degraded states. + */ +export async function resolveLiveRerankerModel(engine: BrainEngine): Promise { + try { + const { loadSearchModeConfig, resolveSearchMode } = await import('../core/search/mode.ts'); + const input = await loadSearchModeConfig(engine); + const resolved = resolveSearchMode(input); + return resolved.reranker_enabled ? resolved.reranker_model : undefined; + } catch { + const { getRerankerModel } = await import('../core/ai/gateway.ts'); + return getRerankerModel(); + } +} + +/** + * Resolve the reranker timeout the same way live search does, via + * `loadSearchModeConfig` + `resolveSearchMode`. Precedence chain: + * per-call > `search.reranker.timeout_ms` config > recipe `default_timeout_ms` > mode bundle. + * + * Codex outside-voice (Pass 9 of the wave) caught the probe lying either way + * when the operator sets `search.reranker.timeout_ms`: the probe used the + * recipe default (30s for llama) while production search used the (lower) + * config value, so doctor reported reachable while production always + * timed out. Same fix shape as `resolveLiveRerankerModel`. + */ +export async function resolveLiveRerankerTimeoutMs(engine: BrainEngine): Promise { + try { + const { loadSearchModeConfig, resolveSearchMode } = await import('../core/search/mode.ts'); + const input = await loadSearchModeConfig(engine); + const resolved = resolveSearchMode(input); + return resolved.reranker_timeout_ms ?? 5000; + } catch { + return 5000; + } +} + /** * v0.35.0.0+: zero-network reranker config probe. Validates that the * configured reranker model resolves through the recipe registry, that the @@ -280,16 +328,19 @@ async function probeEmbeddingConfig(): Promise { * this, `search.reranker.model=zeroentropyai:made-up-name` would silently * pass config probes and fail at first rerank call. * + * v0.40.6.1: resolves via `resolveLiveRerankerModel(engine)` so probe and + * live search read the same value (closes the file-plane / DB-plane + * divergence flagged in plan review). + * * Returns 'ok' when reranker is unconfigured (default state — opt-in * feature). Surfaces `status: 'config'` with paste-ready fix hint when * model is invalid. */ -async function probeRerankerConfig(): Promise { +async function probeRerankerConfig(engine: BrainEngine): Promise { const start = Date.now(); - const { getRerankerModel } = await import('../core/ai/gateway.ts'); const { resolveRecipe } = await import('../core/ai/model-resolver.ts'); - const modelStr = getRerankerModel(); + const modelStr = await resolveLiveRerankerModel(engine); if (!modelStr) { // Reranker not configured. Default state for fresh installs and any // brain that hasn't opted in. Not an error; doctor reports 'ok' so the @@ -298,7 +349,7 @@ async function probeRerankerConfig(): Promise { model: '(none)', touchpoint: 'reranker_config', status: 'ok', - message: 'reranker not configured (set GBRAIN_RERANKER_MODEL or `gbrain config set search.reranker.enabled true`)', + message: 'reranker not configured (set `gbrain config set search.reranker.model ` and `search.reranker.enabled true`)', elapsed_ms: Date.now() - start, }; } @@ -346,29 +397,45 @@ async function probeRerankerConfig(): Promise { } /** - * v0.35.0.0+: 1-token-equivalent reranker reachability probe. Sends a minimal - * `{query, documents: [doc]}` request to verify auth + URL. Uses the same - * AbortController + 5s timeout pattern as probeModel. + * v0.35.0.0+: 1-doc reachability probe. Sends a real `POST ` + * with `{query, documents: [doc]}` so the probe actually verifies the + * server is in reranking mode (not just alive). For llama.cpp specifically, + * `--reranking` is mutually exclusive with `--embeddings`, and a server in + * embedding mode would 404/501 the rerank path — which this probe catches + * via classifyError(). * * Returns 'ok' silently when reranker is unconfigured (no probe needed) — * probeRerankerConfig already surfaced the missing-config state. + * + * v0.40.6.1: uses the resolved live model (same path live search uses), + * and reads the per-call timeout from the recipe's `default_timeout_ms` + * when set — so a CPU-only local reranker's cold-start warmup doesn't + * cause the probe to false-fail with `network`/timeout. */ -async function probeRerankerReachability(): Promise { - const { getRerankerModel } = await import('../core/ai/gateway.ts'); - const modelStr = getRerankerModel(); +async function probeRerankerReachability(engine: BrainEngine): Promise { + const modelStr = await resolveLiveRerankerModel(engine); if (!modelStr) return null; + // Use the same timeout resolution live search uses: per-call > config > + // recipe > bundle. Pre-fix the probe read only the recipe default, so an + // operator who set `search.reranker.timeout_ms=1000` would see doctor wait + // 30s and report reachable while production search timed out at 1s + // (codex Pass-9 finding). resolveLiveRerankerTimeoutMs reuses the full + // precedence chain via mode.ts. + const probeTimeoutMs = await resolveLiveRerankerTimeoutMs(engine); + const start = Date.now(); try { const { rerank } = await import('../core/ai/gateway.ts'); const controller = new AbortController(); - const timeoutId = setTimeout(() => controller.abort(new Error('probe timed out after 5s')), 5000); + const timeoutId = setTimeout(() => controller.abort(new Error(`probe timed out after ${probeTimeoutMs}ms`)), probeTimeoutMs); try { await rerank({ + model: modelStr, query: 'probe', documents: ['probe document'], signal: controller.signal, - timeoutMs: 5000, + timeoutMs: probeTimeoutMs, }); return { model: modelStr, @@ -392,6 +459,50 @@ async function probeRerankerReachability(): Promise { } } +/** + * v0.40.x: embedding reachability probe. Mirrors probeRerankerReachability — + * sends a real 1-input `embed(['probe'])` to verify the configured embedding + * provider actually answers (auth + URL + model loaded). probeEmbeddingConfig + * is zero-network and only validates dims/recipe shape; for LOCAL providers + * (ollama, llama-server) it can't tell whether the server is up, so a dead or + * embedding-mode-off endpoint was previously only discovered at first real + * embed. Caller gates this on probeEmbeddingConfig returning 'ok' so a config + * failure isn't reported twice. + * + * Cold-start note: a local CPU embedder loading a model on first call can take + * several seconds; the 5s timeout may trip on the very first probe. Re-run if so. + */ +async function probeEmbeddingReachability(): Promise { + const { getEmbeddingModel, embed } = await import('../core/ai/gateway.ts'); + const modelStr = getEmbeddingModel(); + if (!modelStr) return null; + + const start = Date.now(); + const controller = new AbortController(); + const timeoutId = setTimeout(() => controller.abort(new Error('probe timed out after 5s')), 5000); + try { + await embed(['probe'], { inputType: 'query', abortSignal: controller.signal }); + return { + model: modelStr, + touchpoint: 'embedding_reachability', + status: 'ok', + message: 'reachable', + elapsed_ms: Date.now() - start, + }; + } catch (err) { + const { status, message } = classifyError(err); + return { + model: modelStr, + touchpoint: 'embedding_reachability', + status, + message, + elapsed_ms: Date.now() - start, + }; + } finally { + clearTimeout(timeoutId); + } +} + async function probeModel(modelStr: string, touchpoint: 'chat' | 'expansion'): Promise { const start = Date.now(); try { @@ -472,9 +583,12 @@ Tiers: utility (haiku-class) | reasoning (sonnet) | deep (opus) | subagent (Anth // Config-only probe runs first: zero tokens, catches the bug class where a // brain misconfigured for Voyage with the wrong embedding_dimensions would // 400 on first embed. Fast feedback before we spend a single token. - results.push(await probeEmbeddingConfig()); + const embeddingConfig = await probeEmbeddingConfig(); + results.push(embeddingConfig); // v0.35.0.0+ reranker config probe — same zero-network model as embedding. - results.push(await probeRerankerConfig()); + // v0.40.6.1: takes the engine so it can read the same `search.reranker.*` + // config keys live search reads (closes file-plane / DB-plane divergence). + results.push(await probeRerankerConfig(engine)); for (const [modelStr, touchpoint] of [[chatModel, 'chat'], [expansionModel, 'expansion']] as const) { if (shouldSkipProvider(modelStr, skip)) { @@ -484,11 +598,20 @@ Tiers: utility (haiku-class) | reasoning (sonnet) | deep (opus) | subagent (Anth results.push(await probeModel(modelStr, touchpoint)); } - // v0.35.0.0+: reranker reachability (only when configured + provider not in --skip). - const { getRerankerModel } = await import('../core/ai/gateway.ts'); - const rerankerModel = getRerankerModel(); - if (rerankerModel && !shouldSkipProvider(rerankerModel, skip)) { - const r = await probeRerankerReachability(); + // v0.40.x: embedding reachability — only when the config probe passed + // (codex #8: a config failure shouldn't be reported twice) AND the provider + // isn't in --skip. Catches a dead/misconfigured LOCAL embed server early. + if (embeddingConfig.status === 'ok' && !shouldSkipProvider(embeddingConfig.model, skip)) { + const er = await probeEmbeddingReachability(); + if (er) results.push(er); + } + + // v0.40.6.1: reranker reachability uses the live-search resolution path + // (file-plane / DB-plane divergence fix); only fires when reranker is + // actually enabled per the resolved mode bundle. + const liveRerankerModel = await resolveLiveRerankerModel(engine); + if (liveRerankerModel && !shouldSkipProvider(liveRerankerModel, skip)) { + const r = await probeRerankerReachability(engine); if (r) results.push(r); } diff --git a/src/commands/providers.ts b/src/commands/providers.ts index 6d2fc211b..5977009b7 100644 --- a/src/commands/providers.ts +++ b/src/commands/providers.ts @@ -60,8 +60,16 @@ export function envReady(recipe: Recipe, env: NodeJS.ProcessEnv = process.env): */ export function formatRecipeTable(recipes: Recipe[], env: NodeJS.ProcessEnv = process.env): string { const rows: string[] = []; - rows.push('PROVIDER'.padEnd(14) + 'TIER'.padEnd(18) + 'EMBED'.padEnd(8) + 'EXPAND'.padEnd(8) + 'CHAT'.padEnd(8) + 'STATUS'); - rows.push('-'.repeat(78)); + // Dynamic column width: longest recipe id + 1 space, floor at 14 (the + // historical default). v0.40.6.1 introduced `llama-server-reranker` (21 chars) + // which overflowed the static 14-char column and made the row start with the + // tier name (no space delimiter), breaking `each recipe appears at most once` + // in test/providers.test.ts. Auto-widening keeps the contract — every row's + // id is followed by at least one space — without per-recipe column tuning. + const idCol = Math.max(14, ...recipes.map(r => r.id.length + 1)); + const totalWidth = idCol + 18 + 8 + 8 + 8 + 16; // tier+embed+expand+chat+status + rows.push('PROVIDER'.padEnd(idCol) + 'TIER'.padEnd(18) + 'EMBED'.padEnd(8) + 'EXPAND'.padEnd(8) + 'CHAT'.padEnd(8) + 'STATUS'); + rows.push('-'.repeat(totalWidth)); for (const r of recipes) { const hasEmbed = !!r.touchpoints.embedding && (r.touchpoints.embedding.models.length > 0); const hasExpand = !!r.touchpoints.expansion; @@ -69,7 +77,7 @@ export function formatRecipeTable(recipes: Recipe[], env: NodeJS.ProcessEnv = pr const ready = envReady(r, env); const status = ready ? '✓ ready' : `✗ missing ${r.auth_env?.required?.[0] ?? 'setup'}`; rows.push( - r.id.padEnd(14) + + r.id.padEnd(idCol) + r.tier.padEnd(18) + (hasEmbed ? 'yes' : '—').padEnd(8) + (hasExpand ? 'yes' : '—').padEnd(8) + diff --git a/src/core/ai/gateway.ts b/src/core/ai/gateway.ts index 05ac7a7b3..e7df81a8b 100644 --- a/src/core/ai/gateway.ts +++ b/src/core/ai/gateway.ts @@ -2767,7 +2767,12 @@ export async function rerank(input: RerankInput): Promise { // Resolve base URL + auth from the recipe (same path Voyage/ZE embeddings use). const cfg = requireConfig(); const compat = applyOpenAICompatConfig(recipe, cfg); - const url = `${compat.baseURL.replace(/\/$/, '')}/models/rerank`; + // v0.40.6.1: rerank URL path is recipe-pluggable. Defaults to ZeroEntropy's + // legacy `/models/rerank`; openai-style providers like llama.cpp's + // llama-server set `/v1/rerank`. Wire shape is unchanged — any provider + // whose request/response shape differs from ZE/llama.cpp (e.g. Voyage with + // `top_k` / `data[]`) needs separate adapter hooks in a follow-up plan. + const url = `${compat.baseURL.replace(/\/$/, '')}${tp.path ?? '/models/rerank'}`; const auth = applyResolveAuth(recipe, cfg, 'reranker'); // applyResolveAuth returns { apiKey } for Bearer-style auth (SDK's native // path) or { headers } for custom-header providers (Azure). v0.37.6.0: diff --git a/src/core/ai/recipes/index.ts b/src/core/ai/recipes/index.ts index 244f00e83..383350fff 100644 --- a/src/core/ai/recipes/index.ts +++ b/src/core/ai/recipes/index.ts @@ -22,6 +22,7 @@ import { dashscope } from './dashscope.ts'; import { zhipu } from './zhipu.ts'; import { azureOpenAI } from './azure-openai.ts'; import { zeroentropyai } from './zeroentropyai.ts'; +import { llamaServerReranker } from './llama-server-reranker.ts'; const ALL: Recipe[] = [ openai, @@ -35,6 +36,7 @@ const ALL: Recipe[] = [ groq, together, llamaServer, + llamaServerReranker, minimax, dashscope, zhipu, diff --git a/src/core/ai/recipes/llama-server-reranker.ts b/src/core/ai/recipes/llama-server-reranker.ts new file mode 100644 index 000000000..2f6e958b4 --- /dev/null +++ b/src/core/ai/recipes/llama-server-reranker.ts @@ -0,0 +1,84 @@ +import type { Recipe } from '../types.ts'; + +/** + * llama.cpp's `llama-server --reranking` exposes a cross-encoder reranker + * over an OpenAI-style HTTP surface. Distinct from the sibling `llama-server` + * recipe (which serves embeddings) because `--reranking` and `--embeddings` + * are mutually exclusive at server-launch time — one process can't do both, + * so two recipes with independent base URLs is the cleanest topology. + * + * Wire shape matches ZeroEntropy: request is `{model, query, documents, + * top_n?}`, response is `{results: [{index, relevance_score}]}`. Path is + * the only delta — llama-server serves `/rerank` under its `/v1` prefix. + * Because the recipe's `base_url_default` already ends in `/v1` (matching + * the convention every other openai-compat recipe uses), the touchpoint + * `path` here is the LEAF only (`/rerank`); the gateway concatenates + * `${base_url}${path}` to produce the actual `…/v1/rerank` URL. + * + * Like the embedding recipe, this ships with `models: []` because the model + * identity is whatever the user launched llama-server with. Users MUST set + * `search.reranker.model llama-server-reranker:` where `` matches + * the `--alias` they passed at launch — without `--alias`, `/v1/models` + * defaults the id to the gguf file path, which makes provider:model strings + * ugly. The setup_hint guides them. + * + * Reference: + * https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md + * + * Covers two user-facing cases: + * - Qwen3-Reranker (0.6B / 4B / 8B) GGUF via llama.cpp + * - ZeroEntropy zerank-2 / zerank-1-small self-hosted via llama.cpp + * (FEASIBLE — wire shapes match — but quality parity with ZE-hosted is + * NOT guaranteed; users self-hosting ZE should pin their own eval) + */ +export const llamaServerReranker: Recipe = { + id: 'llama-server-reranker', + name: 'llama.cpp llama-server (reranker, local)', + tier: 'openai-compat', + implementation: 'openai-compatible', + // Distinct default port from the embedding recipe (8080) so a user + // running both locally can keep them on separate servers. + base_url_default: 'http://localhost:8081/v1', + auth_env: { + required: [], + optional: ['LLAMA_SERVER_RERANKER_BASE_URL', 'LLAMA_SERVER_RERANKER_API_KEY'], + setup_url: + 'https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md', + }, + touchpoints: { + reranker: { + models: [], // user-provided; whatever model the server was launched with + // Informational placeholder for docs/wizard copy. Real model id is set + // by the user via `gbrain config set search.reranker.model + // llama-server-reranker:<--alias value>`. + default_model: 'qwen3-reranker-4b', + // Local inference cost — consumed by budget-tracker.ts's rerank + // pricing lookup (via FREE_LOCAL_RERANK_PROVIDERS) so callers with + // `--max-cost` don't hard-fail. NOT for API billing; local rerank + // costs electricity, not tokens. + cost_per_1m_tokens_usd: 0, + price_last_verified: '2026-05-23', + // Match ZE's per-request cap; llama.cpp has no upstream cap of its + // own but the pre-flight guard is a defensive ceiling. + max_payload_bytes: 5_000_000, + // Leaf-only path. `base_url_default` already provides the `/v1` + // prefix; the gateway concatenates the two to call `…/v1/rerank`. + // llama-server also serves `/reranking`, `/v1/reranking`, and bare + // `/rerank` aliases — we pin the OpenAI-style `/rerank` path under + // the existing `/v1` prefix. + path: '/rerank', + // CPU-only first-call warmup on a 4B cross-encoder can take 8-15s. + // The default 5s in gateway.ts:DEFAULT_RERANK_TIMEOUT_MS would + // fail-open silently. Caller's `input.timeoutMs` and the + // `search.reranker.timeout_ms` config key still win when set. + default_timeout_ms: 30_000, + }, + }, + setup_hint: + 'Build llama.cpp, then `llama-server --model --alias ' + + ' --reranking --port 8081`. The --alias makes provider:model ' + + 'strings short (without it, /v1/models defaults the id to the gguf file ' + + 'path). Then `gbrain config set search.reranker.model ' + + 'llama-server-reranker:` and `gbrain config set ' + + 'provider_base_urls.llama-server-reranker http://:8081/v1`.', +}; diff --git a/src/core/ai/types.ts b/src/core/ai/types.ts index bb8fe76b4..79e4122e9 100644 --- a/src/core/ai/types.ts +++ b/src/core/ai/types.ts @@ -186,6 +186,20 @@ export interface RerankerTouchpoint { cost_per_1m_tokens_usd?: number; price_last_verified?: string; max_payload_bytes: number; + /** + * Override the rerank URL path. Defaults to '/models/rerank' (ZeroEntropy's + * legacy path; ZE-compatible-wire-shape providers like llama.cpp set + * '/v1/rerank'). + */ + path?: string; + /** + * Recipe-level timeout fallback for `gateway.rerank()` and search-mode + * resolution. Caller's `input.timeoutMs` and `search.reranker.timeout_ms` + * config still win when set. Used to give CPU-only local rerankers (e.g. + * llama.cpp serving Qwen3-Reranker-4B) headroom for first-call warmup + * without forcing every user to discover the config key. + */ + default_timeout_ms?: number; } export interface ChatTouchpoint { diff --git a/src/core/brain-score-recommendations.ts b/src/core/brain-score-recommendations.ts index ba64a7312..8103e7c76 100644 --- a/src/core/brain-score-recommendations.ts +++ b/src/core/brain-score-recommendations.ts @@ -2,6 +2,67 @@ import { createHash } from 'crypto'; import type { BrainHealth } from './types.ts'; import { ANTHROPIC_PRICING } from './anthropic-pricing.ts'; import { lookupEmbeddingPrice, estimateCostFromChars } from './embedding-pricing.ts'; +import { getRecipe } from './ai/recipes/index.ts'; +import { parseModelId } from './ai/model-resolver.ts'; + +/** + * v0.40.x: env-var name → file/DB config field, for hosted embedding providers + * whose config-plane key is actually propagated to the AI gateway. Producers of + * RecommendationContext (doctor + autopilot) use this to build a sync + * `resolveKey` closure without re-parsing recipes. + * + * Only OPENAI_API_KEY and ZEROENTROPY_API_KEY appear here because those are the + * only embedding keys `buildGatewayConfig` (src/cli.ts) folds from config into + * the gateway env. VOYAGE_API_KEY / GOOGLE_GENERATIVE_AI_API_KEY are deliberately + * absent: their config fields are NOT threaded to the gateway today, so the + * producer closures fall through to checking `process.env` ONLY for them. That + * matches what the gateway can actually use (the recipes read those keys from + * env). Counting a config-plane voyage_api_key/google_api_key here would be a + * false positive: doctor/autopilot would call the provider "configured" and + * dispatch an embed.stale job that then fails auth at the gateway. When a future + * change threads voyage_api_key/google_api_key into buildGatewayConfig (the open + * voyage-config-mapping work), re-add the matching entry here in the same change. + */ +export const HOSTED_EMBED_KEY_CONFIG: Record = { + OPENAI_API_KEY: 'openai_api_key', + ZEROENTROPY_API_KEY: 'zeroentropy_api_key', +}; + +/** + * v0.40.x: is the configured embedding provider usable for the remediation + * planner? Recipe-aware: + * - empty `auth_env.required` (ollama, llama-server, ...) ⇒ local, no hosted + * key needed ⇒ true. + * - hosted (openai, zeroentropyai, voyage, google, ...) ⇒ true iff every + * required key resolves. + * + * `resolveKey(envVar)` is supplied by the caller so each producer reads config + * from its own source (doctor → file plane; autopilot → engine.getConfig). + * Only the recipe logic is shared, not the config lookup. + * + * NOTE: deliberately NOT the same as `gateway.isAvailable('embedding')`. + * isAvailable returns false for user_provided_models recipes (llama-server, + * models: []) because it can't validate the model id. For a remediation + * verdict we WANT true there — local embeddings work. Do not "align" them. + * Uses the recipe registry (pure data), not the gateway runtime, so this + * module stays free of AI-SDK coupling and works before engine.connect(). + */ +export function embeddingProviderConfigured( + embeddingModel: string | undefined, + resolveKey: (envVar: string) => boolean, +): boolean { + if (!embeddingModel) return false; + let providerId: string; + try { + ({ providerId } = parseModelId(embeddingModel)); + } catch { + return false; // malformed model id — mirror gateway.isAvailable's catch + } + const recipe = getRecipe(providerId); + if (!recipe?.touchpoints?.embedding) return false; + const required = recipe.auth_env?.required ?? []; + return required.length === 0 ? true : required.every(resolveKey); +} /** Minimal Check shape consumed by classifyChecks. Subset of doctor.ts's * Check; we intentionally don't import from doctor.ts (would create a @@ -74,8 +135,13 @@ export interface RecommendationContext { embeddingModel?: string; /** Configured embedding dimension (3072 / 1536 / 1024 / etc.). */ embeddingDimensions?: number; - /** Whether the embedding provider has a usable API key. */ - hasEmbeddingApiKey?: boolean; + /** + * Whether the configured embedding provider is usable. For hosted providers + * this means the required API key resolves; for local providers (ollama, + * llama-server — empty auth_env.required) it's true once configured, no key + * needed. Compute via `embeddingProviderConfigured()`. + */ + embeddingProviderConfigured?: boolean; /** Configured chat / synthesis model id. */ chatModel?: string; /** Whether the chat provider has a usable API key. */ @@ -130,7 +196,7 @@ export function computeRecommendations( // --------------------------------------------------------------------- // embed.stale — missing embeddings. Critical: invisible to vector search // --------------------------------------------------------------------- - if (health.missing_embeddings > 0 && ctx.hasEmbeddingApiKey !== false) { + if (health.missing_embeddings > 0 && ctx.embeddingProviderConfigured !== false) { const params = { stale: true, sourceId: ctx.sourceId }; const embedModel = ctx.embeddingModel ?? 'openai:text-embedding-3-large'; const embedDims = ctx.embeddingDimensions ?? 3072; @@ -242,8 +308,8 @@ function classifyOne(check: Check, ctx: RecommendationContext): CheckClassificat } return { check: check.name, status: 'remediable' }; case 'missing_embeddings': - if (ctx.hasEmbeddingApiKey === false) { - return { check: check.name, status: 'blocked', reason: 'missing embedding API key' }; + if (ctx.embeddingProviderConfigured === false) { + return { check: check.name, status: 'blocked', reason: 'embedding provider not configured' }; } return { check: check.name, status: 'remediable' }; case 'dead_links': diff --git a/src/core/budget/budget-tracker.ts b/src/core/budget/budget-tracker.ts index 929351f6a..d6b5e7961 100644 --- a/src/core/budget/budget-tracker.ts +++ b/src/core/budget/budget-tracker.ts @@ -123,6 +123,37 @@ function defaultAuditPath(): string { return `${dir}/${isoWeekFilename('budget')}`; } +/** + * Provider id prefixes that always price at $0 for the rerank kind + * (electricity, not API tokens). Centralized here so `--max-cost` callers + * don't hard-fail TX2 when a local rerank provider is configured. Matched + * against the provider half of the `provider:model` string. Extend this set + * when adding new local-inference rerank recipes. + */ +const FREE_LOCAL_RERANK_PROVIDERS: ReadonlySet = new Set([ + 'llama-server-reranker', +]); + +/** + * Provider id prefixes whose embeddings run on local inference (electricity, + * not API tokens) and so price at $0. Without this, a `--max-cost`-bounded + * embed/reindex job configured for a local provider TX2 hard-fails because + * lookupEmbeddingPrice has no entry for them. Matched against the provider + * half of the `provider:model` string. + * + * 'lmstudio' is intentionally excluded — no lmstudio recipe is registered, so + * `lmstudio:` model strings never resolve (the env mapping in cli.ts is + * pre-existing dead plumbing). 'litellm' is excluded too — a LiteLLM proxy can + * front a paid provider, so pricing-unknown is the honest state there. + * + * Sibling to FREE_LOCAL_RERANK_PROVIDERS; v0.41+ TODO unifies them via + * recipe-cost-driven resolution. + */ +const FREE_LOCAL_EMBED_PROVIDERS: ReadonlySet = new Set([ + 'ollama', + 'llama-server', +]); + /** * Look up `modelId` in the chat or embedding pricing maps. Returns a * per-1M-token price tuple, or null when unknown. @@ -130,10 +161,13 @@ function defaultAuditPath(): string { * Strategy: * - Chat: try the bare model id in ANTHROPIC_PRICING first (legacy keys * are bare claude-* ids). Fall back to the provider-prefixed key. - * - Embed: lookupEmbeddingPrice already handles the provider:model form, - * defaulting to openai when the colon is missing. - * - Rerank: not priced today — treat as a chat call with no output cost - * when caller passes ANTHROPIC_PRICING-shaped id, else unknown. + * - Embed: lookupEmbeddingPrice handles the provider:model form; on a miss, + * local-inference providers (FREE_LOCAL_EMBED_PROVIDERS) price at $0 so + * `--max-cost` callers don't hard-fail. + * - Rerank: try ANTHROPIC_PRICING (legacy path for any Claude-priced + * rerank); else if the provider half is in FREE_LOCAL_RERANK_PROVIDERS, + * return zero pricing so `--max-cost` callers don't TX2 hard-fail on + * local inference recipes (electricity, not tokens); else unknown. */ function lookupPricing(modelId: string, kind: BudgetKind): ModelPricing | null { if (kind === 'embed') { @@ -141,16 +175,27 @@ function lookupPricing(modelId: string, kind: BudgetKind): ModelPricing | null { if (hit.kind === 'known') { return { input: hit.pricePerMTok, output: 0 }; } + // v0.40.x: local-inference embed providers cost electricity, not tokens. + if (hit.kind === 'unknown' && FREE_LOCAL_EMBED_PROVIDERS.has(hit.provider)) { + return { input: 0, output: 0 }; + } return null; } // chat or rerank: try bare key first, then provider:model const bare = ANTHROPIC_PRICING[modelId]; if (bare) return bare; - const [, modelTail] = modelId.includes(':') ? modelId.split(':', 2) : [null, modelId]; + const [providerId, modelTail] = modelId.includes(':') ? modelId.split(':', 2) : [null, modelId]; if (modelTail) { const tailHit = ANTHROPIC_PRICING[modelTail]; if (tailHit) return tailHit; } + // v0.40.6.1: zero-price local-inference rerank providers so the budget + // tracker's TX2 hard-fail doesn't trip on `llama-server-reranker:` + // under `--max-cost`. Only the rerank kind — chat/embed already have + // their own provider-specific pricing surfaces. + if (kind === 'rerank' && providerId && FREE_LOCAL_RERANK_PROVIDERS.has(providerId)) { + return { input: 0, output: 0 }; + } return null; } diff --git a/src/core/cycle/synthesize.ts b/src/core/cycle/synthesize.ts index 9828a4f3a..a90c55d44 100644 --- a/src/core/cycle/synthesize.ts +++ b/src/core/cycle/synthesize.ts @@ -26,8 +26,12 @@ * - Daily token budget cap (cooldown bounds spend at v1 scale). */ -import Anthropic from '@anthropic-ai/sdk'; +import type Anthropic from '@anthropic-ai/sdk'; import { readFileSync, existsSync, writeFileSync, mkdirSync } from 'node:fs'; +import { chat as gatewayChat, type ChatResult } from '../ai/gateway.ts'; +import { resolveRecipe } from '../ai/model-resolver.ts'; +import { AIConfigError } from '../ai/errors.ts'; +import { loadConfig } from '../config.ts'; import { join, dirname } from 'node:path'; import type { BrainEngine } from '../engine.ts'; import type { PhaseResult, PhaseError } from '../cycle.ts'; @@ -296,7 +300,12 @@ export async function runPhaseSynthesize( // Significance verdicts (cached in dream_verdicts; Haiku on miss). const worthProcessing: DiscoveredTranscript[] = []; const verdicts: Array<{ filePath: string; worth: boolean; reasons: string[]; cached: boolean }> = []; - const haiku = makeHaikuClient(); // null if no API key + // Provider-aware judge client routes through gateway.chat, so any + // configured provider works (Anthropic, DeepSeek, OpenRouter, Voyage, + // Ollama, llama-server, etc.). Returns null when the resolved verdict + // model has no reachable provider (legacy "no API key" branch preserved + // as the cheap pre-flight check). + const judge = makeJudgeClient(config.verdictModel); for (const t of transcripts) { const cached = await engine.getDreamVerdict(t.filePath, t.contentHash); if (cached) { @@ -304,15 +313,38 @@ export async function runPhaseSynthesize( if (cached.worth_processing) worthProcessing.push(t); continue; } - if (!haiku) { - // No API key — can't judge. Skip with explicit reason; don't crash phase. - verdicts.push({ filePath: t.filePath, worth: false, reasons: ['no ANTHROPIC_API_KEY for significance judge'], cached: false }); + if (!judge) { + // No configured provider for the verdict model — can't judge. + // Skip with explicit reason; don't crash phase. + verdicts.push({ + filePath: t.filePath, + worth: false, + reasons: [`no configured provider for verdict model: ${config.verdictModel}`], + cached: false, + }); continue; } - const verdict = await judgeSignificance(haiku, t, config.verdictModel); - await engine.putDreamVerdict(t.filePath, t.contentHash, verdict); - verdicts.push({ filePath: t.filePath, worth: verdict.worth_processing, reasons: verdict.reasons, cached: false }); - if (verdict.worth_processing) worthProcessing.push(t); + try { + const verdict = await judgeSignificance(judge, t, config.verdictModel); + await engine.putDreamVerdict(t.filePath, t.contentHash, verdict); + verdicts.push({ filePath: t.filePath, worth: verdict.worth_processing, reasons: verdict.reasons, cached: false }); + if (verdict.worth_processing) worthProcessing.push(t); + } catch (e) { + // AIConfigError at chat time = provider auth/config went bad mid-run + // (revoked key, recipe misconfig surfacing at first real call). Skip + // this transcript with the gateway error message so the user sees the + // shape of the problem in `gbrain dream --phase synthesize --dry-run`. + if (e instanceof AIConfigError) { + verdicts.push({ + filePath: t.filePath, + worth: false, + reasons: [`gateway error: ${e.message}`], + cached: false, + }); + continue; + } + throw e; + } } // Dry-run stops here: significance filter ran (Haiku verdicts cached), @@ -633,16 +665,125 @@ async function loadAllowedSlugPrefixes(): Promise { return []; } -// ── Significance judge (Haiku) ─────────────────────────────────────── +// ── Significance judge (gateway-routed; provider-agnostic) ────────────── +// +// The JudgeClient interface is unchanged for test-seam stability — existing +// tests that pass a mock client to judgeSignificance keep working byte- +// identically. Only the construction path moved from `new Anthropic()` to +// `gateway.chat()` so any provider with a registered recipe (Anthropic, +// DeepSeek, OpenRouter, Voyage, Ollama, llama-server, etc.) is reachable +// via `gbrain config set models.dream.synthesize_verdict :`. +// +// This mirrors v0.35.5.0's `tryBuildGatewayClient` in src/core/think/index.ts +// (which closed #952 for runThink). Same pattern, same trade-offs: +// construction-time provider/key probe returns null on a clear miss (cheap +// pre-flight), and the verdict loop wraps the actual chat call in try/catch +// for AIConfigError surfacing mid-run. export interface JudgeClient { create: (params: Anthropic.MessageCreateParamsNonStreaming) => Promise; } -function makeHaikuClient(): JudgeClient | null { - if (!process.env.ANTHROPIC_API_KEY) return null; - const client = new Anthropic(); - return { create: client.messages.create.bind(client.messages) }; +/** + * Build a gateway-routed JudgeClient for the resolved verdict model. + * Returns null when no chat provider is reachable for `verdictModel`: + * - Unknown provider id (resolveRecipe throws AIConfigError). + * - Anthropic provider with no key (env or config) — preserves the legacy + * "no ANTHROPIC_API_KEY" cheap-skip semantics. + * On null, the verdict loop short-circuits each transcript with an explicit + * "no configured provider" reason and continues the phase. + * + * For non-Anthropic providers (deepseek, openrouter, voyage, ollama, + * llama-server, ...), we delegate auth probing to the gateway's own + * recipe `auth_env.required` machinery — AIConfigError at gateway.chat() + * time is caught by the verdict loop and surfaced per-transcript. + */ +export function makeJudgeClient(verdictModel: string): JudgeClient | null { + // Normalize: ensure provider:model shape. resolveModel returns bare + // anthropic ids (e.g. `claude-haiku-4-5-20251001`); gateway.chat needs + // `anthropic:...`. + const modelStr = verdictModel.includes(':') ? verdictModel : `anthropic:${verdictModel}`; + + // Availability probe: resolveRecipe throws AIConfigError on unknown provider. + let providerId: string; + try { + const { parsed } = resolveRecipe(modelStr); + providerId = parsed.providerId; + } catch (e) { + if (e instanceof AIConfigError) return null; + throw e; + } + + // Anthropic key probe (legacy behavior preserved). Other providers' + // key checks happen lazily at chat call time and surface as + // AIConfigError, which the verdict loop catches per-transcript. + if (providerId === 'anthropic' && !hasAnthropicKey()) return null; + + return { + create: async (params): Promise => { + // Map Anthropic.MessageCreateParamsNonStreaming → gateway.ChatOpts. + // `judgeSignificance` always sends string content + string system, + // and the adapter only TEXT-flattens the array-of-blocks shape — + // `tool_use`, `tool_result`, image, and other non-text blocks become + // empty strings. If a future caller wires tool-use or image content + // through this client, extend the mapping instead of relying on the + // current silent drop. Same pattern as think/index.ts:607-615. + const messages = params.messages.map(m => ({ + role: m.role, + content: typeof m.content === 'string' + ? m.content + : (Array.isArray(m.content) + ? m.content.map(b => ('text' in b ? b.text : '')).join('') + : ''), + })); + const system = typeof params.system === 'string' + ? params.system + : (Array.isArray(params.system) + ? params.system.map(b => ('text' in b ? b.text : '')).join('') + : undefined); + + const result: ChatResult = await gatewayChat({ + model: modelStr, + system, + messages, + maxTokens: params.max_tokens, + }); + + // Map gateway.ChatResult → Anthropic.Message shape. judgeSignificance + // reads `.content[0].type === 'text'` and `.content[0].text`; other + // fields are best-effort for downstream telemetry parity. + return { + id: '', + type: 'message', + role: 'assistant', + model: modelStr, + content: [{ type: 'text', text: result.text }], + stop_reason: 'end_turn', + stop_sequence: null, + usage: { + input_tokens: result.usage.input_tokens, + output_tokens: result.usage.output_tokens, + }, + } as unknown as Anthropic.Message; + }, + }; +} + +/** + * Anthropic key availability probe. Reads BOTH env (`ANTHROPIC_API_KEY`) + * AND the gbrain config file (`anthropic_api_key` set via + * `gbrain config set`) so stdio MCP launches that don't inherit shell env + * keep working (mirrors `hasAnthropicKey()` in src/core/think/index.ts). + */ +function hasAnthropicKey(): boolean { + if (process.env.ANTHROPIC_API_KEY) return true; + try { + const cfg = loadConfig(); + if (cfg?.anthropic_api_key) return true; + } catch { + // loadConfig may throw on first-run installs; treat as no key. + } + return false; } interface VerdictResult { diff --git a/src/core/operations.ts b/src/core/operations.ts index d69ac7d40..00f101144 100644 --- a/src/core/operations.ts +++ b/src/core/operations.ts @@ -534,7 +534,7 @@ const get_page: Operation = { const put_page: Operation = { name: 'put_page', - description: 'Write/update a page (markdown with frontmatter). Chunks, embeds, reconciles tags, and (when auto_link/auto_timeline are enabled) extracts + reconciles graph links and timeline entries.', + description: 'Write/update a page (markdown with frontmatter). Chunks, embeds, reconciles tags, and (when auto_link/auto_timeline are enabled) extracts + reconciles graph links and timeline entries. For large content on Windows (pipe-buffer limit ~45KB) or any file-as-input workflow, use `gbrain capture --file PATH --slug SLUG` — capture reads the file as a Buffer with a binary-NUL guard and adds provenance write-through (v0.39.3.0).', params: { slug: { type: 'string', required: true, description: 'Page slug' }, content: { type: 'string', required: true, description: 'Full markdown content with YAML frontmatter' }, diff --git a/src/core/search/mode.ts b/src/core/search/mode.ts index c2d401d25..384f61e30 100644 --- a/src/core/search/mode.ts +++ b/src/core/search/mode.ts @@ -25,6 +25,32 @@ import { createHash } from 'crypto'; import { CR_MODES, type CRMode } from '../types.ts'; +import { getRecipe } from '../ai/recipes/index.ts'; + +/** + * Look up the `reranker.default_timeout_ms` declared by the resolved + * reranker model's recipe touchpoint. Returns undefined when: + * - modelStr is empty/null, + * - the provider id doesn't resolve to a registered recipe, + * - the recipe has no reranker touchpoint, or + * - the touchpoint doesn't declare a default_timeout_ms. + * + * Used by `resolveSearchMode()` to slot the recipe default between the + * config-key override and the mode-bundle fallback for `reranker_timeout_ms`. + * Local rerankers (CPU-only llama.cpp + 4B+ cross-encoder) need >5s for + * first-call warmup; without this, the recipe field is dead because + * hybridSearch always passes the bundle's 5000ms value to gateway.rerank(). + * + * Crosses a layer boundary (mode → recipes) deliberately and bounded: + * only the touchpoint timeout. Other touchpoint fields stay on the recipe. + */ +function lookupRerankerRecipeDefaultTimeout(modelStr: string | undefined): number | undefined { + if (!modelStr) return undefined; + const colon = modelStr.indexOf(':'); + const providerId = colon === -1 ? modelStr : modelStr.slice(0, colon); + const recipe = getRecipe(providerId); + return recipe?.touchpoints?.reranker?.default_timeout_ms; +} export type SearchMode = 'conservative' | 'balanced' | 'tokenmax'; @@ -462,6 +488,21 @@ export function resolveSearchMode(input: ResolveSearchModeInput): ResolvedSearch return bundle[key]; }; + // v0.40.6.1: `reranker_timeout_ms` resolution slots the resolved recipe's + // touchpoint default between override and bundle, so local rerankers + // (llama.cpp serving Qwen3-Reranker / self-hosted ZE on CPU) inherit + // their cold-start headroom without forcing users to discover the + // `search.reranker.timeout_ms` config key. + // Precedence: per-call > config override > recipe.touchpoints.reranker.default_timeout_ms > mode bundle. + const resolvedRerankerModel = pick('reranker_model'); + const pickRerankerTimeoutMs = (): number => { + if (pc.reranker_timeout_ms !== undefined) return pc.reranker_timeout_ms; + if (ov.reranker_timeout_ms !== undefined) return ov.reranker_timeout_ms; + const recipeDefault = lookupRerankerRecipeDefaultTimeout(resolvedRerankerModel); + if (recipeDefault !== undefined) return recipeDefault; + return bundle.reranker_timeout_ms; + }; + return { cache_enabled: pick('cache_enabled'), cache_similarity_threshold: pick('cache_similarity_threshold'), @@ -471,10 +512,10 @@ export function resolveSearchMode(input: ResolveSearchModeInput): ResolvedSearch expansion: pick('expansion'), searchLimit: pick('searchLimit'), reranker_enabled: pick('reranker_enabled'), - reranker_model: pick('reranker_model'), + reranker_model: resolvedRerankerModel, reranker_top_n_in: pick('reranker_top_n_in'), reranker_top_n_out: pick('reranker_top_n_out'), - reranker_timeout_ms: pick('reranker_timeout_ms'), + reranker_timeout_ms: pickRerankerTimeoutMs(), // v0.35.6.0 — floor-ratio resolved via the same pick chain. floor_ratio: pick('floor_ratio'), // v0.36 cross-modal knobs diff --git a/test/ai/recipe-llama-server-reranker.test.ts b/test/ai/recipe-llama-server-reranker.test.ts new file mode 100644 index 000000000..9a91432ed --- /dev/null +++ b/test/ai/recipe-llama-server-reranker.test.ts @@ -0,0 +1,82 @@ +/** + * llama-server-reranker recipe smoke (v0.40.6.1). + * + * Sibling of recipe-llama-server.test.ts. Pins the recipe shape so: + * - id + tier + implementation + base_url stay byte-stable + * - reranker touchpoint declares the v0.40.6.1 `path` + `default_timeout_ms` + * fields that the gateway + mode resolution depend on + * - models: [] (user-provided), with realistic default_model placeholder + * - env vars LLAMA_SERVER_RERANKER_BASE_URL + _API_KEY are declared optional + * - cost_per_1m_tokens_usd: 0 so BudgetTracker's FREE_LOCAL_RERANK_PROVIDERS + * contract holds at the recipe layer + */ + +import { describe, expect, test } from 'bun:test'; +import { getRecipe } from '../../src/core/ai/recipes/index.ts'; + +describe('recipe: llama-server-reranker', () => { + test('registered with expected shape', () => { + const r = getRecipe('llama-server-reranker'); + expect(r).toBeDefined(); + expect(r!.id).toBe('llama-server-reranker'); + expect(r!.tier).toBe('openai-compat'); + expect(r!.implementation).toBe('openai-compatible'); + expect(r!.base_url_default).toBe('http://localhost:8081/v1'); + }); + + test('declares the required + optional env vars', () => { + const r = getRecipe('llama-server-reranker')!; + expect(r.auth_env?.required ?? []).toEqual([]); + expect(r.auth_env?.optional ?? []).toContain('LLAMA_SERVER_RERANKER_BASE_URL'); + expect(r.auth_env?.optional ?? []).toContain('LLAMA_SERVER_RERANKER_API_KEY'); + }); + + test('declares reranker touchpoint with path + default_timeout_ms (v0.40.6.1)', () => { + const r = getRecipe('llama-server-reranker')!; + const tp = r.touchpoints.reranker; + expect(tp).toBeDefined(); + // Leaf-only path; gateway concatenates with base_url's `/v1` prefix + // to produce `…/v1/rerank`. The codex diff-review caught the original + // `/v1/rerank` shape here that would have doubled the prefix. + expect(tp!.path).toBe('/rerank'); + expect(tp!.default_timeout_ms).toBe(30_000); + }); + + test('base_url + path concatenation produces /v1/rerank, NOT /v1/v1/rerank', () => { + // Direct shape check: regardless of how the gateway URL builder + // handles concatenation, the recipe's two URL-building inputs must + // produce a single `/v1` prefix when combined. Pins the codex-found + // regression at the recipe layer in addition to the gateway test. + const r = getRecipe('llama-server-reranker')!; + const combined = + r.base_url_default!.replace(/\/$/, '') + (r.touchpoints.reranker!.path ?? '/models/rerank'); + expect(combined).toBe('http://localhost:8081/v1/rerank'); + expect(combined).not.toContain('/v1/v1/'); + }); + + test('reranker touchpoint uses empty models[] for user-provided model ids', () => { + const r = getRecipe('llama-server-reranker')!; + const tp = r.touchpoints.reranker!; + expect(tp.models).toEqual([]); + // default_model is informational placeholder for docs/wizard — real + // id is whatever the user passes to --alias at server launch. + expect(typeof tp.default_model).toBe('string'); + expect(tp.default_model.length).toBeGreaterThan(0); + }); + + test('cost is zero so BudgetTracker FREE_LOCAL_RERANK_PROVIDERS contract holds', () => { + const r = getRecipe('llama-server-reranker')!; + expect(r.touchpoints.reranker!.cost_per_1m_tokens_usd).toBe(0); + }); + + test('max_payload_bytes matches the upstream 5MB ceiling', () => { + const r = getRecipe('llama-server-reranker')!; + expect(r.touchpoints.reranker!.max_payload_bytes).toBe(5_000_000); + }); + + test('setup_hint mentions --alias (paste-ready UX)', () => { + const r = getRecipe('llama-server-reranker')!; + expect(r.setup_hint).toMatch(/--alias/); + expect(r.setup_hint).toMatch(/--reranking/); + }); +}); diff --git a/test/ai/rerank.test.ts b/test/ai/rerank.test.ts index 881fc4af8..4de11f367 100644 --- a/test/ai/rerank.test.ts +++ b/test/ai/rerank.test.ts @@ -325,3 +325,87 @@ describe('gateway.rerank() — guard rails', () => { } }); }); + +describe('gateway.rerank() — v0.40.6.1 RerankerTouchpoint.path override', () => { + test('honors tp.path when recipe declares one (llama-server-reranker → /v1/rerank)', async () => { + configureGateway({ + reranker_model: 'llama-server-reranker:qwen3-reranker-4b', + env: {}, + }); + let capturedUrl = ''; + __setRerankTransportForTests(async (url) => { + capturedUrl = url; + return mockResp({ results: [{ index: 0, relevance_score: 0.8 }] }); + }); + await rerank({ query: 'q', documents: ['d'] }); + // Recipe `base_url_default` is `http://localhost:8081/v1` and the + // touchpoint `path` is the LEAF `/rerank` — gateway must concatenate + // them to `http://localhost:8081/v1/rerank` exactly. The codex + // diff-review caught a /v1 path-doubling bug here that the prior + // `endsWith('/v1/rerank')` assertion silently passed through; the + // exact-equality assertion below is the regression guard. + expect(capturedUrl).toBe('http://localhost:8081/v1/rerank'); + expect(capturedUrl).not.toContain('/v1/v1/'); + expect(capturedUrl).not.toContain('/models/rerank'); + }); + + test('falls through to /models/rerank when recipe omits path (ZE regression)', async () => { + configureZE(); + let capturedUrl = ''; + __setRerankTransportForTests(async (url) => { + capturedUrl = url; + return mockResp({ results: [{ index: 0, relevance_score: 0.9 }] }); + }); + await rerank({ query: 'q', documents: ['d'] }); + expect(capturedUrl).toBe('https://api.zeroentropy.dev/v1/models/rerank'); + }); +}); + +describe('gateway.rerank() — v0.40.6.1 user-provided models (empty allowlist)', () => { + test('empty models[] on the recipe accepts any model id', async () => { + // llama-server-reranker declares `models: []` — anything goes. + configureGateway({ + reranker_model: 'llama-server-reranker:some-custom-model-id', + env: {}, + }); + let called = false; + __setRerankTransportForTests(async () => { + called = true; + return mockResp({ results: [{ index: 0, relevance_score: 0.7 }] }); + }); + const out = await rerank({ query: 'q', documents: ['d'] }); + expect(called).toBe(true); + expect(out.length).toBe(1); + }); + + test('still rejects empty models[] on a different model resolved via input.model', async () => { + // Even when caller overrides via input.model, the resolved recipe still + // governs the allowlist. Empty allowlist = no restriction. + configureGateway({ + reranker_model: 'zeroentropyai:zerank-2', + env: { ZEROENTROPY_API_KEY: 'sk-test' }, + }); + __setRerankTransportForTests(async () => + mockResp({ results: [{ index: 0, relevance_score: 0.6 }] }), + ); + const out = await rerank({ + query: 'q', + documents: ['d'], + model: 'llama-server-reranker:whatever-id', // recipe with empty allowlist + }); + expect(out.length).toBe(1); + }); +}); + +describe('gateway.rerank() — v0.40.6.1 path regression: zerank-1-small unaffected', () => { + test('legacy ZE allowlist members still hit /models/rerank', async () => { + configureZE('zeroentropyai:zerank-1-small'); + let capturedUrl = ''; + __setRerankTransportForTests(async (url) => { + capturedUrl = url; + return mockResp({ results: [{ index: 0, relevance_score: 0.5 }] }); + }); + await rerank({ query: 'q', documents: ['d'] }); + expect(capturedUrl.endsWith('/models/rerank')).toBe(true); + }); +}); diff --git a/test/brain-score-recommendations.test.ts b/test/brain-score-recommendations.test.ts index e866eb53e..998d5f1b4 100644 --- a/test/brain-score-recommendations.test.ts +++ b/test/brain-score-recommendations.test.ts @@ -4,9 +4,69 @@ import { classifyChecks, maxReachableScore, estimateAnthropicCost, + embeddingProviderConfigured, + HOSTED_EMBED_KEY_CONFIG, } from '../src/core/brain-score-recommendations.ts'; import type { BrainHealth } from '../src/core/types.ts'; +/** + * v0.40.x — recipe-aware embedding-provider check (shared by doctor + + * autopilot). Pure: `resolveKey` is injected, so no env mutation (R1-safe). + */ +describe('embeddingProviderConfigured (recipe-aware helper)', () => { + const alwaysTrue = () => true; + const alwaysFalse = () => false; + + test('empty / undefined model → false', () => { + expect(embeddingProviderConfigured(undefined, alwaysTrue)).toBe(false); + expect(embeddingProviderConfigured('', alwaysTrue)).toBe(false); + }); + + test('local providers (empty auth_env.required) → true regardless of keys', () => { + // The core fix: ollama / llama-server need no hosted key. + expect(embeddingProviderConfigured('ollama:nomic-embed-text', alwaysFalse)).toBe(true); + expect(embeddingProviderConfigured('llama-server:my-gguf', alwaysFalse)).toBe(true); + }); + + test('hosted provider configured iff its required key resolves', () => { + expect(embeddingProviderConfigured('openai:text-embedding-3-small', (k) => k === 'OPENAI_API_KEY')).toBe(true); + // REGRESSION: hosted without its key still blocks. + expect(embeddingProviderConfigured('openai:text-embedding-3-small', alwaysFalse)).toBe(false); + }); + + test('behavior-change: voyage judged by VOYAGE_API_KEY, not an OpenAI key', () => { + // New (correct) behavior — pre-fix doctor judged voyage by whether any + // openai/ZE key existed. + expect(embeddingProviderConfigured('voyage:voyage-3', (k) => k === 'VOYAGE_API_KEY')).toBe(true); + expect(embeddingProviderConfigured('voyage:voyage-3', (k) => k === 'OPENAI_API_KEY')).toBe(false); + }); + + test('unknown provider (no recipe) → false', () => { + expect(embeddingProviderConfigured('made-up-provider:foo', alwaysTrue)).toBe(false); + }); + + test('malformed model id → false (parseModelId throws, caught)', () => { + expect(embeddingProviderConfigured('noColon', alwaysTrue)).toBe(false); + expect(embeddingProviderConfigured('::', alwaysTrue)).toBe(false); + expect(embeddingProviderConfigured(':model', alwaysTrue)).toBe(false); + expect(embeddingProviderConfigured('provider:', alwaysTrue)).toBe(false); + }); + + // Regression (codex review of the local-embeddings PR): the producer closures + // (doctor + autopilot) consult HOSTED_EMBED_KEY_CONFIG to decide which config + // field backs each env key. Only keys that buildGatewayConfig actually folds + // into the gateway env may appear, or a config-plane key the gateway ignores + // would make the provider look "configured" and dispatch a doomed embed job. + test('HOSTED_EMBED_KEY_CONFIG only maps gateway-propagated config keys', () => { + expect(HOSTED_EMBED_KEY_CONFIG.OPENAI_API_KEY).toBe('openai_api_key'); + expect(HOSTED_EMBED_KEY_CONFIG.ZEROENTROPY_API_KEY).toBe('zeroentropy_api_key'); + // Not propagated to the gateway today → must NOT be backed by a config field + // (producer closures fall through to process.env only for these). + expect(HOSTED_EMBED_KEY_CONFIG.VOYAGE_API_KEY).toBeUndefined(); + expect(HOSTED_EMBED_KEY_CONFIG.GOOGLE_GENERATIVE_AI_API_KEY).toBeUndefined(); + }); +}); + /** * D6 #5 + D13 + D14 pinning tests for brain-score-recommendations. * @@ -38,13 +98,13 @@ function makeHealth(overrides: Partial = {}): BrainHealth { describe('computeRecommendations', () => { test('healthy brain (score 100) produces empty plan', () => { const health = makeHealth(); - const recs = computeRecommendations(health, { repoPath: '/brain', hasEmbeddingApiKey: true }); + const recs = computeRecommendations(health, { repoPath: '/brain', embeddingProviderConfigured: true }); expect(recs).toEqual([]); }); test('missing embeddings produces embed.stale remediation', () => { const health = makeHealth({ missing_embeddings: 1432, brain_score: 65 }); - const recs = computeRecommendations(health, { repoPath: '/brain', hasEmbeddingApiKey: true }); + const recs = computeRecommendations(health, { repoPath: '/brain', embeddingProviderConfigured: true }); const ids = recs.map((r) => r.id); expect(ids).toContain('embed.stale'); const embedRec = recs.find((r) => r.id === 'embed.stale')!; @@ -55,7 +115,7 @@ describe('computeRecommendations', () => { test('missing embeddings + API key absent: NOT emitted (blocked surfaces separately)', () => { const health = makeHealth({ missing_embeddings: 1432 }); - const recs = computeRecommendations(health, { repoPath: '/brain', hasEmbeddingApiKey: false }); + const recs = computeRecommendations(health, { repoPath: '/brain', embeddingProviderConfigured: false }); expect(recs.find((r) => r.id === 'embed.stale')).toBeUndefined(); }); @@ -65,7 +125,7 @@ describe('computeRecommendations', () => { dead_links: 8, brain_score: 70, }); - const recs = computeRecommendations(health, { repoPath: '/brain', hasEmbeddingApiKey: true }); + const recs = computeRecommendations(health, { repoPath: '/brain', embeddingProviderConfigured: true }); const ids = recs.map((r) => r.id); expect(ids).toContain('sync.repo'); expect(ids).toContain('backlinks.fix'); @@ -74,7 +134,7 @@ describe('computeRecommendations', () => { test('extract.all depends on sync.repo (D14: stable ids)', () => { const health = makeHealth({ stale_pages: 10 }); - const recs = computeRecommendations(health, { repoPath: '/brain', hasEmbeddingApiKey: true }); + const recs = computeRecommendations(health, { repoPath: '/brain', embeddingProviderConfigured: true }); const extract = recs.find((r) => r.id === 'extract.all'); expect(extract?.depends_on).toContain('sync.repo'); }); @@ -84,14 +144,14 @@ describe('computeRecommendations', () => { stale_pages: 10, missing_embeddings: 100, }); - const recs = computeRecommendations(health, { repoPath: '/brain', hasEmbeddingApiKey: true }); + const recs = computeRecommendations(health, { repoPath: '/brain', embeddingProviderConfigured: true }); const embed = recs.find((r) => r.id === 'embed.stale'); expect(embed?.depends_on).toContain('sync.repo'); }); test('embed.stale has no sync dependency when nothing stale', () => { const health = makeHealth({ missing_embeddings: 100 }); - const recs = computeRecommendations(health, { repoPath: '/brain', hasEmbeddingApiKey: true }); + const recs = computeRecommendations(health, { repoPath: '/brain', embeddingProviderConfigured: true }); const embed = recs.find((r) => r.id === 'embed.stale'); expect(embed?.depends_on).toEqual([]); }); @@ -101,7 +161,7 @@ describe('computeRecommendations', () => { missing_embeddings: 100, // critical stale_pages: 80, // high }); - const recs = computeRecommendations(health, { repoPath: '/brain', hasEmbeddingApiKey: true }); + const recs = computeRecommendations(health, { repoPath: '/brain', embeddingProviderConfigured: true }); const critIdx = recs.findIndex((r) => r.severity === 'critical'); const highIdx = recs.findIndex((r) => r.severity === 'high'); expect(critIdx).toBeLessThan(highIdx); @@ -114,7 +174,7 @@ describe('computeRecommendations', () => { missing_embeddings: 50, dead_links: 3, }); - const ctx = { repoPath: '/brain', hasEmbeddingApiKey: true, sourceId: 'default' }; + const ctx = { repoPath: '/brain', embeddingProviderConfigured: true, sourceId: 'default' }; const run1 = computeRecommendations(health, ctx); const run2 = computeRecommendations(health, ctx); expect(JSON.stringify(run1)).toBe(JSON.stringify(run2)); @@ -124,7 +184,7 @@ describe('computeRecommendations', () => { const health = makeHealth({ missing_embeddings: 50 }); const recs = computeRecommendations(health, { repoPath: '/brain', - hasEmbeddingApiKey: true, + embeddingProviderConfigured: true, sourceId: 'default', }); const embed = recs.find((r) => r.id === 'embed.stale')!; @@ -136,14 +196,14 @@ describe('computeRecommendations', () => { test('D9: different sources produce different idempotency keys', () => { const health = makeHealth({ missing_embeddings: 50 }); - const a = computeRecommendations(health, { repoPath: '/brain', hasEmbeddingApiKey: true, sourceId: 'A' }); - const b = computeRecommendations(health, { repoPath: '/brain', hasEmbeddingApiKey: true, sourceId: 'B' }); + const a = computeRecommendations(health, { repoPath: '/brain', embeddingProviderConfigured: true, sourceId: 'A' }); + const b = computeRecommendations(health, { repoPath: '/brain', embeddingProviderConfigured: true, sourceId: 'B' }); expect(a[0]!.idempotency_key).not.toBe(b[0]!.idempotency_key); }); test('status field is always remediable in the output list (D13)', () => { const health = makeHealth({ missing_embeddings: 50 }); - const recs = computeRecommendations(health, { repoPath: '/brain', hasEmbeddingApiKey: true }); + const recs = computeRecommendations(health, { repoPath: '/brain', embeddingProviderConfigured: true }); for (const r of recs) expect(r.status).toBe('remediable'); }); @@ -151,7 +211,7 @@ describe('computeRecommendations', () => { const health = makeHealth({ missing_embeddings: 1000 }); const recs = computeRecommendations(health, { repoPath: '/brain', - hasEmbeddingApiKey: true, + embeddingProviderConfigured: true, embeddingModel: 'openai:text-embedding-3-large', embeddingDimensions: 3072, }); @@ -163,14 +223,14 @@ describe('computeRecommendations', () => { describe('classifyChecks (D13)', () => { test('remediable: missing_embeddings with API key', () => { const result = classifyChecks([{ name: 'missing_embeddings', status: 'fail' }], { - hasEmbeddingApiKey: true, + embeddingProviderConfigured: true, }); expect(result[0]).toEqual({ check: 'missing_embeddings', status: 'remediable' }); }); test('blocked: missing_embeddings without API key', () => { const result = classifyChecks([{ name: 'missing_embeddings', status: 'fail' }], { - hasEmbeddingApiKey: false, + embeddingProviderConfigured: false, }); expect(result[0]?.status).toBe('blocked'); expect(result[0]?.reason).toContain('embedding'); diff --git a/test/core/budget/budget-tracker.test.ts b/test/core/budget/budget-tracker.test.ts index 034bbe4d1..f30834e26 100644 --- a/test/core/budget/budget-tracker.test.ts +++ b/test/core/budget/budget-tracker.test.ts @@ -161,6 +161,101 @@ describe('BudgetTracker.reserve', () => { const audit = readAudit(); expect(audit.filter((e) => e.event === 'reserve_unpriced').length).toBe(2); }); + + test('v0.40.6.1: rerank kind for llama-server-reranker prices at $0 (no TX2 throw under --max-cost)', () => { + // The FREE_LOCAL_RERANK_PROVIDERS contract — local inference costs + // electricity, not API tokens. Pre-v0.40.6.1 setting --max-cost while + // configured for a local reranker would TX2 hard-fail because the + // lookupPricing fall-through path returned null for any provider not + // in ANTHROPIC_PRICING. Now the rerank kind recognizes the local + // provider prefix and returns zero pricing. + const t = new BudgetTracker({ maxCostUsd: 0.0001, label: 'test', auditPath }); + expect(() => + t.reserve({ + modelId: 'llama-server-reranker:qwen3-reranker-4b', + estimatedInputTokens: 5000, + maxOutputTokens: 0, + kind: 'rerank', + }), + ).not.toThrow(); + expect(t.totalSpent).toBe(0); + }); + + test('v0.40.6.1: rerank kind for arbitrary model id under llama-server-reranker provider still zero-priced', () => { + // Empty allowlist on the recipe means any model id is valid; budget + // path must agree. + const t = new BudgetTracker({ maxCostUsd: 0.0001, label: 'test', auditPath }); + expect(() => + t.reserve({ + modelId: 'llama-server-reranker:some-custom-gguf-id', + estimatedInputTokens: 5000, + maxOutputTokens: 0, + kind: 'rerank', + }), + ).not.toThrow(); + }); + + test('v0.40.6.1: chat kind for the same provider prefix is NOT zero-priced (rerank-only contract)', () => { + // The free-local zero-price applies ONLY to the rerank kind. If someone + // wires a chat call through this provider with --max-cost, the TX2 hard- + // fail behavior is preserved so the user gets a clear "no pricing entry" + // signal rather than silent zero accounting. + const t = new BudgetTracker({ maxCostUsd: 1.0, label: 'test', auditPath }); + let caught: unknown = null; + try { + t.reserve({ + modelId: 'llama-server-reranker:not-actually-a-chat-model', + estimatedInputTokens: 100, + maxOutputTokens: 100, + kind: 'chat', + }); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(BudgetExhausted); + expect((caught as BudgetExhausted).reason).toBe('no_pricing'); + }); + + test('v0.40.x: local embed providers price at $0 (no TX2 throw under --max-cost)', () => { + // FREE_LOCAL_EMBED_PROVIDERS — ollama / llama-server run on local inference + // (electricity, not tokens). Pre-fix a --max-cost embed/reindex job + // configured for a local provider TX2 hard-failed because + // lookupEmbeddingPrice has no entry for them. + for (const modelId of ['ollama:nomic-embed-text', 'llama-server:my-gguf']) { + const t = new BudgetTracker({ maxCostUsd: 0.0001, label: 'test', auditPath }); + expect(() => + t.reserve({ modelId, estimatedInputTokens: 5000, maxOutputTokens: 0, kind: 'embed' }), + ).not.toThrow(); + expect(t.totalSpent).toBe(0); + } + }); + + test('v0.40.x REGRESSION: unknown hosted embed provider still TX2 hard-fails under cap', () => { + const t = new BudgetTracker({ maxCostUsd: 1.0, label: 'test', auditPath }); + let caught: unknown = null; + try { + t.reserve({ modelId: 'mystery:some-embed-model', estimatedInputTokens: 100, maxOutputTokens: 0, kind: 'embed' }); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(BudgetExhausted); + expect((caught as BudgetExhausted).reason).toBe('no_pricing'); + }); + + test('v0.40.x REGRESSION: known hosted embed (openai) still real-priced (trips a tiny cap)', () => { + // 3-small is $0.02/1M tokens. 1M tokens projects $0.02 > $0.0001 cap, so a + // real (nonzero) price trips the cost gate — proving it's NOT on the $0 + // local-provider path. The local providers above do NOT throw at the same cap. + const t = new BudgetTracker({ maxCostUsd: 0.0001, label: 'test', auditPath }); + let caught: unknown = null; + try { + t.reserve({ modelId: 'openai:text-embedding-3-small', estimatedInputTokens: 1_000_000, maxOutputTokens: 0, kind: 'embed' }); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(BudgetExhausted); + expect((caught as BudgetExhausted).reason).toBe('cost'); + }); }); describe('BudgetTracker.record', () => { diff --git a/test/cycle/regression-pr-wave-r1-r2-r4.test.ts b/test/cycle/regression-pr-wave-r1-r2-r4.test.ts new file mode 100644 index 000000000..3d3c2f653 --- /dev/null +++ b/test/cycle/regression-pr-wave-r1-r2-r4.test.ts @@ -0,0 +1,144 @@ +/** + * Critical regression pins for the community-PR-wave landing (R1+R2+R4). + * + * Per the wave's eng-review plan (IRON RULE — no skipping): + * R1 — get_page handler accepts calls without `content` param. Pre-wave PR #1365 + * landed a `!p.content → throw` check inside the WRONG handler (get_page + * instead of put_page), which would have broken every read in the system. + * This test pins that get_page calls without `content` don't error on a + * param-shape check. + * R2 — put_page schema content stays `required: true`. PR #1365 also flipped + * `content` from required→optional. The schema regression pin asserts the + * contract stays at `required: true`. + * R4 — Cross-platform stdin behavior. PR #1325 swapped `'/dev/stdin'` for + * fd 0. The behavior test mocks `process.stdin.isTTY = false` and + * confirms parseOpArgs reads stdin via the fd path. A supplementary + * source-grep guards against literal `'/dev/stdin'` regressing. + * + * R3 (gateway-adapter parsed-verdict parity) lives in the sibling file + * `test/cycle/synthesize-gateway-adapter.test.ts`. + */ + +import { describe, test, expect } from 'bun:test'; +import { readFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { operations, OperationError } from '../../src/core/operations.ts'; +import type { OperationContext, Operation } from '../../src/core/operations.ts'; +import type { BrainEngine } from '../../src/core/engine.ts'; + +const get_page = operations.find(o => o.name === 'get_page') as Operation; +const put_page = operations.find(o => o.name === 'put_page') as Operation; +if (!get_page) throw new Error('get_page op missing'); +if (!put_page) throw new Error('put_page op missing'); + +function makeCtx(overrides: Partial = {}): OperationContext { + const engine = { + getPage: async (_slug: string) => ({ + id: 1, + slug: 'people/alice', + type: 'person', + title: 'Alice', + compiled_truth: 'stub', + timeline: '', + tags: [], + created_at: new Date('2026-05-24'), + updated_at: new Date('2026-05-24'), + content_hash: 'sha-stub', + source_id: 'default', + effective_date: null, + deleted_at: null, + }), + getTags: async () => [], + resolveSlugs: async () => [], + putPage: async () => ({ slug: 'stub', id: 1, created: true }), + } as unknown as BrainEngine; + return { + engine, + config: { engine: 'postgres' } as any, + logger: { info: () => {}, warn: () => {}, error: () => {} }, + dryRun: false, + remote: false, + sourceId: 'default', + ...overrides, + }; +} + +describe('R1 — get_page handler accepts calls without content param', () => { + test('R1: get_page with only slug (no content) does NOT throw param-shape error', async () => { + const ctx = makeCtx(); + // If PR #1365's broken handler-block lived in get_page, this call would + // throw OperationError('invalid_request', 'put_page requires either content + // or file parameter'). The pin: get_page MUST NOT require content. + const result = await get_page.handler(ctx, { slug: 'people/alice' }); + expect(result).toBeDefined(); + }); + + test('R1 corollary: get_page schema has no `content` param (read op)', () => { + expect(get_page.params).toBeDefined(); + expect('content' in (get_page.params ?? {})).toBe(false); + expect('file' in (get_page.params ?? {})).toBe(false); + }); +}); + +describe('R2 — put_page schema content stays required: true', () => { + test('R2: content param exists and is marked required: true', () => { + expect(put_page.params).toBeDefined(); + const params = put_page.params as Record; + expect(params.content).toBeDefined(); + expect(params.content.type).toBe('string'); + expect(params.content.required).toBe(true); + }); + + test('R2 corollary: put_page schema does NOT carry the closed PR #1365 `file` param', () => { + const params = put_page.params as Record; + expect('file' in params).toBe(false); + }); + + test('R2: put_page handler still throws when content is missing (server-side schema enforced)', async () => { + const ctx = makeCtx({ dryRun: true }); + // Missing content — handler should reject (required: true is enforced by + // dispatch layer; the dry-run short-circuit happens AFTER schema checks + // in the real op path. Here we assert the call surface still requires it.) + const params = put_page.params as Record; + expect(params.content.required).toBe(true); + }); +}); + +describe('R4 — cross-platform stdin via fd 0 (PR #1325 regression pin)', () => { + test('R4 source-grep: src/cli.ts uses readFileSync(0, ...) not readFileSync("/dev/stdin", ...)', () => { + // Belt-and-suspenders source-grep guard. The behavior of fd 0 is OS-level + // and hard to unit-test deterministically across platforms; this guard + // catches a future contributor reverting the cross-platform fix. + const path = join(import.meta.dir ?? '.', '..', '..', 'src', 'cli.ts'); + const src = readFileSync(path, 'utf-8'); + + // The exact pattern the PR replaced. If anyone reintroduces it, R4 fires. + // Look for `'/dev/stdin'` with surrounding quote so we don't false-fire + // on docs/comments that quote the legacy form mid-sentence. + const badPattern = /readFileSync\(\s*['"]\/dev\/stdin['"]/; + expect(src).not.toMatch(badPattern); + + // The replacement pattern MUST be present. `readFileSync(0, ...)` — fd 0 + // is the canonical Node cross-platform stdin idiom. + const goodPattern = /readFileSync\(\s*0\s*,/; + expect(src).toMatch(goodPattern); + }); + + test('R4 behavior: parseOpArgs reads stdin when isTTY=false and no positional content provided', () => { + // Lighter-weight than mocking readFileSync (which is module-scoped at + // import time and brittle to stub safely without mock.module). The + // source-grep above is the primary regression guard; this test asserts + // the surrounding shape of the parseOpArgs stdin-reading branch hasn't + // drifted (existence of the branch + 5MB cap), since the branch itself + // is what was modified by PR #1325. + const path = join(import.meta.dir ?? '.', '..', '..', 'src', 'cli.ts'); + const src = readFileSync(path, 'utf-8'); + + // Stdin reading branch still exists in parseOpArgs. + expect(src).toMatch(/op\.cliHints\?\.stdin/); + // 5MB cap is still in place. + expect(src).toMatch(/MAX_STDIN\s*=\s*5_000_000/); + // isTTY check still gates stdin reading. + expect(src).toMatch(/!process\.stdin\.isTTY/); + }); +}); diff --git a/test/cycle/synthesize-gateway-adapter.test.ts b/test/cycle/synthesize-gateway-adapter.test.ts new file mode 100644 index 000000000..631310edc --- /dev/null +++ b/test/cycle/synthesize-gateway-adapter.test.ts @@ -0,0 +1,319 @@ +/** + * Gateway-adapter tests for the dream-cycle significance judge (T5 + T6 wave). + * + * Replaces the v0.23-era `new Anthropic()` direct-SDK construction with a + * gateway-routed JudgeClient that works for any provider with a registered + * recipe (Anthropic, DeepSeek, OpenRouter, Voyage, Ollama, llama-server, ...). + * + * Mirrors the test pattern from test/think-gateway-adapter.test.ts for parity + * with src/core/think/index.ts (v0.35.5.0). The IRON RULE regression R3 lives + * here too — given identical canned LLM text, judgeSignificance produces the + * same {worth_processing, reasons} via the gateway-adapter shape as it would + * via the legacy Anthropic SDK shape. The contract that matters is parsed- + * verdict SEMANTIC PARITY (not byte-identical Anthropic.Message struct, which + * codex correctly flagged as a meaningless gate). + */ + +import { describe, test, expect, afterEach } from 'bun:test'; +import { + __setChatTransportForTests, + resetGateway, + type ChatResult, +} from '../../src/core/ai/gateway.ts'; +import { AIConfigError } from '../../src/core/ai/errors.ts'; +import { makeJudgeClient, judgeSignificance, type JudgeClient } from '../../src/core/cycle/synthesize.ts'; +import { withEnv } from '../helpers/with-env.ts'; +import type { DiscoveredTranscript } from '../../src/core/cycle/transcript-discovery.ts'; + +afterEach(() => { + __setChatTransportForTests(null); + resetGateway(); +}); + +// Canned "worth processing" LLM text used by the parsed-verdict parity tests. +// Mirrors what a well-tuned Haiku would emit for a substantive transcript. +const WORTH_PROCESSING_JSON = JSON.stringify({ + worth_processing: true, + reasons: ['user reflects on portfolio framework', 'concrete strategic call'], +}); + +// Synthetic transcript fixture for judgeSignificance — only `content` and +// `basename` are read by the judge. +const FIXTURE_TRANSCRIPT: DiscoveredTranscript = { + filePath: '/dev/null/fixture.txt', + basename: 'fixture', + content: 'Synthetic transcript content for gateway-adapter parity tests.', + contentHash: 'sha-fixture-1', + inferredDate: '2026-05-24', +}; + +describe('makeJudgeClient — construction-time provider probe', () => { + test('A1: returns null when verdict model is anthropic and no API key is configured', async () => { + await withEnv({ ANTHROPIC_API_KEY: undefined }, async () => { + // Use a synthetic config path to avoid surfacing a stored anthropic_api_key. + await withEnv({ GBRAIN_HOME: '/tmp/nonexistent-gbrain-home-for-A1' }, async () => { + const judge = makeJudgeClient('claude-haiku-4-5-20251001'); + expect(judge).toBeNull(); + }); + }); + }); + + test('A2: returns a JudgeClient when chat provider is reachable (anthropic key set)', async () => { + await withEnv({ ANTHROPIC_API_KEY: 'sk-test-A2' }, async () => { + const judge = makeJudgeClient('claude-haiku-4-5-20251001'); + expect(judge).not.toBeNull(); + expect(typeof judge?.create).toBe('function'); + }); + }); + + test('A8: returns null when verdict model has unknown provider prefix', async () => { + // resolveRecipe throws AIConfigError on unknown provider id; + // makeJudgeClient catches it and returns null. + const judge = makeJudgeClient('notarealprovider:some-model'); + expect(judge).toBeNull(); + }); + + test('A9: returns a JudgeClient for non-anthropic providers without probing env (delegates to gateway)', async () => { + // Non-anthropic providers don't get the hasAnthropicKey() short-circuit. + // The deepseek recipe declares DEEPSEEK_API_KEY in auth_env.required; + // makeJudgeClient delegates that probe to gateway.chat at call time + // (where it would throw AIConfigError, caught per-transcript by the loop). + await withEnv({ DEEPSEEK_API_KEY: undefined }, async () => { + const judge = makeJudgeClient('deepseek:deepseek-chat'); + expect(judge).not.toBeNull(); + expect(typeof judge?.create).toBe('function'); + }); + }); +}); + +describe('JudgeClient.create — gateway routing + shape adapter', () => { + test('A3: routes through gateway.chat (verified via __setChatTransportForTests stub)', async () => { + await withEnv({ ANTHROPIC_API_KEY: 'sk-test-A3' }, async () => { + const judge = makeJudgeClient('claude-haiku-4-5-20251001'); + expect(judge).not.toBeNull(); + + let transportCalled = false; + let receivedSystem: string | undefined; + let receivedModel: string | undefined; + __setChatTransportForTests(async (opts): Promise => { + transportCalled = true; + receivedSystem = opts.system; + receivedModel = opts.model; + return { + text: WORTH_PROCESSING_JSON, + blocks: [], + stopReason: 'end', + usage: { input_tokens: 10, output_tokens: 20, cache_read_tokens: 0, cache_creation_tokens: 0 }, + model: 'test:stub', + providerId: 'test', + }; + }); + + const result = await judge!.create({ + model: 'claude-haiku-4-5-20251001', + max_tokens: 200, + system: 'judge system prompt', + messages: [{ role: 'user', content: 'judge this' }], + }); + + expect(transportCalled).toBe(true); + expect(receivedSystem).toBe('judge system prompt'); + // Gateway model gets the anthropic: prefix normalized + expect(receivedModel).toBe('anthropic:claude-haiku-4-5-20251001'); + // Anthropic.Message shape returned + expect(result.content?.[0]?.type).toBe('text'); + expect((result.content?.[0] as { type: string; text: string }).text).toBe(WORTH_PROCESSING_JSON); + }); + }); + + test('A4: ChatResult.text → Anthropic.Message.content[0].text mapping', async () => { + await withEnv({ ANTHROPIC_API_KEY: 'sk-test-A4' }, async () => { + const judge = makeJudgeClient('claude-haiku-4-5-20251001'); + __setChatTransportForTests(async (): Promise => ({ + text: 'mapped text content', + blocks: [], + stopReason: 'end', + usage: { input_tokens: 5, output_tokens: 5, cache_read_tokens: 0, cache_creation_tokens: 0 }, + model: 'test:stub', + providerId: 'test', + })); + + const result = await judge!.create({ + model: 'claude-haiku-4-5-20251001', + max_tokens: 100, + system: 's', + messages: [{ role: 'user', content: 'u' }], + }); + + expect(result.role).toBe('assistant'); + expect(result.type).toBe('message'); + expect(result.content?.[0]?.type).toBe('text'); + expect((result.content?.[0] as { type: string; text: string }).text).toBe('mapped text content'); + expect(result.usage.input_tokens).toBe(5); + expect(result.usage.output_tokens).toBe(5); + }); + }); + + test('A5: empty text from gateway → returns Anthropic.Message with empty text content (graceful)', async () => { + await withEnv({ ANTHROPIC_API_KEY: 'sk-test-A5' }, async () => { + const judge = makeJudgeClient('claude-haiku-4-5-20251001'); + __setChatTransportForTests(async (): Promise => ({ + text: '', + blocks: [], + stopReason: 'end', + usage: { input_tokens: 0, output_tokens: 0, cache_read_tokens: 0, cache_creation_tokens: 0 }, + model: 'test:stub', + providerId: 'test', + })); + + const result = await judge!.create({ + model: 'claude-haiku-4-5-20251001', + max_tokens: 100, + system: 's', + messages: [{ role: 'user', content: 'u' }], + }); + + // Doesn't throw; produces a well-shaped Anthropic.Message with empty text. + expect(result.content?.[0]?.type).toBe('text'); + expect((result.content?.[0] as { type: string; text: string }).text).toBe(''); + }); + }); + + test('A6: non-AIConfigError from gateway propagates to caller (no swallowing)', async () => { + await withEnv({ ANTHROPIC_API_KEY: 'sk-test-A6' }, async () => { + const judge = makeJudgeClient('claude-haiku-4-5-20251001'); + __setChatTransportForTests(async (): Promise => { + throw new Error('network blip'); + }); + + let caught: unknown = null; + try { + await judge!.create({ + model: 'claude-haiku-4-5-20251001', + max_tokens: 100, + system: 's', + messages: [{ role: 'user', content: 'u' }], + }); + } catch (e) { + caught = e; + } + expect(caught).toBeInstanceOf(Error); + expect((caught as Error).message).toBe('network blip'); + }); + }); + + test('A7: AIConfigError from gateway propagates as AIConfigError (caught by verdict loop in production)', async () => { + await withEnv({ ANTHROPIC_API_KEY: 'sk-test-A7' }, async () => { + const judge = makeJudgeClient('claude-haiku-4-5-20251001'); + __setChatTransportForTests(async (): Promise => { + throw new AIConfigError('anthropic_api_key revoked mid-run'); + }); + + let caught: unknown = null; + try { + await judge!.create({ + model: 'claude-haiku-4-5-20251001', + max_tokens: 100, + system: 's', + messages: [{ role: 'user', content: 'u' }], + }); + } catch (e) { + caught = e; + } + expect(caught).toBeInstanceOf(AIConfigError); + }); + }); +}); + +describe('R3 — parsed-verdict semantic parity (IRON RULE regression)', () => { + /** + * The contract that matters: given identical canned LLM text content, + * judgeSignificance produces the same {worth_processing, reasons} parsed + * values whether the JudgeClient is a gateway-routed adapter or a hand- + * rolled stub matching the pre-v0.40.x Anthropic SDK shape. Byte-identity + * of the underlying Anthropic.Message struct is NOT the contract (per + * codex outside-voice review of the wave plan). + */ + test('R3: gateway-routed JudgeClient produces same parsed verdict as legacy SDK-shape JudgeClient', async () => { + // The "legacy" path — a JudgeClient that returns an Anthropic.Message + // shape directly, bypassing the gateway. This is the shape + // makeHaikuClient() used to construct via `new Anthropic()`. + const legacyJudge: JudgeClient = { + create: async () => ({ + id: 'msg_legacy', + type: 'message', + role: 'assistant', + model: 'claude-haiku-4-5-20251001', + content: [{ type: 'text', text: WORTH_PROCESSING_JSON }], + stop_reason: 'end_turn', + stop_sequence: null, + usage: { input_tokens: 100, output_tokens: 50 }, + // eslint-disable-next-line @typescript-eslint/no-explicit-any + } as any), + }; + + await withEnv({ ANTHROPIC_API_KEY: 'sk-test-R3' }, async () => { + const gatewayJudge = makeJudgeClient('claude-haiku-4-5-20251001'); + expect(gatewayJudge).not.toBeNull(); + __setChatTransportForTests(async (): Promise => ({ + text: WORTH_PROCESSING_JSON, + blocks: [], + stopReason: 'end', + usage: { input_tokens: 100, output_tokens: 50, cache_read_tokens: 0, cache_creation_tokens: 0 }, + model: 'anthropic:claude-haiku-4-5-20251001', + providerId: 'anthropic', + })); + + const [legacyVerdict, gatewayVerdict] = await Promise.all([ + judgeSignificance(legacyJudge, FIXTURE_TRANSCRIPT, 'claude-haiku-4-5-20251001'), + judgeSignificance(gatewayJudge!, FIXTURE_TRANSCRIPT, 'claude-haiku-4-5-20251001'), + ]); + + // The parsed-verdict semantic-parity contract. + expect(gatewayVerdict.worth_processing).toBe(legacyVerdict.worth_processing); + expect(gatewayVerdict.reasons).toEqual(legacyVerdict.reasons); + // Sanity: both produced the expected verdict (not just both empty). + expect(legacyVerdict.worth_processing).toBe(true); + expect(legacyVerdict.reasons.length).toBeGreaterThan(0); + }); + }); + + test('R3 corollary: unparseable LLM output → both paths return cheap-fallback verdict', async () => { + // Pre-rework AND post-rework both fall through to the + // "judge response unparseable" branch when content isn't JSON. + const legacyJudge: JudgeClient = { + create: async () => ({ + id: 'msg_legacy_garbage', + type: 'message', + role: 'assistant', + model: 'claude-haiku-4-5-20251001', + content: [{ type: 'text', text: 'not json at all' }], + stop_reason: 'end_turn', + stop_sequence: null, + usage: { input_tokens: 100, output_tokens: 50 }, + // eslint-disable-next-line @typescript-eslint/no-explicit-any + } as any), + }; + + await withEnv({ ANTHROPIC_API_KEY: 'sk-test-R3b' }, async () => { + const gatewayJudge = makeJudgeClient('claude-haiku-4-5-20251001'); + __setChatTransportForTests(async (): Promise => ({ + text: 'not json at all', + blocks: [], + stopReason: 'end', + usage: { input_tokens: 100, output_tokens: 50, cache_read_tokens: 0, cache_creation_tokens: 0 }, + model: 'anthropic:claude-haiku-4-5-20251001', + providerId: 'anthropic', + })); + + const [legacyVerdict, gatewayVerdict] = await Promise.all([ + judgeSignificance(legacyJudge, FIXTURE_TRANSCRIPT, 'claude-haiku-4-5-20251001'), + judgeSignificance(gatewayJudge!, FIXTURE_TRANSCRIPT, 'claude-haiku-4-5-20251001'), + ]); + + expect(legacyVerdict.worth_processing).toBe(false); + expect(gatewayVerdict.worth_processing).toBe(false); + expect(gatewayVerdict.reasons).toEqual(legacyVerdict.reasons); + }); + }); +}); diff --git a/test/e2e/dream-synthesize-pglite.test.ts b/test/e2e/dream-synthesize-pglite.test.ts index cd9f95bf2..7cb2082bf 100644 --- a/test/e2e/dream-synthesize-pglite.test.ts +++ b/test/e2e/dream-synthesize-pglite.test.ts @@ -45,18 +45,28 @@ async function setupRig(): Promise { } /** - * Run `body` with ANTHROPIC_API_KEY temporarily cleared, restoring the - * prior value (set or unset) on return — even on throw — so this never - * leaks state to sibling test files in the suite. + * Run `body` with ANTHROPIC_API_KEY temporarily cleared AND GBRAIN_HOME + * pointed at a fresh tmpdir, restoring both on return — even on throw — so + * the developer's real ~/.gbrain/config.json never leaks the anthropic_api_key + * into the test's hasAnthropicKey() probe. Required after the v0.41 gateway- + * adapter rework: makeJudgeClient now checks BOTH env AND config file (the + * same hasAnthropicKey() pattern think/index.ts uses since v0.35.5.0), so + * clearing only the env var is insufficient hermeticity. */ async function withoutAnthropicKey(body: () => Promise): Promise { - const saved = process.env.ANTHROPIC_API_KEY; + const savedKey = process.env.ANTHROPIC_API_KEY; + const savedHome = process.env.GBRAIN_HOME; + const tmpHome = mkdtempSync(join(tmpdir(), 'gbrain-synth-isol-')); delete process.env.ANTHROPIC_API_KEY; + process.env.GBRAIN_HOME = tmpHome; try { return await body(); } finally { - if (saved === undefined) delete process.env.ANTHROPIC_API_KEY; - else process.env.ANTHROPIC_API_KEY = saved; + if (savedKey === undefined) delete process.env.ANTHROPIC_API_KEY; + else process.env.ANTHROPIC_API_KEY = savedKey; + if (savedHome === undefined) delete process.env.GBRAIN_HOME; + else process.env.GBRAIN_HOME = savedHome; + try { rmSync(tmpHome, { recursive: true, force: true }); } catch { /* */ } } } @@ -110,6 +120,60 @@ describe('E2E synthesize — empty corpus', () => { }, 30_000); }); +describe('E2E synthesize — gateway-adapter mid-run AIConfigError catch (v0.41 T5 rework)', () => { + test('AIConfigError thrown by gateway.chat is caught per-transcript; phase continues', async () => { + // Exercises the new try/catch in the verdict loop. Stubs the gateway + // chat transport to throw AIConfigError on every call (simulates a + // revoked key surfacing mid-run). The expected behavior: each + // transcript records a "gateway error: ..." reason, worth=false, and + // the phase completes with status='ok' (NOT a crash). + const { __setChatTransportForTests, resetGateway } = await import('../../src/core/ai/gateway.ts'); + const { AIConfigError } = await import('../../src/core/ai/errors.ts'); + + const rig = await setupRig(); + try { + // Make hasAnthropicKey() return true (a fake key is enough — the + // gateway transport stub throws below regardless). + const savedKey = process.env.ANTHROPIC_API_KEY; + process.env.ANTHROPIC_API_KEY = 'sk-test-mid-run-throw'; + + __setChatTransportForTests(async () => { + throw new AIConfigError('simulated mid-run provider auth failure'); + }); + + try { + await rig.engine.setConfig('dream.synthesize.enabled', 'true'); + await rig.engine.setConfig('dream.synthesize.session_corpus_dir', rig.corpusDir); + writeFileSync( + join(rig.corpusDir, '2026-04-25-mid-run.txt'), + 'a meaningful conversation\n'.repeat(200), + ); + + const result = await runPhaseSynthesize(rig.engine, { + brainDir: rig.brainDir, + dryRun: false, + }); + + // The phase did NOT throw; it converted the AIConfigError into a + // per-transcript "worth=false, reasons=['gateway error: ...']" + // verdict and moved on. + expect(result.status).toBe('ok'); + const verdicts = (result.details as { verdicts: Array<{ worth: boolean; reasons: string[] }> }).verdicts; + expect(verdicts).toHaveLength(1); + expect(verdicts[0].worth).toBe(false); + expect(verdicts[0].reasons[0]).toMatch(/gateway error:.*simulated mid-run provider auth failure/); + } finally { + if (savedKey === undefined) delete process.env.ANTHROPIC_API_KEY; + else process.env.ANTHROPIC_API_KEY = savedKey; + __setChatTransportForTests(null); + resetGateway(); + } + } finally { + await rig.cleanup(); + } + }, 30_000); +}); + describe('E2E synthesize — no API key skip path', () => { test('without ANTHROPIC_API_KEY, every transcript verdict is "no key" and zero pages written', async () => { const rig = await setupRig(); @@ -131,7 +195,11 @@ describe('E2E synthesize — no API key skip path', () => { const verdicts = (result.details as { verdicts: Array<{ worth: boolean; reasons: string[] }> }).verdicts; expect(verdicts).toHaveLength(1); expect(verdicts[0].worth).toBe(false); - expect(verdicts[0].reasons[0]).toMatch(/ANTHROPIC_API_KEY/); + // v0.41 gateway-adapter rework: reason text now names the verdict + // model so the user can see WHICH provider was missing. Pre-rework + // string was 'no ANTHROPIC_API_KEY for significance judge'; post- + // rework is 'no configured provider for verdict model: '. + expect(verdicts[0].reasons[0]).toMatch(/no configured provider for verdict model/); }); } finally { await rig.cleanup(); @@ -344,7 +412,11 @@ describe('E2E synthesize — round-trip self-consumption guard (v0.23.2)', () => // the no-key path makes it worth=false. const verdicts = (result.details as { verdicts: Array<{ worth: boolean; reasons: string[] }> }).verdicts; expect(verdicts).toHaveLength(1); - expect(verdicts[0].reasons[0]).toMatch(/ANTHROPIC_API_KEY/); + // v0.41 gateway-adapter rework: reason text now names the verdict + // model so the user can see WHICH provider was missing. Pre-rework + // string was 'no ANTHROPIC_API_KEY for significance judge'; post- + // rework is 'no configured provider for verdict model: '. + expect(verdicts[0].reasons[0]).toMatch(/no configured provider for verdict model/); // Loud warning fired at phase entry so the operator never wonders // why the guard quietly let dream output through. expect(stderr).toMatch(/\[dream\] WARNING: --unsafe-bypass-dream-guard set/); diff --git a/test/models-doctor-embed.test.ts b/test/models-doctor-embed.test.ts new file mode 100644 index 000000000..b7594ba7b --- /dev/null +++ b/test/models-doctor-embed.test.ts @@ -0,0 +1,50 @@ +import { describe, test, expect } from 'bun:test'; +import { readFileSync } from 'node:fs'; +import { join } from 'node:path'; + +/** + * v0.40.x — `gbrain models doctor` embedding reachability probe. + * + * `probeEmbeddingReachability` is an internal (non-exported) function with a + * network side effect, mirroring the existing `probeRerankerReachability` + * which has no behavioral test either. Rather than export it purely for a + * test and reconstruct the embed dim-check stub, this pins the three + * structural invariants codex flagged during plan review (the error-branch + * classification itself comes from `classifyError`, already covered): + * + * #6 — uses `embed(...)`, NOT `embedQuery` (embedQuery takes no abortSignal) + * #7 — a distinct `'embedding_reachability'` ProbeResult.touchpoint member + * #8 — gated on probeEmbeddingConfig returning 'ok' (no double-reporting) + * + * Source-text assertions, same convention as test/v0_37_gap_fill.serial.test.ts. + */ +describe('models doctor — embedding reachability probe (v0.40.x)', () => { + const src = readFileSync(join(__dirname, '..', 'src', 'commands', 'models.ts'), 'utf-8'); + + test("ProbeResult.touchpoint declares 'embedding_reachability' (codex #7)", () => { + expect(src).toContain("'embedding_reachability'"); + // Distinct from the zero-network config probe's touchpoint. + expect(src).toContain("'embedding_config'"); + }); + + test('probeEmbeddingReachability uses embed() with inputType query + abort signal (codex #6)', () => { + const fnIdx = src.indexOf('async function probeEmbeddingReachability'); + expect(fnIdx).toBeGreaterThan(0); + const slice = src.slice(fnIdx, fnIdx + 1500); + expect(slice).toContain('embed(['); + expect(slice).toContain("inputType: 'query'"); + expect(slice).toContain('abortSignal'); + // Must NOT use embedQuery (no abort signal support — codex #6). + expect(slice).not.toContain('embedQuery('); + }); + + test('runModels gates embedding reachability on config-probe ok (codex #8)', () => { + const runIdx = src.indexOf('export async function runModels'); + expect(runIdx).toBeGreaterThan(0); + const slice = src.slice(runIdx); + // The config probe result is captured and the reachability call is gated + // on its status === 'ok' before firing. + expect(slice).toContain('const embeddingConfig = await probeEmbeddingConfig()'); + expect(slice).toMatch(/embeddingConfig\.status === 'ok'[\s\S]*probeEmbeddingReachability\(\)/); + }); +}); diff --git a/test/models-doctor-reranker.test.ts b/test/models-doctor-reranker.test.ts new file mode 100644 index 000000000..1f8f0a7bb --- /dev/null +++ b/test/models-doctor-reranker.test.ts @@ -0,0 +1,99 @@ +/** + * v0.40.6.1 — gbrain models doctor reranker probe divergence fix. + * + * Pre-v0.40.6.1 the reranker probe read `getRerankerModel()` from the + * gateway, which is fed from `GBrainConfig.reranker_model` — a file-plane + * field nothing writes. Meanwhile live search resolves + * `search.reranker.model` from the DB config plane via `resolveSearchMode`. + * The two paths could disagree silently: doctor said "not configured" + * while every search call was using a mode default. + * + * `resolveLiveRerankerModel(engine)` is the new helper that reads the + * same path live search uses. These tests pin its behavior across the + * config sources mode.ts knows about. + */ + +import { describe, test, expect, afterEach } from 'bun:test'; +import { resolveLiveRerankerModel } from '../src/commands/models.ts'; +import { configureGateway, resetGateway } from '../src/core/ai/gateway.ts'; + +/** + * Minimal engine stub matching the `{ getConfig(key): Promise }` + * shape `loadSearchModeConfig` requires. Keeps the test hermetic — no + * BrainEngine, no DB, no schema. The unused engine methods would throw + * if called, surfacing any accidental hop into wider engine surface. + */ +function makeEngineStub(configMap: Record) { + return { + async getConfig(key: string): Promise { + return configMap[key] ?? null; + }, + // Any other method call should fail the test loudly. + } as any; +} + +afterEach(() => { + resetGateway(); +}); + +describe('resolveLiveRerankerModel — divergence fix', () => { + test('reads search.reranker.model from the DB plane (the path live search uses)', async () => { + configureGateway({ env: {} }); // gateway has NO reranker_model set + const engine = makeEngineStub({ + 'search.reranker.model': 'llama-server-reranker:qwen3-reranker-4b', + 'search.reranker.enabled': 'true', + }); + const resolved = await resolveLiveRerankerModel(engine); + expect(resolved).toBe('llama-server-reranker:qwen3-reranker-4b'); + }); + + test('returns the mode-bundle default when no override is set (balanced enables zerank-2)', async () => { + // balanced mode bundle has reranker_enabled: true + reranker_model: + // 'zeroentropyai:zerank-2' baked in. Pre-fix this case returned + // undefined; post-fix doctor sees what search actually uses. + configureGateway({ env: {} }); + const engine = makeEngineStub({}); + const resolved = await resolveLiveRerankerModel(engine); + expect(resolved).toBe('zeroentropyai:zerank-2'); + }); + + test('returns undefined when reranker is explicitly disabled via config', async () => { + configureGateway({ env: {} }); + const engine = makeEngineStub({ + 'search.reranker.enabled': 'false', + }); + const resolved = await resolveLiveRerankerModel(engine); + expect(resolved).toBeUndefined(); + }); + + test('config override beats the mode default', async () => { + configureGateway({ env: {} }); + const engine = makeEngineStub({ + 'search.mode': 'balanced', + 'search.reranker.model': 'llama-server-reranker:my-alias', + 'search.reranker.enabled': 'true', + }); + const resolved = await resolveLiveRerankerModel(engine); + expect(resolved).toBe('llama-server-reranker:my-alias'); + }); + + test('engine.getConfig throws per-key → still returns mode-bundle default (live search behavior)', async () => { + // Verifies the divergence fix is "graceful all the way down": if the DB + // is intermittently failing, doctor still reports what live search + // would resolve to, not undefined. `loadSearchModeConfig` swallows + // per-key getConfig errors via its internal safeGet wrapper, so the + // mode bundle default surfaces normally — doctor reports the truth + // about what would happen at search time. + configureGateway({ env: { ZEROENTROPY_API_KEY: 'sk-test' } }); + const engine = { + async getConfig(): Promise { + throw new Error('DB unreachable'); + }, + } as any; + const resolved = await resolveLiveRerankerModel(engine); + // balanced mode bundle is the safety fallback when search.mode is unset + // (and here, every config read failed) — and balanced enables + // zeroentropyai:zerank-2 by default. + expect(resolved).toBe('zeroentropyai:zerank-2'); + }); +}); diff --git a/test/search-mode.test.ts b/test/search-mode.test.ts index 77f13ff2a..a31fdc2dd 100644 --- a/test/search-mode.test.ts +++ b/test/search-mode.test.ts @@ -233,6 +233,64 @@ describe('resolveSearchMode resolution chain', () => { }); }); +describe('v0.40.6.1 — reranker_timeout_ms threads recipe default through resolution', () => { + // The dead-default-timeout-ms class of bugs: hybridSearch always passes + // resolvedMode.reranker_timeout_ms to gateway.rerank(). Pre-v0.40.6.1 the + // mode bundle's 5000ms hardcoded value always won, so recipe-level + // default_timeout_ms was dead. These tests pin the new precedence chain: + // per-call > config override > recipe touchpoint default > bundle. + + test('llama-server-reranker resolves to 30000ms recipe default (no override)', () => { + const r = resolveSearchMode({ + mode: 'balanced', + overrides: { reranker_model: 'llama-server-reranker:qwen3-reranker-4b' }, + }); + expect(r.reranker_model).toBe('llama-server-reranker:qwen3-reranker-4b'); + expect(r.reranker_timeout_ms).toBe(30_000); + }); + + test('config override beats recipe default', () => { + const r = resolveSearchMode({ + mode: 'balanced', + overrides: { + reranker_model: 'llama-server-reranker:qwen3-reranker-4b', + reranker_timeout_ms: 90_000, + }, + }); + expect(r.reranker_timeout_ms).toBe(90_000); + }); + + test('per-call override beats config override AND recipe default', () => { + const r = resolveSearchMode({ + mode: 'balanced', + overrides: { + reranker_model: 'llama-server-reranker:qwen3-reranker-4b', + reranker_timeout_ms: 90_000, + }, + perCall: { reranker_timeout_ms: 100 }, + }); + expect(r.reranker_timeout_ms).toBe(100); + }); + + test('ZE (no recipe default) regression: still gets bundle default of 5000ms', () => { + // ZeroEntropy's recipe does not declare default_timeout_ms — its hosted + // path is fast enough that the bundle default suffices. + const r = resolveSearchMode({ + mode: 'balanced', + overrides: { reranker_model: 'zeroentropyai:zerank-2' }, + }); + expect(r.reranker_timeout_ms).toBe(5000); + }); + + test('unknown provider id falls through to bundle default', () => { + const r = resolveSearchMode({ + mode: 'balanced', + overrides: { reranker_model: 'made-up-provider:fake-model' }, + }); + expect(r.reranker_timeout_ms).toBe(5000); + }); +}); + describe('attributeKnob source attribution', () => { test('per-call source labeled correctly', () => { const input = { mode: 'conservative', perCall: { tokenBudget: 999 } }; diff --git a/test/v0_37_gap_fill.serial.test.ts b/test/v0_37_gap_fill.serial.test.ts index fe634daf1..6cbf23f0e 100644 --- a/test/v0_37_gap_fill.serial.test.ts +++ b/test/v0_37_gap_fill.serial.test.ts @@ -326,17 +326,20 @@ describe('Lane E.4 — loadRecommendationContext is provider-aware', () => { // behavior via a public surface (the recommendation context the // `doctor --remediation-plan` output uses) is brittle. Use a // source-text assertion instead. - test('source-text grep: loadRecommendationContext recognizes ZE alongside OpenAI', () => { + test('source-text grep: loadRecommendationContext is provider-aware via the shared helper', () => { const src = readFileSync(join(__dirname, '..', 'src', 'commands', 'doctor.ts'), 'utf-8'); - // Pre-fix this function read DB config + OpenAI-only key check. - // Post-fix it reads gateway + branches on provider for the key. + // Pre-v0.37 this was OpenAI-only; the Lane E.4 fix made it branch on + // provider for the key. v0.40.x replaced the inline prefix ladder with the + // shared recipe-aware helper `embeddingProviderConfigured` (so doctor + + // autopilot can't drift) — assert that shape rather than the old inline + // ZE strings. const fnIdx = src.indexOf('async function loadRecommendationContext'); expect(fnIdx).toBeGreaterThan(0); const slice = src.slice(fnIdx, fnIdx + 3000); - expect(slice).toContain('ZEROENTROPY_API_KEY'); - expect(slice).toContain('zeroentropy_api_key'); - expect(slice).toContain('zeroentropyai:'); - // Reads from gateway, not DB. + // Delegates to the shared helper + the env→config key map. + expect(slice).toContain('embeddingProviderConfigured'); + expect(slice).toContain('HOSTED_EMBED_KEY_CONFIG'); + // Still reads the model from the gateway (not DB-only). expect(slice).toContain('gateway'); }); });