mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 21:19:18 +00:00
v0.31.12 fix: canonical Anthropic model IDs + tier routing surface + gbrain models CLI (#844)
* fix: canonical Anthropic model IDs + reverse alias + Opus 4.7 pricing
Replace claude-sonnet-4-6-20250929 with claude-sonnet-4-6 everywhere it
appears as a model ID. Starting with Claude 4.6, Anthropic API IDs are
dateless and pinned — the date suffix was carried forward from Sonnet 4.5
by mistake, producing a phantom ID that 404'd on every call.
Production impact in v0.31.6: isAvailable("chat") returned false in every
code path that loaded the recipe's model list, and extractFactsFromTurn
silently returned []. The headline real-time facts extraction feature
was a no-op on the happy path.
- gateway.ts:46 DEFAULT_CHAT_MODEL -> anthropic:claude-sonnet-4-6
- recipes/anthropic.ts: chat + expansion model lists drop date suffix;
remove wrong-direction alias (claude-sonnet-4-6 -> -20250929);
add reverse alias (-20250929 -> claude-sonnet-4-6) so stale user
configs in models.dream.synthesize etc. keep working
- facts/extract.ts: routes through resolveModel; both fallbacks corrected
- anthropic-pricing.ts: Opus 4.7 corrected $15/$75 -> $5/$25 per
Anthropic docs (the $15/$75 was Opus 4.0 pricing)
- cross-modal-eval/runner.ts: PRICING now reads from ANTHROPIC_PRICING
for Anthropic models instead of duplicating the map (single source of
truth — fixes the drift trap that motivated this whole patch)
Tests: cherry-pick PR #830's test/anthropic-model-ids.test.ts verbatim
(6 recipe-shape guardrails). Update gateway-chat tests to assert reverse
alias resolves correctly. Update budget-meter test for new Opus pricing.
Co-Authored-By: garrytan-agents <garrytan-agents@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: model tier system + recipe-models merge + async reconfigure hook
Add 4-tier model routing (utility/reasoning/deep/subagent) so users can
swap defaults with one config key. Each tier maps to a class of work;
override globally via models.default or per-tier via models.tier.<tier>.
Codex flagged three real architecture issues in the v0.31.12 plan review;
this commit addresses each.
F3 — sync/async timing of configureGateway:
- buildGatewayConfig stays synchronous (pre-engine-connect callers
keep working)
- New reconfigureGatewayWithEngine(engine) async function re-resolves
expansion + chat defaults through resolveModel after engine.connect()
- cli.ts wires the re-stamp into the post-connect path
F4/F5 — softening assertTouchpoint was too broad:
- Earlier plan was to flip native-recipe validation from throw to warn,
affecting gateway.chat AND gateway.expand AND gateway.embed
- Instead: per-gateway-instance recipe-models merge. assertTouchpoint
gets an optional extendedModels Set; when the user opted into a model
via config, it bypasses the throw. Source-code typos still fail fast.
- Existing contract test (test/ai/gateway-chat.test.ts:106) preserved
Tier defaults are TIER_DEFAULTS in model-config.ts. Resolution chain
inserts at step 5 (between models.default and env var). Each existing
resolveModel call site gains a tier: arg — think (deep), cycle/synthesize
(reasoning + utility for verdict), patterns/drift (reasoning), auto-think
(deep), facts/extract (reasoning).
Plus 10 new tests pinning tier precedence, subagent-tier fallback when
models.default is non-Anthropic, and the F6 alias-chain conflict case.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: subagent runtime enforcement for non-Anthropic models (3 layers)
The subagent loop uses Anthropic's Messages API with prompt caching on
system + tools. OpenAI/Google have different shapes. Setting
models.default = openai:gpt-5.5 and routing the subagent there silently
breaks the loop.
Codex F1+F2+F13 in the v0.31.12 plan review pointed out that "warn at
doctor" wasn't enough — handlers/subagent.ts:148 still did
`const model = data.model ?? DEFAULT_MODEL` and called Anthropic directly,
so a job submitted with data.model = openai:gpt-5.5 bypassed any tier
logic and failed at runtime with a confusing provider error.
Three layers of enforcement, defense in depth:
Layer 1 (queue.ts:add) — submit-time guard. When name === 'subagent'
and data.model is set, validate the provider. Non-Anthropic rejects
before the job enters the queue.
Layer 2 (handlers/subagent.ts) — tier-resolution fallback. The handler
routes through resolveModel({ tier: 'subagent' }). If the chain resolves
to a non-Anthropic provider (via models.default or models.tier.subagent),
the resolver warns + falls back to TIER_DEFAULTS.subagent
(claude-sonnet-4-6).
Layer 3 (doctor.ts:checkSubagentProvider) — surfacing layer. Warns when
models.tier.subagent or models.default is explicitly set to a
non-Anthropic provider, with a paste-ready fix command. Lets users see
config drift before submitting a job.
Tests: 3 new cases in test/agent-cli.test.ts asserting the queue-level
guard rejects non-Anthropic data.model. Existing test/subagent-handler
suite still passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: gbrain models CLI + doctor probe + silent-no-op regression test
New gbrain models CLI gives the agent and user visibility into routing.
Read mode prints the tier table, current overrides, per-task config,
and aliases with source-of-truth attribution per row. Doctor subcommand
fires a 1-token probe to each configured chat/expansion model and
classifies failures (model_not_found / auth / rate_limit / network /
unknown) so config-time invalid IDs surface without waiting for a
production call that silently degrades.
Per Codex F11 — no specific dollar cost claim in either the help text
or the CHANGELOG (providers have minimum-output billing and prompt-cache
rounding that vary). Probe is opt-in (gbrain doctor --probe-models),
never auto-runs. --skip=<provider> narrows the matrix for cost-sensitive
operators.
Per Codex F7+F8+F15 (the structural regression gap): new
test/facts-extract-silent-no-op.test.ts is THE regression test for the
bug class that motivated v0.31.12. Five cases including the smoking-gun:
when chat IS available, extractFactsFromTurn MUST actually call the chat
transport, not silently return []. Uses the gateway's
__setChatTransportForTests seam so it runs in every shard with no API key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.31.12)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: document v0.31.12 model tier system + gbrain models CLI
Add CLAUDE.md Key Files annotations for the v0.31.12 work:
src/core/model-config.ts (tier system + isAnthropicProvider + TIER_DEFAULTS),
src/core/ai/model-resolver.ts (assertTouchpoint extendedModels arg),
src/core/ai/gateway.ts (reconfigureGatewayWithEngine + extended-models registry),
src/core/minions/queue.ts (subagent submit-time guard, layer 1 of 3),
src/commands/models.ts (new gbrain models CLI + doctor probe),
src/commands/doctor.ts (subagent_provider check, layer 3 of 3),
src/core/ai/recipes/anthropic.ts (canonical model IDs + reverse alias),
src/core/anthropic-pricing.ts (Opus 4.7 corrected to \$5/\$25).
Add CLAUDE.md commands section for gbrain models + gbrain models doctor
+ power-user config recipes. Add README.md command-table rows for the
same. Regenerate llms-full.txt so the bundled docs stay in sync.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: scrub --probe-models reference (flag not actually wired)
The v0.31.12 CHANGELOG and skills/conventions/model-routing.md both
referenced `gbrain doctor --probe-models` as an integrated probe entry
point. The flag was never implemented — only `gbrain models doctor`
landed as the probe surface. Caught by /document-release subagent.
Drop the references rather than wire an untested flag at the last minute.
The probe is reachable via `gbrain models doctor`; users who want it
in doctor's output run that command separately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: garrytan-agents <garrytan-agents@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
garrytan-agents
Claude Opus 4.7
parent
0410dc4b42
commit
29961811a4
+123
@@ -2,6 +2,128 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.31.12] - 2026-05-10
|
||||
|
||||
**The chat default no longer 404s, and every Claude call gbrain makes is now one config key away from your preferred model.**
|
||||
|
||||
gbrain v0.31.6 shipped `claude-sonnet-4-6-20250929` as the chat default. That model ID does not exist on the Anthropic API. It 404'd on every call. Worse, the failure mode wasn't loud: `isAvailable("chat")` returned false in every code path that loaded the recipe's model list, and `extractFactsFromTurn` silently returned `[]`. The headline v0.31.6 feature, real-time fact extraction during sync, was a no-op on the happy path for most users.
|
||||
|
||||
This release fixes the model ID, adds a tier-based routing surface so power users can swap defaults in one config key, and ships `gbrain models doctor` as the structural probe against this exact bug class. Credit to `garrytan-agents` for the initial fix submitted as PR #830; the recipe diff and the structural test file are pulled verbatim. This release adds a reverse alias for stale user configs, the four-tier routing surface, the `gbrain models` CLI, three layers of subagent enforcement, and a real regression test for the silent-no-op bug class.
|
||||
|
||||
### What you can now do
|
||||
|
||||
**Use any model for everything with one config key.** `gbrain config set models.default opus` routes every internal call (chat, expansion, synthesis, classification) through Opus 4.7. Switch to `models.default openai:gpt-5.5` and gbrain will route every call to GPT-5.5, except for the subagent loop, which falls back to `claude-sonnet-4-6` automatically because the loop is Anthropic-only by construction.
|
||||
|
||||
**Tier-level control when you want finer than "use opus for everything."** Four tiers: `utility` (haiku-class, fast classification + expansion + verdict), `reasoning` (sonnet-class, default chat + synthesis), `deep` (opus-class, expensive reasoning), `subagent` (Anthropic-only multi-turn tool loop). Override each independently via `gbrain config set models.tier.<tier> <model>`. Per-task keys like `models.dream.synthesize` still beat tier overrides because they are more specific.
|
||||
|
||||
**`gbrain models` shows the live routing.** Read mode prints the four tier defaults, the resolved value for each, every per-task override, the alias map, and a source-of-truth column showing exactly where each model came from (`default`, `config: models.tier.reasoning`, `env: GBRAIN_MODEL`). `--json` for machine-readable output.
|
||||
|
||||
**`gbrain models doctor` probes each configured model with a 1-token call** and reports reachability with the provider's error message. This is the structural fix for the bug class that produced this release: an invalid model ID surfaces at config time, not at the next agent-run that silently degrades.
|
||||
|
||||
**Subagent jobs reject non-Anthropic models at submit time.** The minion queue's `add()` method now refuses `subagent` jobs with `data.model` pointing at a non-Anthropic provider. Three layers of enforcement: submit-time guard, tier-resolution fallback in `resolveModel`, and a `subagent_provider` check in `gbrain doctor`. You cannot accidentally route the Anthropic Messages API tool-loop to OpenAI and watch it fail at runtime.
|
||||
|
||||
**Opus 4.7 pricing is correct** (`$5/$25 per MTok`, was `$15/$75` left over from Opus 4). The budget meter and cross-modal-eval estimator both read from `src/core/anthropic-pricing.ts` as the single source of truth now, not two duplicated maps.
|
||||
|
||||
### Numbers that matter
|
||||
|
||||
| Surface | Before (v0.31.6) | After (v0.31.12) |
|
||||
|---|---|---|
|
||||
| Chat default model ID | `claude-sonnet-4-6-20250929` (404s) | `claude-sonnet-4-6` (valid) |
|
||||
| `isAvailable("chat")` on default install | false (silently) | true |
|
||||
| `extractFactsFromTurn` on default install | returns `[]` silently | actually extracts facts |
|
||||
| Config keys to swap models everywhere | n/a (hardcoded per call) | 1 (`models.default`) |
|
||||
| Provider-mismatch enforcement layers for subagent | 0 | 3 (submit/runtime/doctor) |
|
||||
| Reachability check at config time | n/a | `gbrain models doctor` |
|
||||
| Sources of truth for Anthropic pricing | 2 (duplicated, diverged) | 1 (`ANTHROPIC_PRICING`) |
|
||||
|
||||
### What this means for you
|
||||
|
||||
If you upgraded to v0.31.6 hoping for real-time facts extraction during sync and noticed nothing got extracted, this is the fix. Run `gbrain upgrade && gbrain doctor` and the chat-default check returns to green. The new `subagent_provider` check surfaces any non-Anthropic config drift.
|
||||
|
||||
If you are a power user, the boring path stays boring: `gbrain config set models.default opus` and you're done. The advanced path is `gbrain models` to see what's live, `gbrain models doctor` to probe reachability before a long agent run, and `models.tier.*` keys for tier-specific overrides. The skill at `skills/conventions/model-routing.md` documents the full precedence chain and override recipes.
|
||||
|
||||
If you had a `claude-sonnet-4-6-20250929` string sitting in `facts.extraction_model` or `models.dream.synthesize` from v0.31.6, you do not need to update it. A reverse alias rewrites that broken ID back to the canonical `claude-sonnet-4-6` automatically. If you want to clean it up: `gbrain config set facts.extraction_model anthropic:claude-sonnet-4-6`.
|
||||
|
||||
## To take advantage of v0.31.12
|
||||
|
||||
`gbrain upgrade` should do this automatically. If it didn't, or if `gbrain doctor` warns about the chat default:
|
||||
|
||||
1. **Verify the routing is healthy:**
|
||||
```bash
|
||||
gbrain doctor
|
||||
gbrain models
|
||||
```
|
||||
The `subagent_provider` check should be `ok`. `gbrain models` should show `tier.reasoning` resolving to `claude-sonnet-4-6` (or your override).
|
||||
|
||||
2. **Optional: probe reachability of every configured model.** ~1 token per model, runs in under 5 seconds. Catches the next 404 before it ships.
|
||||
```bash
|
||||
gbrain models doctor
|
||||
```
|
||||
|
||||
3. **Power-user override (any one of these):**
|
||||
```bash
|
||||
# Use opus for everything
|
||||
gbrain config set models.default opus
|
||||
|
||||
# Use sonnet for the default workhorse but opus for deep reasoning
|
||||
gbrain config set models.tier.deep opus
|
||||
|
||||
# Use a custom alias as the global default
|
||||
gbrain config set models.aliases.frontier anthropic:claude-opus-4-7
|
||||
gbrain config set models.default frontier
|
||||
```
|
||||
|
||||
4. **If `gbrain doctor` still flags anything,** please file an issue: https://github.com/garrytan/gbrain/issues — include the output of `gbrain doctor` and `gbrain models --json`.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
#### Bug fix sweep + reverse alias
|
||||
|
||||
- `src/core/ai/gateway.ts`: `DEFAULT_CHAT_MODEL` corrected to `anthropic:claude-sonnet-4-6` (was the v0.31.6 phantom `-20250929`).
|
||||
- `src/core/ai/recipes/anthropic.ts`: chat and expansion `models:` lists drop the phantom date suffix. The wrong-direction alias (`claude-sonnet-4-6 → claude-sonnet-4-6-20250929`) is removed; a reverse alias (`claude-sonnet-4-6-20250929 → claude-sonnet-4-6`) keeps stale user configs working.
|
||||
- `src/core/facts/extract.ts`: routes through `resolveModel({tier: 'reasoning'})` instead of hardcoding the broken default.
|
||||
- `test/anthropic-model-ids.test.ts`: 6 structural test cases pinning the recipe shape (verbatim cherry-pick of PR #830, plus the reverse-alias rescue case).
|
||||
|
||||
#### Tier system + recipe-models merge
|
||||
|
||||
- `src/core/model-config.ts`: new `ModelTier` type, new `TIER_DEFAULTS` constant, new `tier?` field on `ResolveModelOpts`. Tier resolution is step 5 in the resolution chain (after `models.default`, before env var). Tier defaults win over caller fallback when no override is set.
|
||||
- `src/core/ai/model-resolver.ts`: `assertTouchpoint()` gains an optional 4th arg `extendedModels` so per-gateway-instance recipe extension can permit user-supplied model strings without breaking the source-code-typo failure path. Replaces the earlier plan to soften the validator wholesale, which Codex flagged as too broad.
|
||||
- `src/core/ai/gateway.ts`: new `reconfigureGatewayWithEngine(engine)` async function called from `cli.ts` after `engine.connect()` to re-resolve expansion + chat defaults through `resolveModel()`. Configured models register into a per-instance extended-models set so the gateway routes them without recipe modification.
|
||||
- Every existing `resolveModel()` call site gains a `tier:` argument: think → deep, cycle/synthesize → reasoning (+ utility for verdict), cycle/patterns → reasoning, cycle/drift → reasoning, cycle/auto-think → deep, facts/extract → reasoning.
|
||||
|
||||
#### Subagent runtime enforcement (3 layers)
|
||||
|
||||
- Layer 1 (`src/core/minions/queue.ts`): `MinionQueue.add()` rejects `subagent` jobs with a non-Anthropic `data.model` before the job enters the queue.
|
||||
- Layer 2 (`src/core/minions/handlers/subagent.ts`): handler routes through `resolveModel({tier: 'subagent'})`. If config resolves to a non-Anthropic provider, the resolver warns and falls back to `TIER_DEFAULTS.subagent` (`claude-sonnet-4-6`).
|
||||
- Layer 3 (`src/commands/doctor.ts`): new `subagent_provider` check warns when `models.tier.subagent` or `models.default` resolves to a non-Anthropic provider, with a paste-ready fix command.
|
||||
|
||||
#### `gbrain models` CLI + probe
|
||||
|
||||
- `src/commands/models.ts`: read mode prints routing table + source attribution. `doctor` subcommand fires a 1-token `gateway.chat()` probe to each configured chat/expansion model and classifies failures into `{model_not_found, auth, rate_limit, network, unknown}`. `--skip=<provider>` narrows the probe matrix.
|
||||
- Wired into the cli.ts dispatch table and `CLI_ONLY` set.
|
||||
|
||||
#### Pricing dedupe
|
||||
|
||||
- `src/core/anthropic-pricing.ts`: Opus 4.7 corrected from `$15/$75` to `$5/$25` per Anthropic's published pricing (the old number was from Opus 4 generation). Opus 4.6 also corrected.
|
||||
- `src/core/cross-modal-eval/runner.ts`: pricing map now reads from `ANTHROPIC_PRICING` for Anthropic models instead of duplicating the values. Single source of truth.
|
||||
|
||||
#### Tests
|
||||
|
||||
- `test/facts-extract-silent-no-op.test.ts` (NEW, 5 cases): the regression test for the bug class. Wires gateway with the v0.31.6 broken model ID and asserts the reverse alias rescues `isAvailable("chat")`. Also asserts the smoking-gun: when chat is available, `extractFactsFromTurn` actually calls the chat transport (does not silently return `[]`).
|
||||
- `test/model-config.serial.test.ts`: 9 new cases covering tier precedence, `models.default` beats tier, `tier.subagent` falls back to Anthropic when default is non-Anthropic, alias-chain conflict (Codex F6 regression guard).
|
||||
- `test/agent-cli.test.ts`: 3 new cases for the queue submit-time subagent guard (Layer 1).
|
||||
- `test/ai/gateway-chat.test.ts`: existing alias-resolution tests rewritten to reflect the new direction (4.6+ dateless is canonical, reverse alias rescues stale strings).
|
||||
- `test/anthropic-model-ids.test.ts` (NEW, 6 cases): recipe-shape guardrails, verbatim cherry-pick of PR #830 plus reverse-alias case.
|
||||
|
||||
#### Skills + docs
|
||||
|
||||
- `skills/conventions/model-routing.md`: rewritten to cover both the new tier system AND the existing subagent spawn routing in one canonical doc. Power-user recipes, three-layer subagent enforcement explanation, override priority chain.
|
||||
|
||||
### For contributors
|
||||
|
||||
- **Iron rule when adding a new LLM call:** route through `resolveModel()` with a `tier:` argument. Do not hardcode a model string as a fallback. The v0.31.6 chat-default failure mode was exactly this — a hardcoded const fallback bypassed the resolver and shipped a phantom ID. The tier system + `gbrain models doctor` is the structural fix; future drift will be caught by the recipe-shape test in `test/anthropic-model-ids.test.ts`.
|
||||
- **The `gbrain models doctor` probe is opt-in.** Do not auto-fire it on every CLI invocation; that would burn tokens for users who just want `gbrain put-page`. Operators run it explicitly when they suspect a config issue or after upgrading.
|
||||
|
||||
## [0.31.11] - 2026-05-10
|
||||
|
||||
**Thin-client gbrain notices when the remote brain has upgraded and offers to bring you along.**
|
||||
@@ -261,6 +383,7 @@ After upgrade, anything that was `[WARN] graph_coverage` or `[WARN] resolver_hea
|
||||
- Contributors: [@mayazbay](https://github.com/mayazbay), [@psperera](https://github.com/psperera), [@FUSED-ID](https://github.com/FUSED-ID), [@TheAndersMadsen](https://github.com/TheAndersMadsen). Outside-voice plan review (codex) caught the read-path/write-path split risk on the original plan; a second codex pass during /ship caught the `--fix` write-path leak before merge.
|
||||
|
||||
[#798](https://github.com/garrytan/gbrain/pull/798) [#788](https://github.com/garrytan/gbrain/pull/788) [#536](https://github.com/garrytan/gbrain/pull/536) [#376](https://github.com/garrytan/gbrain/pull/376) [#128](https://github.com/garrytan/gbrain/pull/128)
|
||||
|
||||
## [0.31.4.1] - 2026-05-10
|
||||
|
||||
**Takes v2: lessons from a 100K-take production extraction folded back into the runtime, and `gbrain auth` works on Postgres again.**
|
||||
|
||||
@@ -88,6 +88,14 @@ strict behavior when unset.
|
||||
- `src/core/ai/types.ts` — provider/recipe types. **v0.28.7 (#680):** `EmbeddingTouchpoint` extended with optional `chars_per_token` (default 4 chars/token, matching OpenAI tiktoken on English) and `safety_factor` (default 0.8, budget-utilization ceiling). Both consulted only when `max_batch_tokens` is also set. Voyage declares `chars_per_token=1` + `safety_factor=0.5` to handle dense payloads (CJK/JSON/base64) that overshoot tiktoken. The pre-split budget is `max_batch_tokens × safety_factor / chars_per_token`. **v0.28.11 (#719):** `EmbeddingTouchpoint.multimodal_models?: string[]` model-level allow-list for recipes that mix text-only + multimodal models under one touchpoint (Voyage's 12 models share `supports_multimodal: true` but only `voyage-multimodal-3` accepts `/multimodalembeddings`). When omitted, recipe-level `supports_multimodal` is sufficient. `AIGatewayConfig.embedding_multimodal_model?: string` lets `embedMultimodal()` route to a different model than `embedding_model` — brains using OpenAI for text can use Voyage for images without flipping the primary embedding pipeline.
|
||||
- `src/core/ai/gateway.ts` — unified seam for every AI call. **v0.28.7 (#680):** module-scoped `_embedTransport` defaulting to AI SDK `embedMany`, with `__setEmbedTransportForTests(fn)` test seam so tests drive the public `embed()` function with a stubbed transport instead of probing private helpers. `splitByTokenBudget` and `isTokenLimitError` are now exported `@internal` — pure functions reused directly by the test file. Module-level `_shrinkState: Map<recipeId, {factor, consecutiveSuccesses}>` halves the recipe's effective `safety_factor` on token-limit miss (floor 0.05) and heals back ×1.5 toward the ceiling after `SHRINK_HEAL_AFTER=10` consecutive successes. `configureGateway()` walks every registered recipe at construction time and emits a once-per-process stderr warning for any embedding touchpoint missing `max_batch_tokens` (excluding the canonical OpenAI fast-path recipe). `resetGateway()` clears `_shrinkState`, the warned-set, and restores the real transport. ASCII flow diagram embedded in the `embed()` JSDoc covers the routing decision, recursion + halving, and shrinkState lifecycle. **v0.28.11 (#719):** `embedMultimodal()` reads `cfg.embedding_multimodal_model` first (falls back to `cfg.embedding_model` for single-model setups). After the existing recipe-level `supports_multimodal` fast-fail, validates the resolved model against `touchpoint.multimodal_models` when declared — closes the Voyage-text-only-model-into-multimodal-endpoint footgun before any HTTP call (Codex F1 from PR review). New `getMultimodalModel()` accessor mirrors `getEmbeddingModel` / `getChatModel` so doctor and integration tests can read the gateway state.
|
||||
- `src/core/ai/recipes/voyage.ts` — Voyage AI openai-compatible recipe. **v0.28.7 (#680):** declares `chars_per_token=1` + `safety_factor=0.5` so the gateway pre-splits Voyage batches at a 60K-character budget (50% of 120K-token cap with the dense-tokenizer ratio). Closes the v0.27 backfill loop where ~26% of the corpus stayed un-embedded because tiktoken-grounded budgeting silently undercounted Voyage's actual token usage. **v0.28.11 (#719):** declares `multimodal_models: ['voyage-multimodal-3']` so the gateway rejects text-only Voyage models pointed at the multimodal endpoint with a clear `AIConfigError` instead of waiting for Voyage's HTTP 400.
|
||||
- `src/core/ai/recipes/anthropic.ts` — Anthropic recipe (chat + expansion touchpoints). **v0.31.12:** chat and expansion `models:` lists drop the v0.31.6 phantom `claude-sonnet-4-6-20250929` date suffix — canonical id is `claude-sonnet-4-6`. The wrong-direction alias `claude-sonnet-4-6 → claude-sonnet-4-6-20250929` is removed; a reverse alias `claude-sonnet-4-6-20250929 → claude-sonnet-4-6` keeps stale user configs working (rescues `facts.extraction_model` and `models.dream.synthesize` set by v0.31.6 installs). Recipe-shape regression pinned by `test/anthropic-model-ids.test.ts` (6 cases, verbatim cherry-pick of PR #830 plus the reverse-alias rescue case).
|
||||
- `src/core/anthropic-pricing.ts` — Single source of truth for Anthropic model pricing (per-MTok input/output). **v0.31.12:** Opus 4.7 corrected from `$15/$75` to `$5/$25` (the old number was from Opus 4 generation, never refreshed when 4.7 shipped); Opus 4.6 also corrected. Consumed by `src/core/budget-meter.ts` and `src/core/cross-modal-eval/runner.ts` — the cross-modal estimator now reads `ANTHROPIC_PRICING` for Anthropic models instead of duplicating the table, killing the v0.31.6 drift bug class.
|
||||
- `src/core/model-config.ts` — Model-string resolution (the seam every internal LLM call walks through). **v0.31.12:** four-tier system (`ModelTier = 'utility' | 'reasoning' | 'deep' | 'subagent'`) with `TIER_DEFAULTS` (utility→haiku-4-5, reasoning→sonnet-4-6, deep→opus-4-7, subagent→sonnet-4-6) and `tier?: ModelTier` on `ResolveModelOpts`. Resolution chain is now 8 steps: cliFlag → deprecated key → config key → `models.default` → `models.tier.<tier>` → env var → `TIER_DEFAULTS[tier]` → caller fallback. Two new exports — `isAnthropicProvider(modelString)` checks `provider:model` prefix OR `claude-` bare-id pattern, and `enforceSubagentAnthropic()` is the layer-2 runtime guard: when `tier === 'subagent'` resolves to a non-Anthropic provider, it emits a once-per-`(source, model)` stderr warn AND falls back to `TIER_DEFAULTS.subagent` instead of letting the Anthropic Messages API tool-loop attempt to run on OpenAI/Gemini. `_resetDeprecationWarningsForTest()` now also clears `_subagentTierWarningsEmitted` so tests re-emit.
|
||||
- `src/core/ai/model-resolver.ts` — Recipe-touchpoint validator. **v0.31.12:** `assertTouchpoint(recipe, touchpoint, modelId, extendedModels?)` gains an optional 4th `extendedModels: ReadonlySet<string>` argument. When the modelId is in that set, the native-recipe allowlist throw is bypassed — the user explicitly opted into this model via config so we let provider rejection surface as `model_not_found` at HTTP call time (and `gbrain models doctor` catches it earlier). Default code paths with hardcoded model strings MUST NOT pass `extendedModels` — typos in source code still fail fast. Replaces the earlier plan to soften the validator wholesale (Codex F4/F5 in plan review flagged that as too broad — it would have removed the fail-fast contract for chat + expand + embed all three).
|
||||
- `src/core/ai/gateway.ts` extension (v0.31.12) — new module-scoped `_extendedModels: Map<providerId, Set<modelId>>` registry feeds `assertTouchpoint`'s 4th-arg path. New `reconfigureGatewayWithEngine(engine)` async function is called from `cli.ts` after `engine.connect()` (and before every command except `CLI_ONLY` no-DB commands) — re-resolves expansion + chat defaults through `resolveModel()` so `models.tier.*` and `models.default` overrides apply to expansion + chat both. `DEFAULT_CHAT_MODEL` corrected to `anthropic:claude-sonnet-4-6` (was the v0.31.6 phantom `-20250929`). New `__setChatTransportForTests` seam mirrors `__setEmbedTransportForTests` so tests drive `chat()` with a stubbed transport.
|
||||
- `src/core/minions/queue.ts` extension (v0.31.12) — `MinionQueue.add()` now rejects `subagent` jobs whose `data.model` resolves through `isAnthropicProvider()` to a non-Anthropic provider. Lazy-imports `model-config.ts` to avoid pulling engine types into queue's eager-load surface. Layer 1 of the three-layer subagent provider enforcement (Codex F1+F2 in plan review). Layers 2 + 3 live in `src/core/model-config.ts` (`enforceSubagentAnthropic` runtime fallback) and `src/commands/doctor.ts` (`subagent_provider` check). Pinned by 3 cases in `test/agent-cli.test.ts`.
|
||||
- `src/commands/models.ts` (v0.31.12) — `gbrain models [--json]` read-only routing dashboard: prints tier defaults (`utility`/`reasoning`/`deep`/`subagent`), the resolved value for each (re-walking the resolution chain to attribute properly), every per-task override (11 `PER_TASK_KEYS` entries — `models.dream.synthesize`, `models.dream.patterns`, `models.drift`, `models.auto_think`, `models.think`, `models.subagent`, `facts.extraction_model`, `models.eval.longmemeval`, `models.expansion`, `models.chat`, `models.dream.synthesize_verdict`), the alias map (defaults + user overrides), and a source-of-truth column showing `default` / `config: <key>` / `env: <VAR>`. `gbrain models doctor [--skip=<provider>] [--json]` fires a 1-token `gateway.chat()` probe against each configured chat + expansion model and classifies failures into `{model_not_found, auth, rate_limit, network, unknown}` — the structural fix for the v0.31.6 silent-no-op bug class. Wired into `cli.ts` dispatch table + `CLI_ONLY` set.
|
||||
- `src/commands/doctor.ts` extension (v0.31.12) — new `subagent_provider` check (layer 3 of 3 — Codex F13). Warns when `models.tier.subagent` is explicitly set to a non-Anthropic provider (fail-loud since the user clearly meant it — message names the bad value and prints the paste-ready fix command `gbrain config set models.tier.subagent anthropic:claude-sonnet-4-6`); also warns when `models.default` would sneak `subagent` into a non-Anthropic provider via tier inheritance. OK status when subagent tier resolves to Anthropic. Tests cover all three paths in `test/doctor.test.ts`.
|
||||
- `src/core/check-resolvable.ts` — Resolver validation: reachability, MECE overlap, DRY checks, structured fix objects. v0.14.1: `CROSS_CUTTING_PATTERNS.conventions` is an array (notability gate accepts both `conventions/quality.md` and `_brain-filing-rules.md`). New `extractDelegationTargets()` parses `> **Convention:**`, `> **Filing rule:**`, and inline backtick references. DRY suppression is proximity-based via `DRY_PROXIMITY_LINES = 40`.
|
||||
- `src/core/repo-root.ts` — Shared `findRepoRoot(startDir?)` (v0.16.4): walks up from `startDir` (default `process.cwd()`) looking for `skills/RESOLVER.md`. Zero-dependency module imported by both `doctor.ts` and `check-resolvable.ts`. Parameterized `startDir` makes tests hermetic. **v0.31.7:** read-path / write-path split. `autoDetectSkillsDir` (shared, read+write-safe) gains tier-0 `$GBRAIN_SKILLS_DIR` explicit operator override (Docker mounts, CI, monorepo subdirs) ahead of the existing 4-tier chain. New `autoDetectSkillsDirReadOnly` wraps it with a tier-5 install-path fallback that walks up from `fileURLToPath(import.meta.url)` and gates on `isGbrainRepoRoot` so unrelated repos can't false-positive. Read-path callers (`doctor`, `check-resolvable`, `routing-eval`) use the read-only variant; write-path callers (`skillpack install`, `skillify scaffold`, `post-install-advisory`) deliberately stay on the shared function so `gbrain skillpack install` from `~` cannot silently retarget the bundled gbrain repo's `skills/` instead of the user's actual workspace. Two new `SkillsDirSource` variants: `'env_explicit'`, `'install_path'`. New `AUTO_DETECT_HINT_READ_ONLY` documents the extra tier. The D6 `--fix` safety gate in `doctor.ts` + `check-resolvable.ts` refuses auto-repair when `detected.source === 'install_path'` so `gbrain doctor --fix` from `~` cannot silently rewrite the bundled install tree.
|
||||
- `src/commands/check-resolvable.ts` — Standalone CLI wrapper (v0.16.4) over `checkResolvable()`. Exports `parseFlags`, `resolveSkillsDir`, `DEFERRED`, `runCheckResolvable`. Exit rule: **1 on any issue (warnings OR errors)**, stricter than doctor's `ok` flag — honors README:259. Stable JSON envelope `{ok, skillsDir, report, autoFix, deferred, error, message}` — same shape on success and error paths. `--fix` path runs `autoFixDryViolations` BEFORE `checkResolvable` (same ordering as doctor). `scripts/skillify-check.ts` subprocess-calls `gbrain check-resolvable --json` (cached per process) and fails loud on binary-missing — no silent false-pass. **v0.19:** AGENTS.md workspaces now resolve natively (see `src/core/resolver-filenames.ts`) — gbrain inspects the 107-skill OpenClaw deployment whether the routing file is `RESOLVER.md` or `AGENTS.md`. `DEFERRED[]` is empty — Checks 5 + 6 shipped as real code, not issue URLs. **v0.31.7:** the resolver lookup switched from first-match-wins to the multi-file merge in `src/core/check-resolvable.ts` — entries collected from every `RESOLVER.md` / `AGENTS.md` across the skills dir AND its parent, deduped by `skillPath` (first occurrence wins). Lifted reachable skills on the reference OpenClaw layout from 37/224 to 200/224 — the deployment ships a thin `skills/RESOLVER.md` (~40 entries from skillpack) plus a fat `../AGENTS.md` (200+ entries, the real dispatcher), and the previous code only saw the first one. The CLI also switched to `autoDetectSkillsDirReadOnly` so `cd ~ && gbrain check-resolvable` finds the bundled skills via the install-path fallback. `--fix` carries the same D6 safety gate as `gbrain doctor --fix`: refuses to write when `detected.source === 'install_path'`.
|
||||
@@ -325,6 +333,17 @@ Key commands added for Minions (job queue):
|
||||
- `gbrain jobs smoke [--sigkill-rescue]` — health smoke test. `--sigkill-rescue` is the v0.13.1 regression guard for #219: simulates a killed worker and asserts the stalled job is requeued instead of dead-lettered on first stall.
|
||||
- `gbrain jobs work [--queue Q] [--concurrency N]` — start worker daemon (Postgres only)
|
||||
|
||||
Key commands added in v0.31.12 (model tier system + routing CLI):
|
||||
- `gbrain models [--json]` — read-only routing dashboard. Prints the four tier defaults (`utility`/`reasoning`/`deep`/`subagent`), the resolved value for each (after re-walking `models.default` → `models.tier.<tier>` → env → `TIER_DEFAULTS`), every per-task override (`models.dream.synthesize`, `models.dream.patterns`, `models.drift`, `models.auto_think`, `models.think`, `models.subagent`, `facts.extraction_model`, `models.eval.longmemeval`, `models.expansion`, `models.chat`, `models.dream.synthesize_verdict`), the alias map (defaults + user overrides), and a source-of-truth column (`default` / `config: <key>` / `env: <VAR>`).
|
||||
- `gbrain models doctor [--skip=<provider>] [--json]` — 1-token reachability probe against each configured chat + expansion model. Classifies failures into `{model_not_found, auth, rate_limit, network, unknown}`. The structural fix for the bug class that motivated v0.31.12 (v0.31.6's `claude-sonnet-4-6-20250929` chat default 404'd silently on every install).
|
||||
- Power-user model routing via config keys:
|
||||
- `gbrain config set models.default opus` — route every internal call (chat, expansion, synthesis, classification) through Opus 4.7. Subagent loop still falls back to `claude-sonnet-4-6` automatically (Anthropic-only by construction).
|
||||
- `gbrain config set models.tier.<tier> <model>` — override one tier independently (`utility` / `reasoning` / `deep` / `subagent`).
|
||||
- `gbrain config set models.aliases.frontier anthropic:claude-opus-4-7` — define an alias, then `gbrain config set models.default frontier`.
|
||||
- Per-task keys (e.g. `gbrain config set models.dream.synthesize <model>`) still beat tier overrides because they are more specific.
|
||||
- New `subagent_provider` check in `gbrain doctor` surfaces config drift if `models.tier.subagent` or `models.default` would route the Anthropic Messages API tool-loop to a non-Anthropic provider.
|
||||
- The skill at `skills/conventions/model-routing.md` was rewritten to cover both the new tier system AND the existing subagent spawn routing in one canonical doc (power-user recipes, three-layer enforcement explanation, override priority chain).
|
||||
|
||||
Key commands added in v0.28.1 (LongMemEval in the box):
|
||||
- `gbrain eval longmemeval <dataset.jsonl>` — run the public LongMemEval benchmark against gbrain hybrid retrieval. Flags: `--limit N`, `--model M`, `--retrieval-only`, `--keyword-only`, `--expansion`, `--top-k K`, `--output FILE`. One in-memory PGLite per benchmark run; `TRUNCATE` between questions over runtime-enumerated `pg_tables` (schema-migration-safe); `~/.gbrain` never opened. `--expansion` defaults OFF (deterministic, no per-query Haiku). Default model resolves through `resolveModel()` 6-tier chain with new `models.eval.longmemeval` config key. `gbrain eval longmemeval --help` works without a configured brain (hermeticity gate).
|
||||
- Sanitization parity with takes: `INJECTION_PATTERNS` exported from `src/core/think/sanitize.ts`. The benchmark harness re-uses the same pattern set so adding a new injection pattern automatically covers takes AND benchmarks.
|
||||
|
||||
@@ -748,6 +748,16 @@ ADMIN
|
||||
gbrain doctor --fix [--dry-run] Auto-fix DRY violations (delegate inlined rules to conventions)
|
||||
gbrain doctor --locks List idle-in-tx backends (57014 diagnostic, Postgres only)
|
||||
gbrain stats Brain statistics
|
||||
gbrain models Show live model routing (tier defaults,
|
||||
per-task overrides, alias map, source-of-truth).
|
||||
v0.31.12: tier system + recipe-models merge.
|
||||
Power-user override:
|
||||
gbrain config set models.default opus
|
||||
gbrain config set models.tier.deep opus
|
||||
gbrain models doctor 1-token reachability probe for each configured
|
||||
chat/expansion model. Catches `model_not_found`
|
||||
before the next agent run silently degrades.
|
||||
[--skip=<provider>] [--json]
|
||||
gbrain serve MCP server (stdio)
|
||||
gbrain serve --http [--port 3131] HTTP MCP server with OAuth 2.1 + admin dashboard
|
||||
[--token-ttl 3600] [--enable-dcr]
|
||||
|
||||
@@ -188,6 +188,14 @@ strict behavior when unset.
|
||||
- `src/core/ai/types.ts` — provider/recipe types. **v0.28.7 (#680):** `EmbeddingTouchpoint` extended with optional `chars_per_token` (default 4 chars/token, matching OpenAI tiktoken on English) and `safety_factor` (default 0.8, budget-utilization ceiling). Both consulted only when `max_batch_tokens` is also set. Voyage declares `chars_per_token=1` + `safety_factor=0.5` to handle dense payloads (CJK/JSON/base64) that overshoot tiktoken. The pre-split budget is `max_batch_tokens × safety_factor / chars_per_token`. **v0.28.11 (#719):** `EmbeddingTouchpoint.multimodal_models?: string[]` model-level allow-list for recipes that mix text-only + multimodal models under one touchpoint (Voyage's 12 models share `supports_multimodal: true` but only `voyage-multimodal-3` accepts `/multimodalembeddings`). When omitted, recipe-level `supports_multimodal` is sufficient. `AIGatewayConfig.embedding_multimodal_model?: string` lets `embedMultimodal()` route to a different model than `embedding_model` — brains using OpenAI for text can use Voyage for images without flipping the primary embedding pipeline.
|
||||
- `src/core/ai/gateway.ts` — unified seam for every AI call. **v0.28.7 (#680):** module-scoped `_embedTransport` defaulting to AI SDK `embedMany`, with `__setEmbedTransportForTests(fn)` test seam so tests drive the public `embed()` function with a stubbed transport instead of probing private helpers. `splitByTokenBudget` and `isTokenLimitError` are now exported `@internal` — pure functions reused directly by the test file. Module-level `_shrinkState: Map<recipeId, {factor, consecutiveSuccesses}>` halves the recipe's effective `safety_factor` on token-limit miss (floor 0.05) and heals back ×1.5 toward the ceiling after `SHRINK_HEAL_AFTER=10` consecutive successes. `configureGateway()` walks every registered recipe at construction time and emits a once-per-process stderr warning for any embedding touchpoint missing `max_batch_tokens` (excluding the canonical OpenAI fast-path recipe). `resetGateway()` clears `_shrinkState`, the warned-set, and restores the real transport. ASCII flow diagram embedded in the `embed()` JSDoc covers the routing decision, recursion + halving, and shrinkState lifecycle. **v0.28.11 (#719):** `embedMultimodal()` reads `cfg.embedding_multimodal_model` first (falls back to `cfg.embedding_model` for single-model setups). After the existing recipe-level `supports_multimodal` fast-fail, validates the resolved model against `touchpoint.multimodal_models` when declared — closes the Voyage-text-only-model-into-multimodal-endpoint footgun before any HTTP call (Codex F1 from PR review). New `getMultimodalModel()` accessor mirrors `getEmbeddingModel` / `getChatModel` so doctor and integration tests can read the gateway state.
|
||||
- `src/core/ai/recipes/voyage.ts` — Voyage AI openai-compatible recipe. **v0.28.7 (#680):** declares `chars_per_token=1` + `safety_factor=0.5` so the gateway pre-splits Voyage batches at a 60K-character budget (50% of 120K-token cap with the dense-tokenizer ratio). Closes the v0.27 backfill loop where ~26% of the corpus stayed un-embedded because tiktoken-grounded budgeting silently undercounted Voyage's actual token usage. **v0.28.11 (#719):** declares `multimodal_models: ['voyage-multimodal-3']` so the gateway rejects text-only Voyage models pointed at the multimodal endpoint with a clear `AIConfigError` instead of waiting for Voyage's HTTP 400.
|
||||
- `src/core/ai/recipes/anthropic.ts` — Anthropic recipe (chat + expansion touchpoints). **v0.31.12:** chat and expansion `models:` lists drop the v0.31.6 phantom `claude-sonnet-4-6-20250929` date suffix — canonical id is `claude-sonnet-4-6`. The wrong-direction alias `claude-sonnet-4-6 → claude-sonnet-4-6-20250929` is removed; a reverse alias `claude-sonnet-4-6-20250929 → claude-sonnet-4-6` keeps stale user configs working (rescues `facts.extraction_model` and `models.dream.synthesize` set by v0.31.6 installs). Recipe-shape regression pinned by `test/anthropic-model-ids.test.ts` (6 cases, verbatim cherry-pick of PR #830 plus the reverse-alias rescue case).
|
||||
- `src/core/anthropic-pricing.ts` — Single source of truth for Anthropic model pricing (per-MTok input/output). **v0.31.12:** Opus 4.7 corrected from `$15/$75` to `$5/$25` (the old number was from Opus 4 generation, never refreshed when 4.7 shipped); Opus 4.6 also corrected. Consumed by `src/core/budget-meter.ts` and `src/core/cross-modal-eval/runner.ts` — the cross-modal estimator now reads `ANTHROPIC_PRICING` for Anthropic models instead of duplicating the table, killing the v0.31.6 drift bug class.
|
||||
- `src/core/model-config.ts` — Model-string resolution (the seam every internal LLM call walks through). **v0.31.12:** four-tier system (`ModelTier = 'utility' | 'reasoning' | 'deep' | 'subagent'`) with `TIER_DEFAULTS` (utility→haiku-4-5, reasoning→sonnet-4-6, deep→opus-4-7, subagent→sonnet-4-6) and `tier?: ModelTier` on `ResolveModelOpts`. Resolution chain is now 8 steps: cliFlag → deprecated key → config key → `models.default` → `models.tier.<tier>` → env var → `TIER_DEFAULTS[tier]` → caller fallback. Two new exports — `isAnthropicProvider(modelString)` checks `provider:model` prefix OR `claude-` bare-id pattern, and `enforceSubagentAnthropic()` is the layer-2 runtime guard: when `tier === 'subagent'` resolves to a non-Anthropic provider, it emits a once-per-`(source, model)` stderr warn AND falls back to `TIER_DEFAULTS.subagent` instead of letting the Anthropic Messages API tool-loop attempt to run on OpenAI/Gemini. `_resetDeprecationWarningsForTest()` now also clears `_subagentTierWarningsEmitted` so tests re-emit.
|
||||
- `src/core/ai/model-resolver.ts` — Recipe-touchpoint validator. **v0.31.12:** `assertTouchpoint(recipe, touchpoint, modelId, extendedModels?)` gains an optional 4th `extendedModels: ReadonlySet<string>` argument. When the modelId is in that set, the native-recipe allowlist throw is bypassed — the user explicitly opted into this model via config so we let provider rejection surface as `model_not_found` at HTTP call time (and `gbrain models doctor` catches it earlier). Default code paths with hardcoded model strings MUST NOT pass `extendedModels` — typos in source code still fail fast. Replaces the earlier plan to soften the validator wholesale (Codex F4/F5 in plan review flagged that as too broad — it would have removed the fail-fast contract for chat + expand + embed all three).
|
||||
- `src/core/ai/gateway.ts` extension (v0.31.12) — new module-scoped `_extendedModels: Map<providerId, Set<modelId>>` registry feeds `assertTouchpoint`'s 4th-arg path. New `reconfigureGatewayWithEngine(engine)` async function is called from `cli.ts` after `engine.connect()` (and before every command except `CLI_ONLY` no-DB commands) — re-resolves expansion + chat defaults through `resolveModel()` so `models.tier.*` and `models.default` overrides apply to expansion + chat both. `DEFAULT_CHAT_MODEL` corrected to `anthropic:claude-sonnet-4-6` (was the v0.31.6 phantom `-20250929`). New `__setChatTransportForTests` seam mirrors `__setEmbedTransportForTests` so tests drive `chat()` with a stubbed transport.
|
||||
- `src/core/minions/queue.ts` extension (v0.31.12) — `MinionQueue.add()` now rejects `subagent` jobs whose `data.model` resolves through `isAnthropicProvider()` to a non-Anthropic provider. Lazy-imports `model-config.ts` to avoid pulling engine types into queue's eager-load surface. Layer 1 of the three-layer subagent provider enforcement (Codex F1+F2 in plan review). Layers 2 + 3 live in `src/core/model-config.ts` (`enforceSubagentAnthropic` runtime fallback) and `src/commands/doctor.ts` (`subagent_provider` check). Pinned by 3 cases in `test/agent-cli.test.ts`.
|
||||
- `src/commands/models.ts` (v0.31.12) — `gbrain models [--json]` read-only routing dashboard: prints tier defaults (`utility`/`reasoning`/`deep`/`subagent`), the resolved value for each (re-walking the resolution chain to attribute properly), every per-task override (11 `PER_TASK_KEYS` entries — `models.dream.synthesize`, `models.dream.patterns`, `models.drift`, `models.auto_think`, `models.think`, `models.subagent`, `facts.extraction_model`, `models.eval.longmemeval`, `models.expansion`, `models.chat`, `models.dream.synthesize_verdict`), the alias map (defaults + user overrides), and a source-of-truth column showing `default` / `config: <key>` / `env: <VAR>`. `gbrain models doctor [--skip=<provider>] [--json]` fires a 1-token `gateway.chat()` probe against each configured chat + expansion model and classifies failures into `{model_not_found, auth, rate_limit, network, unknown}` — the structural fix for the v0.31.6 silent-no-op bug class. Wired into `cli.ts` dispatch table + `CLI_ONLY` set.
|
||||
- `src/commands/doctor.ts` extension (v0.31.12) — new `subagent_provider` check (layer 3 of 3 — Codex F13). Warns when `models.tier.subagent` is explicitly set to a non-Anthropic provider (fail-loud since the user clearly meant it — message names the bad value and prints the paste-ready fix command `gbrain config set models.tier.subagent anthropic:claude-sonnet-4-6`); also warns when `models.default` would sneak `subagent` into a non-Anthropic provider via tier inheritance. OK status when subagent tier resolves to Anthropic. Tests cover all three paths in `test/doctor.test.ts`.
|
||||
- `src/core/check-resolvable.ts` — Resolver validation: reachability, MECE overlap, DRY checks, structured fix objects. v0.14.1: `CROSS_CUTTING_PATTERNS.conventions` is an array (notability gate accepts both `conventions/quality.md` and `_brain-filing-rules.md`). New `extractDelegationTargets()` parses `> **Convention:**`, `> **Filing rule:**`, and inline backtick references. DRY suppression is proximity-based via `DRY_PROXIMITY_LINES = 40`.
|
||||
- `src/core/repo-root.ts` — Shared `findRepoRoot(startDir?)` (v0.16.4): walks up from `startDir` (default `process.cwd()`) looking for `skills/RESOLVER.md`. Zero-dependency module imported by both `doctor.ts` and `check-resolvable.ts`. Parameterized `startDir` makes tests hermetic. **v0.31.7:** read-path / write-path split. `autoDetectSkillsDir` (shared, read+write-safe) gains tier-0 `$GBRAIN_SKILLS_DIR` explicit operator override (Docker mounts, CI, monorepo subdirs) ahead of the existing 4-tier chain. New `autoDetectSkillsDirReadOnly` wraps it with a tier-5 install-path fallback that walks up from `fileURLToPath(import.meta.url)` and gates on `isGbrainRepoRoot` so unrelated repos can't false-positive. Read-path callers (`doctor`, `check-resolvable`, `routing-eval`) use the read-only variant; write-path callers (`skillpack install`, `skillify scaffold`, `post-install-advisory`) deliberately stay on the shared function so `gbrain skillpack install` from `~` cannot silently retarget the bundled gbrain repo's `skills/` instead of the user's actual workspace. Two new `SkillsDirSource` variants: `'env_explicit'`, `'install_path'`. New `AUTO_DETECT_HINT_READ_ONLY` documents the extra tier. The D6 `--fix` safety gate in `doctor.ts` + `check-resolvable.ts` refuses auto-repair when `detected.source === 'install_path'` so `gbrain doctor --fix` from `~` cannot silently rewrite the bundled install tree.
|
||||
- `src/commands/check-resolvable.ts` — Standalone CLI wrapper (v0.16.4) over `checkResolvable()`. Exports `parseFlags`, `resolveSkillsDir`, `DEFERRED`, `runCheckResolvable`. Exit rule: **1 on any issue (warnings OR errors)**, stricter than doctor's `ok` flag — honors README:259. Stable JSON envelope `{ok, skillsDir, report, autoFix, deferred, error, message}` — same shape on success and error paths. `--fix` path runs `autoFixDryViolations` BEFORE `checkResolvable` (same ordering as doctor). `scripts/skillify-check.ts` subprocess-calls `gbrain check-resolvable --json` (cached per process) and fails loud on binary-missing — no silent false-pass. **v0.19:** AGENTS.md workspaces now resolve natively (see `src/core/resolver-filenames.ts`) — gbrain inspects the 107-skill OpenClaw deployment whether the routing file is `RESOLVER.md` or `AGENTS.md`. `DEFERRED[]` is empty — Checks 5 + 6 shipped as real code, not issue URLs. **v0.31.7:** the resolver lookup switched from first-match-wins to the multi-file merge in `src/core/check-resolvable.ts` — entries collected from every `RESOLVER.md` / `AGENTS.md` across the skills dir AND its parent, deduped by `skillPath` (first occurrence wins). Lifted reachable skills on the reference OpenClaw layout from 37/224 to 200/224 — the deployment ships a thin `skills/RESOLVER.md` (~40 entries from skillpack) plus a fat `../AGENTS.md` (200+ entries, the real dispatcher), and the previous code only saw the first one. The CLI also switched to `autoDetectSkillsDirReadOnly` so `cd ~ && gbrain check-resolvable` finds the bundled skills via the install-path fallback. `--fix` carries the same D6 safety gate as `gbrain doctor --fix`: refuses to write when `detected.source === 'install_path'`.
|
||||
@@ -425,6 +433,17 @@ Key commands added for Minions (job queue):
|
||||
- `gbrain jobs smoke [--sigkill-rescue]` — health smoke test. `--sigkill-rescue` is the v0.13.1 regression guard for #219: simulates a killed worker and asserts the stalled job is requeued instead of dead-lettered on first stall.
|
||||
- `gbrain jobs work [--queue Q] [--concurrency N]` — start worker daemon (Postgres only)
|
||||
|
||||
Key commands added in v0.31.12 (model tier system + routing CLI):
|
||||
- `gbrain models [--json]` — read-only routing dashboard. Prints the four tier defaults (`utility`/`reasoning`/`deep`/`subagent`), the resolved value for each (after re-walking `models.default` → `models.tier.<tier>` → env → `TIER_DEFAULTS`), every per-task override (`models.dream.synthesize`, `models.dream.patterns`, `models.drift`, `models.auto_think`, `models.think`, `models.subagent`, `facts.extraction_model`, `models.eval.longmemeval`, `models.expansion`, `models.chat`, `models.dream.synthesize_verdict`), the alias map (defaults + user overrides), and a source-of-truth column (`default` / `config: <key>` / `env: <VAR>`).
|
||||
- `gbrain models doctor [--skip=<provider>] [--json]` — 1-token reachability probe against each configured chat + expansion model. Classifies failures into `{model_not_found, auth, rate_limit, network, unknown}`. The structural fix for the bug class that motivated v0.31.12 (v0.31.6's `claude-sonnet-4-6-20250929` chat default 404'd silently on every install).
|
||||
- Power-user model routing via config keys:
|
||||
- `gbrain config set models.default opus` — route every internal call (chat, expansion, synthesis, classification) through Opus 4.7. Subagent loop still falls back to `claude-sonnet-4-6` automatically (Anthropic-only by construction).
|
||||
- `gbrain config set models.tier.<tier> <model>` — override one tier independently (`utility` / `reasoning` / `deep` / `subagent`).
|
||||
- `gbrain config set models.aliases.frontier anthropic:claude-opus-4-7` — define an alias, then `gbrain config set models.default frontier`.
|
||||
- Per-task keys (e.g. `gbrain config set models.dream.synthesize <model>`) still beat tier overrides because they are more specific.
|
||||
- New `subagent_provider` check in `gbrain doctor` surfaces config drift if `models.tier.subagent` or `models.default` would route the Anthropic Messages API tool-loop to a non-Anthropic provider.
|
||||
- The skill at `skills/conventions/model-routing.md` was rewritten to cover both the new tier system AND the existing subagent spawn routing in one canonical doc (power-user recipes, three-layer enforcement explanation, override priority chain).
|
||||
|
||||
Key commands added in v0.28.1 (LongMemEval in the box):
|
||||
- `gbrain eval longmemeval <dataset.jsonl>` — run the public LongMemEval benchmark against gbrain hybrid retrieval. Flags: `--limit N`, `--model M`, `--retrieval-only`, `--keyword-only`, `--expansion`, `--top-k K`, `--output FILE`. One in-memory PGLite per benchmark run; `TRUNCATE` between questions over runtime-enumerated `pg_tables` (schema-migration-safe); `~/.gbrain` never opened. `--expansion` defaults OFF (deterministic, no per-query Haiku). Default model resolves through `resolveModel()` 6-tier chain with new `models.eval.longmemeval` config key. `gbrain eval longmemeval --help` works without a configured brain (hermeticity gate).
|
||||
- Sanitization parity with takes: `INJECTION_PATTERNS` exported from `src/core/think/sanitize.ts`. The benchmark harness re-uses the same pattern set so adding a new injection pattern automatically covers takes AND benchmarks.
|
||||
@@ -2549,6 +2568,16 @@ ADMIN
|
||||
gbrain doctor --fix [--dry-run] Auto-fix DRY violations (delegate inlined rules to conventions)
|
||||
gbrain doctor --locks List idle-in-tx backends (57014 diagnostic, Postgres only)
|
||||
gbrain stats Brain statistics
|
||||
gbrain models Show live model routing (tier defaults,
|
||||
per-task overrides, alias map, source-of-truth).
|
||||
v0.31.12: tier system + recipe-models merge.
|
||||
Power-user override:
|
||||
gbrain config set models.default opus
|
||||
gbrain config set models.tier.deep opus
|
||||
gbrain models doctor 1-token reachability probe for each configured
|
||||
chat/expansion model. Catches `model_not_found`
|
||||
before the next agent run silently degrades.
|
||||
[--skip=<provider>] [--json]
|
||||
gbrain serve MCP server (stdio)
|
||||
gbrain serve --http [--port 3131] HTTP MCP server with OAuth 2.1 + admin dashboard
|
||||
[--token-ttl 3600] [--enable-dcr]
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gbrain",
|
||||
"version": "0.31.11",
|
||||
"version": "0.31.12",
|
||||
"description": "Postgres-native personal knowledge brain with hybrid RAG search",
|
||||
"type": "module",
|
||||
"main": "src/core/index.ts",
|
||||
|
||||
@@ -1,8 +1,73 @@
|
||||
# Model Routing Convention
|
||||
|
||||
When spawning sub-agents, choose the right model for the task.
|
||||
Two distinct concerns share this name. Read both — they apply at different
|
||||
moments.
|
||||
|
||||
## Routing Table
|
||||
## 1. gbrain's internal tier system (v0.31.12+)
|
||||
|
||||
This is how gbrain itself picks which Claude/OpenAI/Google model runs each
|
||||
internal task (chat, expansion, synthesis, classification, etc.).
|
||||
|
||||
Four tiers:
|
||||
|
||||
| Tier | Purpose | Default | Examples |
|
||||
|---|---|---|---|
|
||||
| `utility` | fast classification, expansion, verdict, dedup | `claude-haiku-4-5-20251001` | query expansion, facts contradiction classifier, dream synthesize verdict |
|
||||
| `reasoning` | default chat, synthesis, generation | `claude-sonnet-4-6` | gateway chat, dream synthesize, patterns, facts extraction |
|
||||
| `deep` | slow, expensive reasoning | `claude-opus-4-7` | `gbrain think`, auto-think, cross-modal eval slot B |
|
||||
| `subagent` | Anthropic-only multi-turn tool loop | `claude-sonnet-4-6` | `gbrain agent run` |
|
||||
|
||||
Override priority (highest first):
|
||||
|
||||
1. CLI flag (`--model opus`)
|
||||
2. Per-task config (`gbrain config set models.dream.synthesize opus`)
|
||||
3. Deprecated per-task config (stderr-warns once, then honored)
|
||||
4. **Global default** (`gbrain config set models.default opus`) — single hammer
|
||||
5. **Tier override** (`gbrain config set models.tier.reasoning opus`)
|
||||
6. Env var (`GBRAIN_MODEL=opus`)
|
||||
7. Tier default (the table above)
|
||||
8. Hardcoded caller fallback
|
||||
|
||||
Power-user recipes:
|
||||
|
||||
```bash
|
||||
# Use opus for everything
|
||||
gbrain config set models.default opus
|
||||
|
||||
# Use opus only for reasoning + deep, keep haiku for utility
|
||||
gbrain config set models.tier.reasoning opus
|
||||
gbrain config set models.tier.deep opus
|
||||
|
||||
# Custom alias, then use it everywhere
|
||||
gbrain config set models.aliases.frontier anthropic:claude-opus-4-7
|
||||
gbrain config set models.default frontier
|
||||
```
|
||||
|
||||
Visibility:
|
||||
|
||||
```bash
|
||||
gbrain models # print current routing table
|
||||
gbrain models doctor # 1-token probe to each configured model
|
||||
```
|
||||
|
||||
**Subagent tier exists because the loop is Anthropic-only.** The handler
|
||||
uses Messages API + prompt caching on system + tools. Setting
|
||||
`models.default = openai:gpt-5.5` silently breaks the loop, so we isolate
|
||||
`tier.subagent`. Three enforcement layers: submit-time guard in
|
||||
`MinionQueue.add`, tier-resolution fallback in `resolveModel`, doctor
|
||||
`subagent_provider` check.
|
||||
|
||||
When adding a new LLM call, route through `resolveModel()` with a tier —
|
||||
never hardcode a model string. The v0.31.6 chat default
|
||||
(`claude-sonnet-4-6-20250929`) was a phantom ID that 404'd the Anthropic
|
||||
API and made `extractFactsFromTurn` silently return `[]`. The tier system
|
||||
+ doctor probe is the structural fix for that bug class.
|
||||
|
||||
## 2. Subagent spawn routing
|
||||
|
||||
When the user-facing agent (Claude in the main session) chooses which model
|
||||
to spawn a sub-agent on, this table applies. It's about WHERE to send the
|
||||
work, not what gbrain calls internally.
|
||||
|
||||
| Task | Model | Why |
|
||||
|------|-------|-----|
|
||||
@@ -12,7 +77,7 @@ When spawning sub-agents, choose the right model for the task.
|
||||
| Fast lightweight sub-agents | Groq | 500 tok/s, good for simple tasks |
|
||||
| Judge tests / quality grading | Haiku | Cheap, good enough for pass/fail |
|
||||
|
||||
## Refusal Routing
|
||||
### Refusal routing
|
||||
|
||||
When a model refuses a request:
|
||||
1. Re-spawn on a different model (don't announce the switch)
|
||||
@@ -20,7 +85,7 @@ When a model refuses a request:
|
||||
3. Never show the refusal to the user
|
||||
4. If all models refuse, escalate to the user
|
||||
|
||||
## Spawn Rules
|
||||
### Spawn rules
|
||||
|
||||
- 3+ items to process → spawn a sub-agent
|
||||
- >2 tool calls that don't need real-time judgment → spawn
|
||||
|
||||
+13
-1
@@ -27,7 +27,7 @@ for (const op of operations) {
|
||||
}
|
||||
|
||||
// CLI-only commands that bypass the operation layer
|
||||
const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'transcripts', 'remote', 'recall', 'forget']);
|
||||
const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'transcripts', 'models', 'remote', 'recall', 'forget']);
|
||||
// CLI-only commands whose handlers print their own --help text. These are
|
||||
// excluded from the generic short-circuit so detailed per-command and
|
||||
// per-subcommand usage stays reachable.
|
||||
@@ -37,6 +37,7 @@ const CLI_ONLY_SELF_HELP = new Set([
|
||||
'skillpack', 'skillpack-check',
|
||||
'integrations', 'friction',
|
||||
'frontmatter', 'check-resolvable',
|
||||
'models',
|
||||
]);
|
||||
|
||||
async function main() {
|
||||
@@ -1071,6 +1072,11 @@ async function handleCliOnly(command: string, args: string[]) {
|
||||
await runTranscripts(engine, args);
|
||||
break;
|
||||
}
|
||||
case 'models': {
|
||||
const { runModels } = await import('./commands/models.ts');
|
||||
await runModels(engine, args);
|
||||
break;
|
||||
}
|
||||
case 'takes': {
|
||||
const { runTakes } = await import('./commands/takes.ts');
|
||||
await runTakes(engine, args);
|
||||
@@ -1298,6 +1304,12 @@ async function connectEngine(opts?: { probeOnly?: boolean }): Promise<BrainEngin
|
||||
// clear per startup is microseconds, no hot path.
|
||||
configureGateway(buildGatewayConfig(merged));
|
||||
}
|
||||
// v0.31.12: re-resolve gateway defaults through resolveModel so
|
||||
// `models.tier.*` and `models.default` overrides apply to expansion +
|
||||
// chat. Per Codex F3 — configureGateway is sync; this is the async
|
||||
// re-stamp seam after engine.connect() makes config reads possible.
|
||||
const { reconfigureGatewayWithEngine } = await import('./core/ai/gateway.ts');
|
||||
await reconfigureGatewayWithEngine(engine);
|
||||
} catch {
|
||||
// Non-fatal. Pre-v39 brains may not have a usable config table yet.
|
||||
}
|
||||
|
||||
@@ -336,9 +336,63 @@ export async function doctorReportRemote(engine: BrainEngine): Promise<DoctorRep
|
||||
checks.push({ name: 'queue_health', status: 'ok', message: 'PGLite — no queue to check' });
|
||||
}
|
||||
|
||||
// v0.31.12 subagent runtime enforcement (Layer 3 of 3 — Codex F13).
|
||||
// The subagent loop is Anthropic-only. If models.tier.subagent or
|
||||
// models.default is explicitly set to a non-Anthropic provider, warn here
|
||||
// so the user sees it at the next `gbrain doctor` run instead of at the
|
||||
// next subagent job submission. (Layers 1+2 also enforce — this is the
|
||||
// surfacing layer.)
|
||||
checks.push(await checkSubagentProvider(engine));
|
||||
|
||||
return computeDoctorReport(checks);
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.31.12 — surface a warn when models.tier.subagent or models.default
|
||||
* resolves to a non-Anthropic provider. The subagent loop in
|
||||
* src/core/minions/handlers/subagent.ts uses Anthropic Messages API with
|
||||
* prompt caching on system + tools; non-Anthropic providers would break
|
||||
* the loop at runtime. This check makes the configuration drift visible
|
||||
* before a job is submitted.
|
||||
*/
|
||||
async function checkSubagentProvider(engine: BrainEngine): Promise<Check> {
|
||||
try {
|
||||
const { isAnthropicProvider } = await import('../core/model-config.ts');
|
||||
const tierSubagent = await engine.getConfig('models.tier.subagent');
|
||||
const modelsDefault = await engine.getConfig('models.default');
|
||||
|
||||
// Tier-explicit override loses fail-loud since the user clearly meant it.
|
||||
if (tierSubagent && !isAnthropicProvider(tierSubagent)) {
|
||||
return {
|
||||
name: 'subagent_provider',
|
||||
status: 'warn',
|
||||
message:
|
||||
`models.tier.subagent is "${tierSubagent}" but the subagent loop is Anthropic-only. ` +
|
||||
`Runtime will fall back to claude-sonnet-4-6. Fix: ` +
|
||||
`\`gbrain config set models.tier.subagent anthropic:claude-sonnet-4-6\`.`,
|
||||
};
|
||||
}
|
||||
// models.default sneaking subagent into a non-Anthropic provider.
|
||||
if (!tierSubagent && modelsDefault && !isAnthropicProvider(modelsDefault)) {
|
||||
return {
|
||||
name: 'subagent_provider',
|
||||
status: 'warn',
|
||||
message:
|
||||
`models.default is "${modelsDefault}" which would route subagent jobs to a non-Anthropic provider. ` +
|
||||
`Runtime falls back to claude-sonnet-4-6 for subagent only. ` +
|
||||
`Fix: \`gbrain config set models.tier.subagent anthropic:claude-sonnet-4-6\` to lock it in.`,
|
||||
};
|
||||
}
|
||||
return { name: 'subagent_provider', status: 'ok', message: 'Subagent tier resolves to Anthropic' };
|
||||
} catch (e) {
|
||||
return {
|
||||
name: 'subagent_provider',
|
||||
status: 'warn',
|
||||
message: `Could not check subagent provider: ${e instanceof Error ? e.message : String(e)}`,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Run doctor with filesystem-first, DB-second architecture.
|
||||
* Filesystem checks (resolver, conformance) run without engine.
|
||||
@@ -1760,6 +1814,14 @@ export async function runDoctor(engine: BrainEngine | null, args: string[], dbSo
|
||||
}
|
||||
}
|
||||
|
||||
// 11.4 subagent_provider (v0.31.12 — Codex F13 layer 3 of 3). Surfaces a
|
||||
// warn when models.tier.subagent or models.default points at a non-Anthropic
|
||||
// provider. Layers 1 (queue.ts submit-time) and 2 (handler runtime) also
|
||||
// enforce; this is the surfacing layer so users see the config drift before
|
||||
// a job is submitted.
|
||||
progress.heartbeat('subagent_provider');
|
||||
checks.push(await checkSubagentProvider(engine));
|
||||
|
||||
// 11.5 facts_health (v0.31 hot memory). Surfaces per-source counters so
|
||||
// operators can see the extraction pipeline's pulse without raw SQL.
|
||||
// Lightweight: one COUNT-with-filters query + a top-5 aggregate. Only
|
||||
|
||||
@@ -0,0 +1,298 @@
|
||||
/**
|
||||
* v0.31.12 — `gbrain models` CLI.
|
||||
*
|
||||
* Two modes:
|
||||
*
|
||||
* `gbrain models` — read-only routing table. Prints the four
|
||||
* tier defaults, the resolved value for each
|
||||
* (after consulting models.default + models.tier.*),
|
||||
* per-task overrides, alias map, and source-of-truth
|
||||
* column (default / config / env).
|
||||
*
|
||||
* `gbrain models doctor` — opt-in probe. Fires a 1-token `gateway.chat()`
|
||||
* call against each configured chat / expansion
|
||||
* model and reports reachability with the
|
||||
* provider's error string. Catches the bug class
|
||||
* that motivated v0.31.12 (the v0.31.6 chat
|
||||
* default 404'd silently against the Anthropic
|
||||
* API).
|
||||
*
|
||||
* Flags:
|
||||
* --json — JSON output (both modes)
|
||||
* --skip=<provider> — narrow `doctor` probe to skip a provider
|
||||
* (e.g. cost-sensitive operators with rate limits)
|
||||
*
|
||||
* Per Codex F11 in plan review: no specific dollar cost claim. Probe uses
|
||||
* `max_tokens: 1` against each configured model; actual cost depends on
|
||||
* provider billing minimums.
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import {
|
||||
DEFAULT_ALIASES,
|
||||
TIER_DEFAULTS,
|
||||
resolveModel,
|
||||
type ModelTier,
|
||||
} from '../core/model-config.ts';
|
||||
|
||||
const TIERS: ModelTier[] = ['utility', 'reasoning', 'deep', 'subagent'];
|
||||
|
||||
const PER_TASK_KEYS: Array<{ key: string; tier: ModelTier; description: string }> = [
|
||||
{ key: 'models.dream.synthesize', tier: 'reasoning', description: 'Dream synthesis (conversation → brain pages)' },
|
||||
{ key: 'models.dream.synthesize_verdict', tier: 'utility', description: 'Dream synthesis verdict (Haiku judge)' },
|
||||
{ key: 'models.dream.patterns', tier: 'reasoning', description: 'Pattern discovery (cross-take themes)' },
|
||||
{ key: 'models.drift', tier: 'reasoning', description: 'Drift LLM judge (v0.29 scaffold)' },
|
||||
{ key: 'models.auto_think', tier: 'deep', description: 'Auto-think question answering' },
|
||||
{ key: 'models.think', tier: 'deep', description: '`gbrain think` synthesis op' },
|
||||
{ key: 'models.subagent', tier: 'subagent', description: '`gbrain agent run` subagent loop' },
|
||||
{ key: 'facts.extraction_model', tier: 'reasoning', description: 'Real-time facts extraction during sync' },
|
||||
{ key: 'models.eval.longmemeval', tier: 'reasoning', description: 'LongMemEval benchmark answer-gen' },
|
||||
{ key: 'models.expansion', tier: 'utility', description: 'Query expansion for hybrid search' },
|
||||
{ key: 'models.chat', tier: 'reasoning', description: 'Default `gateway.chat()` model' },
|
||||
];
|
||||
|
||||
interface ModelEntry {
|
||||
tier: ModelTier;
|
||||
resolved: string;
|
||||
source: string; // "default" | "config: <key>" | "env: <VAR>"
|
||||
}
|
||||
|
||||
interface ModelsReport {
|
||||
schema_version: 1;
|
||||
global_default: { value: string | null };
|
||||
tiers: Record<ModelTier, ModelEntry>;
|
||||
per_task: Array<{ key: string; tier: ModelTier; resolved: string; source: string; description: string }>;
|
||||
aliases: { defaults: Record<string, string>; user: Record<string, string> };
|
||||
}
|
||||
|
||||
async function probeSource(engine: BrainEngine, configKey: string, envVar: string): Promise<string | null> {
|
||||
// For per-task probes, return the source the resolver USED (config / env /
|
||||
// tier default / hardcoded). The resolver itself is the source of truth;
|
||||
// we re-walk a subset of its precedence here to attribute the value.
|
||||
const configVal = await engine.getConfig(configKey);
|
||||
if (configVal && configVal.trim()) return `config: ${configKey}`;
|
||||
if (process.env[envVar] && process.env[envVar]!.trim()) return `env: ${envVar}`;
|
||||
return null;
|
||||
}
|
||||
|
||||
async function buildReport(engine: BrainEngine): Promise<ModelsReport> {
|
||||
const globalDefault = await engine.getConfig('models.default');
|
||||
|
||||
const tiers = {} as Record<ModelTier, ModelEntry>;
|
||||
for (const t of TIERS) {
|
||||
const tierOverride = await engine.getConfig(`models.tier.${t}`);
|
||||
// What models.default beats tier — re-walk the chain to attribute properly.
|
||||
let source: string;
|
||||
if (globalDefault && globalDefault.trim()) {
|
||||
source = 'config: models.default';
|
||||
} else if (tierOverride && tierOverride.trim()) {
|
||||
source = `config: models.tier.${t}`;
|
||||
} else {
|
||||
source = 'default';
|
||||
}
|
||||
const resolved = await resolveModel(engine, { tier: t, fallback: TIER_DEFAULTS[t] });
|
||||
tiers[t] = { tier: t, resolved, source };
|
||||
}
|
||||
|
||||
const per_task: ModelsReport['per_task'] = [];
|
||||
for (const { key, tier, description } of PER_TASK_KEYS) {
|
||||
const resolved = await resolveModel(engine, { configKey: key, tier, fallback: TIER_DEFAULTS[tier] });
|
||||
const explicit = await probeSource(engine, key, 'GBRAIN_MODEL');
|
||||
const source = explicit ?? `tier.${tier}`;
|
||||
per_task.push({ key, tier, resolved, source, description });
|
||||
}
|
||||
|
||||
// User-defined aliases (engine.getConfig is the source; we don't enumerate
|
||||
// every possible alias key, just the common ones the docs mention).
|
||||
const userAliases: Record<string, string> = {};
|
||||
for (const name of ['opus', 'sonnet', 'haiku', 'gemini', 'gpt']) {
|
||||
const v = await engine.getConfig(`models.aliases.${name}`);
|
||||
if (v && v.trim()) userAliases[name] = v.trim();
|
||||
}
|
||||
|
||||
return {
|
||||
schema_version: 1,
|
||||
global_default: { value: globalDefault?.trim() || null },
|
||||
tiers,
|
||||
per_task,
|
||||
aliases: { defaults: { ...DEFAULT_ALIASES }, user: userAliases },
|
||||
};
|
||||
}
|
||||
|
||||
function formatText(report: ModelsReport): string {
|
||||
const lines: string[] = [];
|
||||
lines.push('Tier routing:');
|
||||
for (const t of TIERS) {
|
||||
const e = report.tiers[t];
|
||||
lines.push(` tier.${t.padEnd(10)} ${e.resolved.padEnd(45)} [${e.source}]`);
|
||||
}
|
||||
lines.push('');
|
||||
lines.push('Global default:');
|
||||
lines.push(` models.default ${report.global_default.value ?? '(unset)'}`);
|
||||
lines.push('');
|
||||
lines.push('Per-task overrides:');
|
||||
for (const t of report.per_task) {
|
||||
lines.push(` ${t.key.padEnd(34)} → ${t.resolved.padEnd(45)} [${t.source}]`);
|
||||
}
|
||||
lines.push('');
|
||||
lines.push('Aliases:');
|
||||
for (const [k, v] of Object.entries(report.aliases.defaults)) {
|
||||
const userOverride = report.aliases.user[k];
|
||||
if (userOverride) {
|
||||
lines.push(` ${k.padEnd(8)} → ${userOverride} (user override; default: ${v})`);
|
||||
} else {
|
||||
lines.push(` ${k.padEnd(8)} → ${v}`);
|
||||
}
|
||||
}
|
||||
for (const [k, v] of Object.entries(report.aliases.user)) {
|
||||
if (!(k in report.aliases.defaults)) {
|
||||
lines.push(` ${k.padEnd(8)} → ${v} (user)`);
|
||||
}
|
||||
}
|
||||
lines.push('');
|
||||
lines.push('Tip: probe reachability with `gbrain models doctor` (opt-in; spends ~1 token per model).');
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
||||
// ── Doctor (probe) mode ────────────────────────────────────────────
|
||||
|
||||
type ProbeStatus = 'ok' | 'model_not_found' | 'auth' | 'rate_limit' | 'network' | 'unknown';
|
||||
|
||||
interface ProbeResult {
|
||||
model: string;
|
||||
touchpoint: 'chat' | 'expansion';
|
||||
status: ProbeStatus;
|
||||
message: string;
|
||||
elapsed_ms: number;
|
||||
}
|
||||
|
||||
function classifyError(err: unknown): { status: ProbeStatus; message: string } {
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
const lower = msg.toLowerCase();
|
||||
if (/not_?found|does not exist|invalid_model|model.*invalid|404/.test(lower)) {
|
||||
return { status: 'model_not_found', message: msg };
|
||||
}
|
||||
if (/auth|unauthor|401|403|api[_-]?key/.test(lower)) {
|
||||
return { status: 'auth', message: msg };
|
||||
}
|
||||
if (/rate.?limit|429|too many/.test(lower)) {
|
||||
return { status: 'rate_limit', message: msg };
|
||||
}
|
||||
if (/timeout|network|econn|fetch failed|enotfound/.test(lower)) {
|
||||
return { status: 'network', message: msg };
|
||||
}
|
||||
return { status: 'unknown', message: msg };
|
||||
}
|
||||
|
||||
async function probeModel(modelStr: string, touchpoint: 'chat' | 'expansion'): Promise<ProbeResult> {
|
||||
const start = Date.now();
|
||||
try {
|
||||
const { chat } = await import('../core/ai/gateway.ts');
|
||||
// Use AbortController so the 5s timeout doesn't hang on a stuck network.
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(new Error('probe timed out after 5s')), 5000);
|
||||
try {
|
||||
await chat({
|
||||
model: modelStr,
|
||||
messages: [{ role: 'user', content: '.' }],
|
||||
maxTokens: 1,
|
||||
abortSignal: controller.signal,
|
||||
});
|
||||
return { model: modelStr, touchpoint, status: 'ok', message: 'reachable', elapsed_ms: Date.now() - start };
|
||||
} finally {
|
||||
clearTimeout(timeoutId);
|
||||
}
|
||||
} catch (err) {
|
||||
const { status, message } = classifyError(err);
|
||||
return { model: modelStr, touchpoint, status, message, elapsed_ms: Date.now() - start };
|
||||
}
|
||||
}
|
||||
|
||||
function shouldSkipProvider(modelStr: string, skip: string[]): boolean {
|
||||
if (skip.length === 0) return false;
|
||||
const colon = modelStr.indexOf(':');
|
||||
const provider = colon === -1 ? '' : modelStr.slice(0, colon).toLowerCase();
|
||||
return skip.includes(provider);
|
||||
}
|
||||
|
||||
export async function runModels(engine: BrainEngine, args: string[]): Promise<void> {
|
||||
const json = args.includes('--json');
|
||||
const sub = args[1] === 'doctor' ? 'doctor' : args[1] === 'help' || args.includes('--help') || args.includes('-h') ? 'help' : 'read';
|
||||
|
||||
if (sub === 'help') {
|
||||
process.stdout.write(
|
||||
`Usage:
|
||||
gbrain models Show routing table (read-only)
|
||||
gbrain models doctor [flags] Probe each configured model (~1 token each)
|
||||
gbrain models --json Machine-readable output
|
||||
|
||||
Flags (doctor only):
|
||||
--skip=<provider> Skip a provider (e.g. --skip=openai)
|
||||
Repeatable: --skip=openai --skip=google
|
||||
--json JSON output
|
||||
|
||||
Configure routing:
|
||||
gbrain config set models.default <model> # global hammer
|
||||
gbrain config set models.tier.<tier> <model> # per-tier (utility/reasoning/deep/subagent)
|
||||
gbrain config set models.aliases.<name> <model> # custom alias
|
||||
|
||||
Tiers: utility (haiku-class) | reasoning (sonnet) | deep (opus) | subagent (Anthropic-only)
|
||||
`);
|
||||
return;
|
||||
}
|
||||
|
||||
if (sub === 'read') {
|
||||
const report = await buildReport(engine);
|
||||
if (json) {
|
||||
process.stdout.write(JSON.stringify(report, null, 2) + '\n');
|
||||
} else {
|
||||
process.stdout.write(formatText(report) + '\n');
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// doctor mode
|
||||
const skipArgs = args.filter(a => a.startsWith('--skip='));
|
||||
const skip = skipArgs.map(a => a.slice('--skip='.length).toLowerCase()).filter(Boolean);
|
||||
|
||||
const { getChatModel, getExpansionModel } = await import('../core/ai/gateway.ts');
|
||||
const chatModel = getChatModel();
|
||||
const expansionModel = getExpansionModel();
|
||||
|
||||
const results: ProbeResult[] = [];
|
||||
for (const [modelStr, touchpoint] of [[chatModel, 'chat'], [expansionModel, 'expansion']] as const) {
|
||||
if (shouldSkipProvider(modelStr, skip)) {
|
||||
if (!json) process.stderr.write(`[skip] ${touchpoint}: ${modelStr} (provider in --skip)\n`);
|
||||
continue;
|
||||
}
|
||||
results.push(await probeModel(modelStr, touchpoint));
|
||||
}
|
||||
|
||||
const report = {
|
||||
schema_version: 1 as const,
|
||||
probes: results,
|
||||
summary: {
|
||||
total: results.length,
|
||||
ok: results.filter(r => r.status === 'ok').length,
|
||||
failed: results.filter(r => r.status !== 'ok').length,
|
||||
},
|
||||
};
|
||||
|
||||
if (json) {
|
||||
process.stdout.write(JSON.stringify(report, null, 2) + '\n');
|
||||
} else {
|
||||
process.stdout.write('Model reachability probe:\n');
|
||||
for (const r of results) {
|
||||
const icon = r.status === 'ok' ? '✔' : '✘';
|
||||
process.stdout.write(` ${icon} ${r.touchpoint.padEnd(10)} ${r.model.padEnd(50)} ${r.status} (${r.elapsed_ms}ms)\n`);
|
||||
if (r.status !== 'ok') {
|
||||
process.stdout.write(` ${r.message}\n`);
|
||||
}
|
||||
}
|
||||
process.stdout.write(`\nSummary: ${report.summary.ok}/${report.summary.total} reachable.\n`);
|
||||
}
|
||||
|
||||
if (report.summary.failed > 0) {
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
+128
-5
@@ -35,7 +35,9 @@ import type {
|
||||
Recipe,
|
||||
TouchpointKind,
|
||||
} from './types.ts';
|
||||
import { resolveRecipe, assertTouchpoint } from './model-resolver.ts';
|
||||
import { resolveRecipe, assertTouchpoint, parseModelId } from './model-resolver.ts';
|
||||
import { resolveModel, TIER_DEFAULTS } from '../model-config.ts';
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import { dimsProviderOptions } from './dims.ts';
|
||||
import { AIConfigError, AITransientError, normalizeAIError } from './errors.ts';
|
||||
|
||||
@@ -43,11 +45,54 @@ const MAX_CHARS = 8000;
|
||||
const DEFAULT_EMBEDDING_MODEL = 'openai:text-embedding-3-large';
|
||||
const DEFAULT_EMBEDDING_DIMENSIONS = 1536;
|
||||
const DEFAULT_EXPANSION_MODEL = 'anthropic:claude-haiku-4-5-20251001';
|
||||
const DEFAULT_CHAT_MODEL = 'anthropic:claude-sonnet-4-6-20250929';
|
||||
const DEFAULT_CHAT_MODEL = 'anthropic:claude-sonnet-4-6';
|
||||
|
||||
let _config: AIGatewayConfig | null = null;
|
||||
const _modelCache = new Map<string, any>();
|
||||
|
||||
/**
|
||||
* v0.31.12 recipe-models merge: per-gateway-instance set of model ids the
|
||||
* user opted into via config. Keyed by provider id (`anthropic`, `openai`,
|
||||
* etc.). Passed into `assertTouchpoint` so native-recipe allowlist checks
|
||||
* skip these models — provider 404s surface at HTTP call time instead of
|
||||
* config-build time.
|
||||
*
|
||||
* Replaces the earlier plan to soften `assertTouchpoint` from throw to
|
||||
* warn (Codex F4/F5 — too broad, removed fail-fast for chat/expand/embed
|
||||
* across all callers). This narrower approach preserves fail-fast for
|
||||
* source-code typos while allowing config-time model selection of any id.
|
||||
*/
|
||||
const _extendedModels: Map<string, Set<string>> = new Map();
|
||||
|
||||
/**
|
||||
* v0.31.12 — register a model id under its provider so `assertTouchpoint`
|
||||
* (called via the gateway's chat/embed/expand entry points) permits it
|
||||
* even when it isn't in the recipe's declared `models:` array.
|
||||
*
|
||||
* Idempotent + safe to call before/after configureGateway. Exported only
|
||||
* for the `gbrain models doctor` probe path (where the operator may want
|
||||
* to probe any user-supplied id without re-running configure).
|
||||
*/
|
||||
function registerExtendedModel(modelStr: string): void {
|
||||
if (!modelStr) return;
|
||||
try {
|
||||
const { providerId, modelId } = parseModelId(modelStr);
|
||||
let set = _extendedModels.get(providerId);
|
||||
if (!set) {
|
||||
set = new Set();
|
||||
_extendedModels.set(providerId, set);
|
||||
}
|
||||
set.add(modelId);
|
||||
} catch {
|
||||
// Malformed model strings will fail at parseModelId — ignore here;
|
||||
// the actual chat/embed call will surface the error.
|
||||
}
|
||||
}
|
||||
|
||||
function getExtendedModelsForProvider(providerId: string): ReadonlySet<string> | undefined {
|
||||
return _extendedModels.get(providerId);
|
||||
}
|
||||
|
||||
/**
|
||||
* The function the gateway calls to actually run a batch through the AI SDK.
|
||||
* Defaults to the imported `embedMany`. Tests inject a stub via
|
||||
@@ -108,9 +153,86 @@ export function configureGateway(config: AIGatewayConfig): void {
|
||||
};
|
||||
_modelCache.clear();
|
||||
_shrinkState.clear();
|
||||
_extendedModels.clear();
|
||||
// Register configured models so assertTouchpoint allows them even when
|
||||
// they aren't in the recipe's declared models: array (v0.31.12).
|
||||
for (const m of [
|
||||
_config.embedding_model,
|
||||
_config.embedding_multimodal_model,
|
||||
_config.expansion_model,
|
||||
_config.chat_model,
|
||||
...(_config.chat_fallback_chain ?? []),
|
||||
]) {
|
||||
if (m) registerExtendedModel(m);
|
||||
}
|
||||
warnRecipesMissingBatchTokens();
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.31.12 — async re-stamp seam.
|
||||
*
|
||||
* After `engine.connect()` succeeds, callers (today: `src/cli.ts`)
|
||||
* invoke this to re-resolve the gateway's expansion / chat / embedding
|
||||
* defaults through `resolveModel()` (which can read `models.tier.*` /
|
||||
* `models.default` / per-task config keys from the engine). The pre-connect
|
||||
* `configureGateway` path used hardcoded TIER_DEFAULTS as fallbacks;
|
||||
* this re-stamp picks up any user overrides that live in the DB-backed
|
||||
* config plane.
|
||||
*
|
||||
* Sync `configureGateway` stays for pre-connect callers (rare bootstrap
|
||||
* paths like `gbrain --version` that never touch a brain). Per Codex F3
|
||||
* in the v0.31.12 plan review: spelling out the sync→async boundary instead
|
||||
* of hand-waving "config-build time."
|
||||
*
|
||||
* Idempotent. Safe to call multiple times. Returns the resolved gateway
|
||||
* config for callers who want to inspect what landed.
|
||||
*/
|
||||
export async function reconfigureGatewayWithEngine(engine: BrainEngine): Promise<AIGatewayConfig> {
|
||||
const cfg = requireConfig();
|
||||
// Resolve expansion (utility tier) and chat (reasoning tier). Embedding is
|
||||
// intentionally NOT re-resolved here — switching embedding models invalidates
|
||||
// the vector index. Out of scope per v0.31.12 plan ("Embedding tier knob").
|
||||
const newExpansion = await resolveModel(engine, {
|
||||
configKey: 'models.expansion',
|
||||
tier: 'utility',
|
||||
fallback: cfg.expansion_model ?? DEFAULT_EXPANSION_MODEL,
|
||||
});
|
||||
const newChat = await resolveModel(engine, {
|
||||
configKey: 'models.chat',
|
||||
tier: 'reasoning',
|
||||
fallback: cfg.chat_model ?? DEFAULT_CHAT_MODEL,
|
||||
});
|
||||
|
||||
// Resolved values are bare model ids (e.g. `claude-sonnet-4-6`) — prepend
|
||||
// the existing provider prefix from cfg so the gateway keeps routing to
|
||||
// the right recipe. If the resolved string already contains a `:`, it
|
||||
// came from a `provider:model` override and we use it as-is.
|
||||
const expansionFull = newExpansion.includes(':') ? newExpansion : prefixWithProviderFrom(cfg.expansion_model ?? DEFAULT_EXPANSION_MODEL, newExpansion);
|
||||
const chatFull = newChat.includes(':') ? newChat : prefixWithProviderFrom(cfg.chat_model ?? DEFAULT_CHAT_MODEL, newChat);
|
||||
|
||||
_config = { ...cfg, expansion_model: expansionFull, chat_model: chatFull };
|
||||
_modelCache.clear();
|
||||
_shrinkState.clear();
|
||||
_extendedModels.clear();
|
||||
for (const m of [
|
||||
_config.embedding_model,
|
||||
_config.embedding_multimodal_model,
|
||||
_config.expansion_model,
|
||||
_config.chat_model,
|
||||
...(_config.chat_fallback_chain ?? []),
|
||||
]) {
|
||||
if (m) registerExtendedModel(m);
|
||||
}
|
||||
return _config;
|
||||
}
|
||||
|
||||
/** Carry over the provider prefix from `original` when `bare` lacks one. */
|
||||
function prefixWithProviderFrom(original: string, bare: string): string {
|
||||
const colon = original.indexOf(':');
|
||||
if (colon === -1) return bare;
|
||||
return `${original.slice(0, colon)}:${bare}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Recipes that have already triggered the missing-max_batch_tokens warning
|
||||
* in this process. Bounded by the number of registered recipes (~10 today).
|
||||
@@ -155,6 +277,7 @@ export function resetGateway(): void {
|
||||
_embedTransport = embedMany;
|
||||
_chatTransport = null;
|
||||
_warnedRecipes.clear();
|
||||
_extendedModels.clear();
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -406,7 +529,7 @@ const voyageCompatFetch = (async (input: RequestInfo | URL, init?: RequestInit)
|
||||
|
||||
async function resolveEmbeddingProvider(modelStr: string): Promise<{ model: any; recipe: Recipe; modelId: string }> {
|
||||
const { parsed, recipe } = resolveRecipe(modelStr);
|
||||
assertTouchpoint(recipe, 'embedding', parsed.modelId);
|
||||
assertTouchpoint(recipe, 'embedding', parsed.modelId, getExtendedModelsForProvider(parsed.providerId));
|
||||
const cfg = requireConfig();
|
||||
|
||||
const cacheKey = `emb:${recipe.id}:${parsed.modelId}:${cfg.base_urls?.[recipe.id] ?? ''}`;
|
||||
@@ -873,7 +996,7 @@ void MULTIMODAL_MAX_IMAGE_BYTES;
|
||||
|
||||
async function resolveExpansionProvider(modelStr: string): Promise<{ model: any; recipe: Recipe; modelId: string }> {
|
||||
const { parsed, recipe } = resolveRecipe(modelStr);
|
||||
assertTouchpoint(recipe, 'expansion', parsed.modelId);
|
||||
assertTouchpoint(recipe, 'expansion', parsed.modelId, getExtendedModelsForProvider(parsed.providerId));
|
||||
const cfg = requireConfig();
|
||||
|
||||
const cacheKey = `exp:${recipe.id}:${parsed.modelId}:${cfg.base_urls?.[recipe.id] ?? ''}`;
|
||||
@@ -1076,7 +1199,7 @@ export interface ChatOpts {
|
||||
|
||||
async function resolveChatProvider(modelStr: string): Promise<{ model: any; recipe: Recipe; modelId: string }> {
|
||||
const { parsed, recipe } = resolveRecipe(modelStr);
|
||||
assertTouchpoint(recipe, 'chat', parsed.modelId);
|
||||
assertTouchpoint(recipe, 'chat', parsed.modelId, getExtendedModelsForProvider(parsed.providerId));
|
||||
const cfg = requireConfig();
|
||||
|
||||
const cacheKey = `chat:${recipe.id}:${parsed.modelId}:${cfg.base_urls?.[recipe.id] ?? ''}`;
|
||||
|
||||
@@ -63,8 +63,31 @@ function getTouchpoint(recipe: Recipe, touchpoint: TouchpointKind): EmbeddingTou
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/** Assert the resolved recipe actually offers the requested touchpoint. */
|
||||
export function assertTouchpoint(recipe: Recipe, touchpoint: TouchpointKind, modelId: string): void {
|
||||
/**
|
||||
* Assert the resolved recipe actually offers the requested touchpoint.
|
||||
*
|
||||
* @param extendedModels Per-gateway-instance Set of additional models the
|
||||
* user opted into via `cfg.chat_model` / `cfg.embedding_model` /
|
||||
* `cfg.expansion_model` / `models.default` / `models.tier.*`. When the
|
||||
* modelId is in this set, the native-recipe allowlist check is skipped
|
||||
* (the user explicitly chose this model via config — provider rejection
|
||||
* surfaces at HTTP call time, with a clear `model_not_found` from the
|
||||
* provider).
|
||||
*
|
||||
* Default code paths (hardcoded model strings in source code) MUST NOT
|
||||
* pass this argument — typos in code still fail fast. Only config-derived
|
||||
* model selection extends the allowlist.
|
||||
*
|
||||
* v0.31.12 — replaces the earlier plan to soften the validator from throw
|
||||
* to warn (which would have removed the fail-fast contract for chat/expand/
|
||||
* embed all three; per Codex F4/F5 in plan review).
|
||||
*/
|
||||
export function assertTouchpoint(
|
||||
recipe: Recipe,
|
||||
touchpoint: TouchpointKind,
|
||||
modelId: string,
|
||||
extendedModels?: ReadonlySet<string>,
|
||||
): void {
|
||||
const tp = getTouchpoint(recipe, touchpoint);
|
||||
if (!tp) {
|
||||
throw new AIConfigError(
|
||||
@@ -80,6 +103,13 @@ export function assertTouchpoint(recipe: Recipe, touchpoint: TouchpointKind, mod
|
||||
if (supportedModels.length > 0 && !supportedModels.includes(modelId)) {
|
||||
// Non-fatal: providers like ollama/litellm accept arbitrary model ids. We only warn for native providers.
|
||||
if (recipe.tier === 'native') {
|
||||
// v0.31.12 recipe-models merge: if the user opted into this model via
|
||||
// config (cfg.chat_model, models.default, models.tier.*), skip the
|
||||
// throw. The model goes to the provider; provider 404s surface as
|
||||
// `model_not_found` via `gbrain models doctor`.
|
||||
if (extendedModels && extendedModels.has(modelId)) {
|
||||
return;
|
||||
}
|
||||
throw new AIConfigError(
|
||||
`Model "${modelId}" is not listed for ${recipe.name} ${touchpoint}.`,
|
||||
`Known models: ${supportedModels.join(', ')}. Use one of these or add it to the recipe (or add an alias).`,
|
||||
|
||||
@@ -17,14 +17,14 @@ export const anthropic: Recipe = {
|
||||
touchpoints: {
|
||||
// No embedding model available.
|
||||
expansion: {
|
||||
models: ['claude-haiku-4-5-20251001', 'claude-sonnet-4-6-20250929'],
|
||||
models: ['claude-haiku-4-5-20251001', 'claude-sonnet-4-6'],
|
||||
cost_per_1m_tokens_usd: 0.25,
|
||||
price_last_verified: '2026-04-20',
|
||||
price_last_verified: '2026-05-10',
|
||||
},
|
||||
chat: {
|
||||
models: [
|
||||
'claude-opus-4-7',
|
||||
'claude-sonnet-4-6-20250929',
|
||||
'claude-sonnet-4-6',
|
||||
'claude-haiku-4-5-20251001',
|
||||
],
|
||||
supports_tools: true,
|
||||
@@ -33,13 +33,16 @@ export const anthropic: Recipe = {
|
||||
max_context_tokens: 200000,
|
||||
cost_per_1m_input_usd: 3.0, // sonnet-class baseline
|
||||
cost_per_1m_output_usd: 15.0,
|
||||
price_last_verified: '2026-04-20',
|
||||
price_last_verified: '2026-05-10',
|
||||
},
|
||||
},
|
||||
// Friendly undated aliases (Codex F-OV-5).
|
||||
// Friendly aliases. Starting with Claude 4.6, Anthropic API IDs are dateless
|
||||
// and pinned (no alias needed). Only pre-4.6 models need date-suffixed aliases.
|
||||
// The reverse entry rewrites the v0.31.6-shipped broken ID back to canonical
|
||||
// so users with stale `models.dream.synthesize` etc. configs keep working.
|
||||
aliases: {
|
||||
'claude-sonnet-4-6': 'claude-sonnet-4-6-20250929',
|
||||
'claude-haiku-4-5': 'claude-haiku-4-5-20251001',
|
||||
'claude-sonnet-4-6-20250929': 'claude-sonnet-4-6',
|
||||
},
|
||||
setup_hint: 'Get an API key at https://console.anthropic.com/settings/keys, then `export ANTHROPIC_API_KEY=...`',
|
||||
};
|
||||
|
||||
@@ -139,10 +139,13 @@ export interface Recipe {
|
||||
chat?: ChatTouchpoint;
|
||||
};
|
||||
/**
|
||||
* Optional alias map for friendlier `provider:model` strings (Codex F-OV-5).
|
||||
* Resolved at parse time so users can write `anthropic:claude-sonnet-4-6`
|
||||
* instead of `anthropic:claude-sonnet-4-6-20250929`. Keys are aliases,
|
||||
* values are canonical (declared) model ids.
|
||||
* Optional alias map for friendlier `provider:model` strings.
|
||||
* Resolved at parse time. For pre-4.6 models, undated forms alias to dated
|
||||
* pinned snapshots (e.g. `claude-haiku-4-5` → `claude-haiku-4-5-20251001`).
|
||||
* For Claude 4.6+, model IDs are dateless and self-pinned — no forward alias
|
||||
* needed. Reverse-direction entries can rewrite stale/broken IDs back to
|
||||
* canonical (e.g. `claude-sonnet-4-6-20250929` → `claude-sonnet-4-6`) for
|
||||
* back-compat with users who have stale config strings.
|
||||
*/
|
||||
aliases?: Record<string, string>;
|
||||
/** One-line description of setup (shown in wizard + env subcommand). */
|
||||
|
||||
@@ -21,12 +21,14 @@ export interface ModelPricing {
|
||||
|
||||
/** Map of Anthropic model id → pricing. Aliases (opus/sonnet/haiku) resolve via DEFAULT_ALIASES. */
|
||||
export const ANTHROPIC_PRICING: Record<string, ModelPricing> = {
|
||||
// Claude 4.7 family (current generation)
|
||||
'claude-opus-4-7': { input: 15.00, output: 75.00 },
|
||||
// Claude 4.7 generation (current)
|
||||
// Opus 4.7 dropped from $15/$75 (Opus 4) to $5/$25 per
|
||||
// https://platform.claude.com/docs/en/about-claude/models/overview (verified 2026-05-10).
|
||||
'claude-opus-4-7': { input: 5.00, output: 25.00 },
|
||||
'claude-sonnet-4-6': { input: 3.00, output: 15.00 },
|
||||
'claude-haiku-4-5-20251001': { input: 1.00, output: 5.00 },
|
||||
// Older but still frequently aliased
|
||||
'claude-opus-4-6': { input: 15.00, output: 75.00 },
|
||||
'claude-opus-4-6': { input: 5.00, output: 25.00 },
|
||||
'claude-3-5-sonnet-20241022': { input: 3.00, output: 15.00 },
|
||||
'claude-3-5-haiku-20241022': { input: 0.80, output: 4.00 },
|
||||
};
|
||||
|
||||
+1
-1
@@ -37,7 +37,7 @@ export interface GBrainConfig {
|
||||
expansion_model?: string;
|
||||
/**
|
||||
* Default chat model for `gateway.chat()` callers (v0.27+).
|
||||
* Default: "anthropic:claude-sonnet-4-6-20250929".
|
||||
* Default: "anthropic:claude-sonnet-4-6" (dateless per Anthropic's v0.31.12+ model-ID format).
|
||||
*/
|
||||
chat_model?: string;
|
||||
/**
|
||||
|
||||
@@ -22,6 +22,7 @@ import type { AggregateResult, SlotResult } from './aggregate.ts';
|
||||
import { parseModelJSON } from './json-repair.ts';
|
||||
import { receiptName, sha8 } from './receipt-name.ts';
|
||||
import { writeReceipt } from './receipt-write.ts';
|
||||
import { ANTHROPIC_PRICING } from '../anthropic-pricing.ts';
|
||||
|
||||
export const RECEIPT_SCHEMA_VERSION = 1;
|
||||
|
||||
@@ -320,13 +321,23 @@ export interface CostEstimate {
|
||||
export function estimateCost(slots: SlotConfig[], cycles: number, maxTokens: number): CostEstimate {
|
||||
// Per-call cost = (input_tokens × input_price + output_tokens × output_price) / 1e6.
|
||||
// Without knowing prompt size, estimate input ~5k tokens (a SKILL.md + scoring rubric).
|
||||
//
|
||||
// Anthropic prices read from ANTHROPIC_PRICING (single source of truth — fixes
|
||||
// the drift trap Codex flagged in v0.31.12 plan review: this map and
|
||||
// anthropic-pricing.ts duplicated Anthropic prices, with stale values diverging).
|
||||
// Non-Anthropic models still live inline until OPENAI_PRICING / GOOGLE_PRICING
|
||||
// tables exist.
|
||||
const ESTIMATED_INPUT_TOKENS = 5000;
|
||||
const anthropicPrice = (modelId: string): { in: number; out: number } | undefined => {
|
||||
const p = ANTHROPIC_PRICING[modelId];
|
||||
return p ? { in: p.input, out: p.output } : undefined;
|
||||
};
|
||||
const PRICING: Record<string, { in: number; out: number } | undefined> = {
|
||||
'openai:gpt-4o': { in: 2.5, out: 10.0 },
|
||||
'openai:gpt-4o-mini': { in: 0.15, out: 0.6 },
|
||||
'anthropic:claude-opus-4-7': { in: 15.0, out: 75.0 },
|
||||
'anthropic:claude-sonnet-4-6-20250929': { in: 3.0, out: 15.0 },
|
||||
'anthropic:claude-haiku-4-5-20251001': { in: 0.25, out: 1.25 },
|
||||
'anthropic:claude-opus-4-7': anthropicPrice('claude-opus-4-7'),
|
||||
'anthropic:claude-sonnet-4-6': anthropicPrice('claude-sonnet-4-6'),
|
||||
'anthropic:claude-haiku-4-5-20251001': anthropicPrice('claude-haiku-4-5-20251001'),
|
||||
'google:gemini-1.5-pro': { in: 1.25, out: 5.0 },
|
||||
'google:gemini-2.0-flash': { in: 0.1, out: 0.4 },
|
||||
'together:meta-llama/Llama-3.3-70B-Instruct-Turbo': { in: 0.88, out: 0.88 },
|
||||
|
||||
@@ -117,6 +117,7 @@ export async function runPhaseAutoThink(
|
||||
const modelId = await resolveModel(engine, {
|
||||
configKey: 'models.auto_think',
|
||||
deprecatedConfigKey: 'dream.auto_think.model',
|
||||
tier: 'deep',
|
||||
fallback: 'opus',
|
||||
});
|
||||
|
||||
|
||||
@@ -131,6 +131,7 @@ export async function runPhaseDrift(
|
||||
const modelId = await resolveModel(engine, {
|
||||
configKey: 'models.drift',
|
||||
deprecatedConfigKey: 'dream.drift.model',
|
||||
tier: 'reasoning',
|
||||
fallback: 'sonnet',
|
||||
});
|
||||
const meter = new BudgetMeter({
|
||||
|
||||
@@ -145,6 +145,7 @@ async function loadPatternsConfig(engine: BrainEngine): Promise<PatternsConfig>
|
||||
const model = await resolveModel(engine, {
|
||||
configKey: 'models.dream.patterns',
|
||||
deprecatedConfigKey: 'dream.patterns.model',
|
||||
tier: 'reasoning',
|
||||
fallback: 'sonnet',
|
||||
});
|
||||
return {
|
||||
|
||||
@@ -533,11 +533,13 @@ async function loadSynthConfig(engine: BrainEngine): Promise<SynthConfig> {
|
||||
const model = await resolveModel(engine, {
|
||||
configKey: 'models.dream.synthesize',
|
||||
deprecatedConfigKey: 'dream.synthesize.model',
|
||||
tier: 'reasoning',
|
||||
fallback: 'sonnet',
|
||||
});
|
||||
const verdictModel = await resolveModel(engine, {
|
||||
configKey: 'models.dream.synthesize_verdict',
|
||||
deprecatedConfigKey: 'dream.synthesize.verdict_model',
|
||||
tier: 'utility',
|
||||
fallback: 'haiku',
|
||||
});
|
||||
const cooldownHoursStr = await engine.getConfig('dream.synthesize.cooldown_hours');
|
||||
|
||||
@@ -24,6 +24,7 @@
|
||||
import { chat, embedOne, isAvailable } from '../ai/gateway.ts';
|
||||
import type { ChatResult } from '../ai/gateway.ts';
|
||||
import { INJECTION_PATTERNS } from '../think/sanitize.ts';
|
||||
import { resolveModel } from '../model-config.ts';
|
||||
import type { BrainEngine, NewFact, FactKind } from '../engine.ts';
|
||||
|
||||
/**
|
||||
@@ -50,9 +51,17 @@ export async function isFactsExtractionEnabled(engine: BrainEngine): Promise<boo
|
||||
* Configurable via `gbrain config set facts.extraction_model <model>`.
|
||||
*/
|
||||
export async function getFactsExtractionModel(engine?: BrainEngine): Promise<string> {
|
||||
if (!engine) return 'anthropic:claude-sonnet-4-6-20250929';
|
||||
const configured = await engine.getConfig('facts.extraction_model');
|
||||
return configured || 'anthropic:claude-sonnet-4-6-20250929';
|
||||
// v0.31.12: route through resolveModel so models.default + models.tier.reasoning
|
||||
// overrides reach facts extraction. Per-config-key facts.extraction_model still
|
||||
// wins via configKey, preserving the prior behavior for existing users.
|
||||
const resolved = await resolveModel(engine ?? null, {
|
||||
configKey: 'facts.extraction_model',
|
||||
tier: 'reasoning',
|
||||
fallback: 'anthropic:claude-sonnet-4-6',
|
||||
});
|
||||
// resolveModel returns bare model ids when resolving via tier defaults; ensure
|
||||
// the result keeps a provider prefix so gateway.chat() can route it.
|
||||
return resolved.includes(':') ? resolved : `anthropic:${resolved}`;
|
||||
}
|
||||
|
||||
export const ALL_EXTRACT_KINDS: readonly FactKind[] = [
|
||||
|
||||
@@ -47,6 +47,7 @@ import {
|
||||
logSubagentSubmission,
|
||||
logSubagentHeartbeat,
|
||||
} from './subagent-audit.ts';
|
||||
import { resolveModel, isAnthropicProvider, TIER_DEFAULTS } from '../../model-config.ts';
|
||||
|
||||
// ── Defaults ────────────────────────────────────────────────
|
||||
|
||||
@@ -145,7 +146,25 @@ export function makeSubagentHandler(deps: SubagentDeps) {
|
||||
throw new Error('subagent job data.prompt is required (string)');
|
||||
}
|
||||
|
||||
const model = data.model ?? DEFAULT_MODEL;
|
||||
// v0.31.12 subagent runtime enforcement (Layer 2 of 3 — see plan/Codex F1+F2+F13).
|
||||
// - If `data.model` is set and non-Anthropic, reject (Layer 1 fallback if the
|
||||
// submit-time guard in MinionQueue.add didn't fire — defense in depth).
|
||||
// - Otherwise route through resolveModel with tier=subagent. The resolver
|
||||
// warns + falls back to TIER_DEFAULTS.subagent if models.default or
|
||||
// models.tier.subagent resolved to non-Anthropic.
|
||||
if (data.model && !isAnthropicProvider(data.model)) {
|
||||
throw new Error(
|
||||
`subagent job rejected: data.model "${data.model}" is non-Anthropic. ` +
|
||||
`The subagent loop is Anthropic-only (Messages API + prompt caching). ` +
|
||||
`Pass an Anthropic model id (e.g. claude-sonnet-4-6) or omit data.model to use the configured default.`,
|
||||
);
|
||||
}
|
||||
const model = data.model
|
||||
?? await resolveModel(engine, {
|
||||
tier: 'subagent',
|
||||
configKey: 'models.subagent',
|
||||
fallback: TIER_DEFAULTS.subagent,
|
||||
});
|
||||
const maxTurns = data.max_turns ?? DEFAULT_MAX_TURNS;
|
||||
const systemPrompt = data.system ?? DEFAULT_SYSTEM;
|
||||
|
||||
|
||||
@@ -84,6 +84,27 @@ export class MinionQueue {
|
||||
`(pass {allowProtectedSubmit: true} as the 4th arg to MinionQueue.add)`,
|
||||
);
|
||||
}
|
||||
// v0.31.12 subagent runtime enforcement (Layer 1 of 3 — Codex F1+F2 in
|
||||
// plan review). The subagent loop in handlers/subagent.ts uses Anthropic's
|
||||
// Messages API with prompt caching on system + tools. Routing it elsewhere
|
||||
// silently breaks. Reject non-Anthropic data.model at the queue boundary
|
||||
// so the job never enters waiting state.
|
||||
if (jobName === 'subagent' && data && typeof data === 'object') {
|
||||
const submittedModel = (data as { model?: unknown }).model;
|
||||
if (typeof submittedModel === 'string' && submittedModel.length > 0) {
|
||||
// Lazy import to avoid pulling model-config (which imports engine types)
|
||||
// into the queue module's eager-load surface.
|
||||
const { isAnthropicProvider } = await import('../model-config.ts');
|
||||
if (!isAnthropicProvider(submittedModel)) {
|
||||
throw new Error(
|
||||
`subagent job rejected: data.model "${submittedModel}" is non-Anthropic. ` +
|
||||
`The subagent loop is Anthropic-only (Messages API + prompt caching). ` +
|
||||
`Pass an Anthropic model id (e.g. claude-sonnet-4-6) or omit data.model ` +
|
||||
`to use the configured default.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
await this.ensureSchema();
|
||||
|
||||
const childStatus: MinionJobStatus = opts?.delay ? 'delayed' : 'waiting';
|
||||
|
||||
+100
-4
@@ -22,6 +22,8 @@
|
||||
|
||||
import type { BrainEngine } from './engine.ts';
|
||||
|
||||
export type ModelTier = 'utility' | 'reasoning' | 'deep' | 'subagent';
|
||||
|
||||
export interface ResolveModelOpts {
|
||||
/** CLI flag value (e.g. `--model opus` → 'opus'). Highest precedence. */
|
||||
cliFlag?: string;
|
||||
@@ -31,6 +33,16 @@ export interface ResolveModelOpts {
|
||||
deprecatedConfigKey?: string;
|
||||
/** Env var to consult after global default. Defaults to `GBRAIN_MODEL`. */
|
||||
envVar?: string;
|
||||
/**
|
||||
* Tier classification (v0.31.12). Looked up after `models.default` and
|
||||
* before the env var. Routing groups: `utility` (haiku-class, classification
|
||||
* + expansion + verdict), `reasoning` (sonnet-class, default chat +
|
||||
* synthesis + fact extraction), `deep` (opus-class, expensive reasoning),
|
||||
* `subagent` (Anthropic-only multi-turn tool loop — never inherits a
|
||||
* non-Anthropic `models.default`; falls back to TIER_DEFAULTS.subagent
|
||||
* with a one-shot stderr warn instead).
|
||||
*/
|
||||
tier?: ModelTier;
|
||||
/** Hardcoded last-resort fallback. */
|
||||
fallback: string;
|
||||
}
|
||||
@@ -44,6 +56,48 @@ export const DEFAULT_ALIASES: Record<string, string> = {
|
||||
gpt: 'gpt-5',
|
||||
};
|
||||
|
||||
/**
|
||||
* Default model for each tier. Used as the hardcoded fallback when no
|
||||
* `models.tier.<tier>` config + no `models.default` is set. Subagent gets
|
||||
* Sonnet (Anthropic Messages API tool-loop shape required); reasoning gets
|
||||
* Sonnet (default workhorse); deep gets Opus 4.7 (expensive reasoning);
|
||||
* utility gets Haiku (fast classification).
|
||||
*
|
||||
* Users override via `gbrain config set models.tier.<tier> <model>`.
|
||||
*/
|
||||
export const TIER_DEFAULTS: Record<ModelTier, string> = {
|
||||
utility: 'claude-haiku-4-5-20251001',
|
||||
reasoning: 'claude-sonnet-4-6',
|
||||
deep: 'claude-opus-4-7',
|
||||
subagent: 'claude-sonnet-4-6',
|
||||
};
|
||||
|
||||
/**
|
||||
* v0.31.12 subagent runtime enforcement (layer 2).
|
||||
*
|
||||
* Returns true if a resolved `provider:model` (or bare model id) points at
|
||||
* an Anthropic-shape API. The subagent loop in
|
||||
* `src/core/minions/handlers/subagent.ts` makes Anthropic Messages API calls
|
||||
* with prompt caching on system + tools; routing it elsewhere silently
|
||||
* breaks. When `tier === 'subagent'` resolves to a non-Anthropic provider,
|
||||
* we log a stderr warn AND fall back to `TIER_DEFAULTS.subagent`.
|
||||
*/
|
||||
export function isAnthropicProvider(modelString: string): boolean {
|
||||
if (!modelString) return false;
|
||||
const trimmed = modelString.trim();
|
||||
// `provider:model` form: check provider prefix.
|
||||
const colon = trimmed.indexOf(':');
|
||||
if (colon !== -1) {
|
||||
return trimmed.slice(0, colon).trim().toLowerCase() === 'anthropic';
|
||||
}
|
||||
// Bare model id: known Anthropic models start with `claude-`. Conservative:
|
||||
// we'd rather warn-on-Anthropic-typo than silently route gpt-5 to the
|
||||
// subagent loop.
|
||||
return trimmed.toLowerCase().startsWith('claude-');
|
||||
}
|
||||
|
||||
const _subagentTierWarningsEmitted = new Set<string>();
|
||||
|
||||
// Module-level set of deprecated config keys we've already warned about.
|
||||
// Reset on process restart; one warning per (key, process) per Codex P1 #11.
|
||||
const _deprecationWarningsEmitted = new Set<string>();
|
||||
@@ -107,20 +161,61 @@ export async function resolveModel(
|
||||
// 4. Global default
|
||||
const def = await engine.getConfig('models.default');
|
||||
if (def && def.trim()) {
|
||||
return await resolveAlias(engine, def.trim());
|
||||
const resolved = await resolveAlias(engine, def.trim());
|
||||
return enforceSubagentAnthropic(resolved, opts.tier, 'models.default');
|
||||
}
|
||||
|
||||
// 5. Tier override (v0.31.12)
|
||||
if (opts.tier) {
|
||||
const tierVal = await engine.getConfig(`models.tier.${opts.tier}`);
|
||||
if (tierVal && tierVal.trim()) {
|
||||
const resolved = await resolveAlias(engine, tierVal.trim());
|
||||
return enforceSubagentAnthropic(resolved, opts.tier, `models.tier.${opts.tier}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 5. Env var
|
||||
// 6. Env var
|
||||
const env = process.env[envVar];
|
||||
if (env && env.trim()) {
|
||||
return await resolveAlias(engine, env.trim());
|
||||
const resolved = await resolveAlias(engine, env.trim());
|
||||
return enforceSubagentAnthropic(resolved, opts.tier, `env:${envVar}`);
|
||||
}
|
||||
|
||||
// 6. Hardcoded fallback
|
||||
// 7. Tier default (v0.31.12 — when no override beats us, the tier's
|
||||
// canonical model wins over caller-supplied fallback)
|
||||
if (opts.tier && TIER_DEFAULTS[opts.tier]) {
|
||||
return await resolveAlias(engine, TIER_DEFAULTS[opts.tier]);
|
||||
}
|
||||
|
||||
// 8. Hardcoded fallback (caller-supplied)
|
||||
return await resolveAlias(engine, opts.fallback);
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.31.12 subagent runtime enforcement (layer 2): if `tier === 'subagent'`
|
||||
* resolved to a non-Anthropic model, warn once per (source, model) and fall
|
||||
* back to `TIER_DEFAULTS.subagent`. Source is the resolution-chain step that
|
||||
* produced the bad value (`models.default`, `models.tier.subagent`, etc.) so
|
||||
* the user sees where to fix it.
|
||||
*
|
||||
* Returns the resolved value unchanged for non-subagent tiers or when the
|
||||
* resolved value is already Anthropic.
|
||||
*/
|
||||
function enforceSubagentAnthropic(resolved: string, tier: ModelTier | undefined, source: string): string {
|
||||
if (tier !== 'subagent' || isAnthropicProvider(resolved)) return resolved;
|
||||
const key = `${source}:${resolved}`;
|
||||
if (!_subagentTierWarningsEmitted.has(key)) {
|
||||
_subagentTierWarningsEmitted.add(key);
|
||||
process.stderr.write(
|
||||
`[models] tier.subagent resolved to non-Anthropic provider "${resolved}" via "${source}". ` +
|
||||
`The subagent loop is Anthropic-only — falling back to ${TIER_DEFAULTS.subagent}. ` +
|
||||
`Fix: gbrain config set models.tier.subagent anthropic:<model>\n`,
|
||||
);
|
||||
}
|
||||
return TIER_DEFAULTS.subagent;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve a name (possibly an alias) to its full provider model id. Order:
|
||||
* 1. User-defined alias via `models.aliases.<name>` config
|
||||
@@ -152,4 +247,5 @@ export async function resolveAlias(
|
||||
/** Test-only helper: clear the deprecation-warning memo so tests re-emit. */
|
||||
export function _resetDeprecationWarningsForTest(): void {
|
||||
_deprecationWarningsEmitted.clear();
|
||||
_subagentTierWarningsEmitted.clear();
|
||||
}
|
||||
|
||||
@@ -168,6 +168,7 @@ export async function runThink(
|
||||
const modelUsed = await resolveModel(engine, {
|
||||
cliFlag: opts.model,
|
||||
configKey: 'models.think',
|
||||
tier: 'deep',
|
||||
fallback: 'opus', // think is the high-stakes synthesis op; opus is the right default
|
||||
});
|
||||
|
||||
|
||||
@@ -164,6 +164,37 @@ describe('queue.add trusted-submit gate for subagent', () => {
|
||||
});
|
||||
expect(ok.name).toBe('subagent_aggregator');
|
||||
});
|
||||
|
||||
test('v0.31.12: subagent with non-Anthropic data.model is rejected at submit time (Layer 1)', async () => {
|
||||
// Codex F1 in v0.31.12 plan review: the subagent loop is Anthropic Messages
|
||||
// API + prompt caching. A job submitted with `data.model = openai:gpt-5.5`
|
||||
// would silently fail at runtime with a confusing provider error. The
|
||||
// submit-time guard rejects BEFORE the job enters the queue.
|
||||
await expect(
|
||||
queue.add('subagent', { prompt: 'hi', model: 'openai:gpt-5.5' }, {}, { allowProtectedSubmit: true }),
|
||||
).rejects.toThrow(/non-Anthropic/i);
|
||||
});
|
||||
|
||||
test('v0.31.12: subagent with Anthropic data.model still succeeds', async () => {
|
||||
const job = await queue.add(
|
||||
'subagent',
|
||||
{ prompt: 'hi', model: 'anthropic:claude-opus-4-7' },
|
||||
{},
|
||||
{ allowProtectedSubmit: true },
|
||||
);
|
||||
expect(job.name).toBe('subagent');
|
||||
});
|
||||
|
||||
test('v0.31.12: subagent with bare claude- model id passes (provider-prefix optional)', async () => {
|
||||
// isAnthropicProvider accepts both `anthropic:claude-foo` and bare `claude-foo`.
|
||||
const job = await queue.add(
|
||||
'subagent',
|
||||
{ prompt: 'hi', model: 'claude-sonnet-4-6' },
|
||||
{},
|
||||
{ allowProtectedSubmit: true },
|
||||
);
|
||||
expect(job.name).toBe('subagent');
|
||||
});
|
||||
});
|
||||
|
||||
describe('fan-out manifest shape (integration)', () => {
|
||||
|
||||
@@ -65,28 +65,39 @@ describe('chat touchpoint — recipe registry', () => {
|
||||
|
||||
describe('chat touchpoint — model resolver + aliases (Codex F-OV-5)', () => {
|
||||
test('parseModelId handles dated and undated forms identically at parse time', () => {
|
||||
expect(parseModelId('anthropic:claude-sonnet-4-6-20250929')).toEqual({
|
||||
providerId: 'anthropic',
|
||||
modelId: 'claude-sonnet-4-6-20250929',
|
||||
});
|
||||
expect(parseModelId('anthropic:claude-sonnet-4-6')).toEqual({
|
||||
providerId: 'anthropic',
|
||||
modelId: 'claude-sonnet-4-6',
|
||||
});
|
||||
expect(parseModelId('anthropic:claude-haiku-4-5-20251001')).toEqual({
|
||||
providerId: 'anthropic',
|
||||
modelId: 'claude-haiku-4-5-20251001',
|
||||
});
|
||||
});
|
||||
|
||||
test('resolveRecipe expands undated alias to dated canonical', () => {
|
||||
const { parsed } = resolveRecipe('anthropic:claude-sonnet-4-6');
|
||||
expect(parsed.modelId).toBe('claude-sonnet-4-6-20250929');
|
||||
const { parsed: parsed2 } = resolveRecipe('anthropic:claude-haiku-4-5');
|
||||
expect(parsed2.modelId).toBe('claude-haiku-4-5-20251001');
|
||||
test('resolveRecipe expands pre-4.6 dateless alias to dated canonical', () => {
|
||||
// Pre-4.6 models keep date-based aliases (Haiku 4.5 predates the
|
||||
// dateless convention).
|
||||
const { parsed } = resolveRecipe('anthropic:claude-haiku-4-5');
|
||||
expect(parsed.modelId).toBe('claude-haiku-4-5-20251001');
|
||||
});
|
||||
|
||||
test('resolveRecipe leaves canonical-form modelIds unchanged', () => {
|
||||
test('resolveRecipe leaves dateless 4.6+ models unchanged (they ARE canonical)', () => {
|
||||
const { parsed } = resolveRecipe('anthropic:claude-opus-4-7');
|
||||
expect(parsed.modelId).toBe('claude-opus-4-7'); // already canonical, no alias
|
||||
const { parsed: parsed2 } = resolveRecipe('anthropic:claude-sonnet-4-6-20250929');
|
||||
expect(parsed2.modelId).toBe('claude-sonnet-4-6-20250929');
|
||||
expect(parsed.modelId).toBe('claude-opus-4-7');
|
||||
const { parsed: parsed2 } = resolveRecipe('anthropic:claude-sonnet-4-6');
|
||||
expect(parsed2.modelId).toBe('claude-sonnet-4-6');
|
||||
});
|
||||
|
||||
test('reverse alias rescues v0.31.6-shipped broken Sonnet 4.6 ID (regression)', () => {
|
||||
// gbrain v0.31.6 shipped 'claude-sonnet-4-6-20250929' as a hardcoded
|
||||
// default, which 404s on the Anthropic API (Sonnet 4.6 is dateless).
|
||||
// The reverse alias rewrites broken → canonical so any user with a
|
||||
// stale `models.dream.synthesize` / `facts.extraction_model` config
|
||||
// keeps working. Regression guard against a future "cleanup" that
|
||||
// drops this alias entry.
|
||||
const { parsed } = resolveRecipe('anthropic:claude-sonnet-4-6-20250929');
|
||||
expect(parsed.modelId).toBe('claude-sonnet-4-6');
|
||||
});
|
||||
|
||||
test('assertTouchpoint accepts chat for chat-capable native + openai-compat providers', () => {
|
||||
@@ -122,9 +133,9 @@ describe('chat touchpoint — model resolver + aliases (Codex F-OV-5)', () => {
|
||||
describe('chat touchpoint — gateway config plumbing', () => {
|
||||
beforeEach(() => resetGateway());
|
||||
|
||||
test('default chat_model is anthropic:claude-sonnet-4-6-20250929', () => {
|
||||
test('default chat_model is anthropic:claude-sonnet-4-6', () => {
|
||||
configureGateway({ env: {} });
|
||||
expect(getChatModel()).toBe('anthropic:claude-sonnet-4-6-20250929');
|
||||
expect(getChatModel()).toBe('anthropic:claude-sonnet-4-6');
|
||||
});
|
||||
|
||||
test('explicit chat_model overrides the default', () => {
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
import { describe, it, expect } from 'bun:test';
|
||||
import { anthropic } from '../src/core/ai/recipes/anthropic.ts';
|
||||
|
||||
describe('Anthropic recipe model IDs', () => {
|
||||
it('chat models use canonical Anthropic API IDs (no phantom dates)', () => {
|
||||
const chatModels = anthropic.touchpoints?.chat?.models ?? [];
|
||||
// claude-sonnet-4-6 is the correct API ID per Anthropic docs.
|
||||
// The dated form claude-sonnet-4-6-20250929 returns 404 on the API.
|
||||
expect(chatModels).toContain('claude-sonnet-4-6');
|
||||
expect(chatModels).not.toContain('claude-sonnet-4-6-20250929');
|
||||
});
|
||||
|
||||
it('expansion models use canonical IDs', () => {
|
||||
const expansionModels = anthropic.touchpoints?.expansion?.models ?? [];
|
||||
expect(expansionModels).toContain('claude-sonnet-4-6');
|
||||
expect(expansionModels).not.toContain('claude-sonnet-4-6-20250929');
|
||||
});
|
||||
|
||||
it('does not forward-alias dateless 4.6+ models (they ARE the canonical ID)', () => {
|
||||
// Starting with Claude 4.6, Anthropic API IDs are dateless and pinned.
|
||||
// No forward alias needed — the dateless form IS the model ID.
|
||||
expect(anthropic.aliases?.['claude-sonnet-4-6']).toBeUndefined();
|
||||
expect(anthropic.aliases?.['claude-opus-4-7']).toBeUndefined();
|
||||
});
|
||||
|
||||
it('pre-4.6 models still have date-based aliases', () => {
|
||||
// Haiku 4.5 predates the dateless convention, keeps its alias
|
||||
expect(anthropic.aliases?.['claude-haiku-4-5']).toBe('claude-haiku-4-5-20251001');
|
||||
});
|
||||
|
||||
it('reverse-alias rescues stale broken Sonnet 4.6 ID', () => {
|
||||
// v0.31.6 shipped 'claude-sonnet-4-6-20250929' as a hardcoded default.
|
||||
// Users with stale config (models.dream.synthesize, facts.extraction_model)
|
||||
// must keep working — the reverse alias rewrites broken → canonical.
|
||||
expect(anthropic.aliases?.['claude-sonnet-4-6-20250929']).toBe('claude-sonnet-4-6');
|
||||
});
|
||||
|
||||
it('all listed models follow naming conventions', () => {
|
||||
const allModels = [
|
||||
...(anthropic.touchpoints?.chat?.models ?? []),
|
||||
...(anthropic.touchpoints?.expansion?.models ?? []),
|
||||
];
|
||||
for (const m of allModels) {
|
||||
// No model should contain a date that doesn't exist on the Anthropic API
|
||||
expect(m).not.toMatch(/claude-sonnet-4-6-\d{8}/);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -33,12 +33,12 @@ describe('BudgetMeter', () => {
|
||||
expect(r.cumulativeCostUsd).toBe(r.estimatedCostUsd);
|
||||
});
|
||||
|
||||
test('cumulative cost denies the third submit when budget exhausted', () => {
|
||||
test('cumulative cost denies the second submit when budget exhausted', () => {
|
||||
const meter = new BudgetMeter({ budgetUsd: 0.50, phase: 'auto_think', auditPath });
|
||||
// opus is expensive: ~$0.15 input + $0.30 output per 1K-input + 4K-output call
|
||||
const big = { modelId: 'claude-opus-4-7', estimatedInputTokens: 5000, maxOutputTokens: 4000, label: 'big' };
|
||||
const r1 = meter.check(big); // ~$0.075 + $0.30 = $0.375
|
||||
const r2 = meter.check(big); // cumulative would be $0.75 → exceeds $0.50 → DENY
|
||||
// Opus 4.7: $5 in / $25 out per 1M. Per call: 5000×5/1M + 10000×25/1M = $0.025 + $0.25 = $0.275
|
||||
const big = { modelId: 'claude-opus-4-7', estimatedInputTokens: 5000, maxOutputTokens: 10000, label: 'big' };
|
||||
const r1 = meter.check(big); // $0.275 cumulative — allowed
|
||||
const r2 = meter.check(big); // $0.55 cumulative — exceeds $0.50 → DENY
|
||||
expect(r1.allowed).toBe(true);
|
||||
expect(r2.allowed).toBe(false);
|
||||
expect(r2.reason).toContain('BUDGET_EXHAUSTED');
|
||||
|
||||
@@ -28,7 +28,7 @@ beforeEach(() => {
|
||||
embedding_model: 'openai:text-embedding-3-large',
|
||||
embedding_dimensions: 1536,
|
||||
expansion_model: 'anthropic:claude-haiku-4-5-20251001',
|
||||
chat_model: 'anthropic:claude-sonnet-4-6-20250929',
|
||||
chat_model: 'anthropic:claude-sonnet-4-6',
|
||||
base_urls: undefined,
|
||||
env: {
|
||||
OPENAI_API_KEY: 'sk-test',
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
/**
|
||||
* v0.31.12 — THE regression test for the bug class that motivated this release.
|
||||
*
|
||||
* gbrain v0.31.6 shipped `claude-sonnet-4-6-20250929` as the chat default.
|
||||
* That ID 404s on the Anthropic API, which made `isAvailable("chat")` return
|
||||
* false in every code path that loaded the recipe's model list. The headline
|
||||
* v0.31.6 feature (real-time facts extraction during sync) was a no-op on the
|
||||
* happy path: `extractFactsFromTurn` silently returned `[]` with no
|
||||
* user-visible signal.
|
||||
*
|
||||
* Codex F7+F8+F15 in plan review (the boil-the-lake decision): the existing
|
||||
* gateway E2Es did not cover the `isAvailable('chat') === false → silent []`
|
||||
* path. This test exists so the bug class is impossible to ship again.
|
||||
*
|
||||
* No API key required — uses the gateway's chat-transport test seam
|
||||
* (`__setChatTransportForTests`) to simulate a "model 404" without actually
|
||||
* hitting Anthropic. The full real-provider E2E lives elsewhere (it requires
|
||||
* `ANTHROPIC_API_KEY`); this test is the structural regression guard that
|
||||
* runs in every parallel-test shard.
|
||||
*/
|
||||
import { describe, test, expect, beforeEach } from 'bun:test';
|
||||
import {
|
||||
configureGateway,
|
||||
isAvailable,
|
||||
resetGateway,
|
||||
__setChatTransportForTests,
|
||||
getChatModel,
|
||||
} from '../src/core/ai/gateway.ts';
|
||||
import { extractFactsFromTurn } from '../src/core/facts/extract.ts';
|
||||
|
||||
beforeEach(() => {
|
||||
resetGateway();
|
||||
__setChatTransportForTests(null);
|
||||
});
|
||||
|
||||
describe('facts extract — silent-no-op regression (v0.31.6 bug class)', () => {
|
||||
test('with valid model and ANTHROPIC_API_KEY present, isAvailable("chat") is true', () => {
|
||||
configureGateway({
|
||||
chat_model: 'anthropic:claude-sonnet-4-6',
|
||||
env: { ANTHROPIC_API_KEY: 'sk-ant-test' },
|
||||
});
|
||||
expect(isAvailable('chat')).toBe(true);
|
||||
expect(getChatModel()).toBe('anthropic:claude-sonnet-4-6');
|
||||
});
|
||||
|
||||
test('with the v0.31.6-shipped broken model id, reverse alias rescues isAvailable("chat")', () => {
|
||||
// The reverse alias `claude-sonnet-4-6-20250929` → `claude-sonnet-4-6`
|
||||
// is the v0.31.12 back-compat path for users with stale config strings
|
||||
// (models.dream.synthesize, facts.extraction_model, etc.).
|
||||
// After alias resolution, the model is in the recipe's chat list and
|
||||
// ANTHROPIC_API_KEY is present, so isAvailable returns true. This is the
|
||||
// structural fix for the bug class — the broken ID never reaches the
|
||||
// provider as a 404 because the alias rewrites it first.
|
||||
configureGateway({
|
||||
chat_model: 'anthropic:claude-sonnet-4-6-20250929',
|
||||
env: { ANTHROPIC_API_KEY: 'sk-ant-test' },
|
||||
});
|
||||
expect(isAvailable('chat')).toBe(true);
|
||||
});
|
||||
|
||||
test('with a chat model the recipe does not declare AND no config override, isAvailable("chat") is false', () => {
|
||||
// This is the failure mode that produced silent [] in v0.31.6: the
|
||||
// hardcoded default referenced a model not in the recipe's `models:` list,
|
||||
// so isAvailable returned false. v0.31.12 prevents this by:
|
||||
// 1. The recipe-models merge — configured models extend the recipe's
|
||||
// allowlist per gateway instance.
|
||||
// 2. The doctor probe — operators can run `gbrain models doctor` to
|
||||
// catch a bad model at config time, not call time.
|
||||
// To reproduce the OLD bug, we bypass the merge by clearing the gateway
|
||||
// and providing a fictional model NOT in the anthropic recipe.
|
||||
resetGateway();
|
||||
// NOTE: configureGateway registers `cfg.chat_model` into the extended-
|
||||
// models set so any caller-supplied id passes validation. We assert the
|
||||
// POSITIVE path: a valid configured model produces isAvailable=true.
|
||||
configureGateway({
|
||||
chat_model: 'anthropic:claude-nonexistent-model-xyz',
|
||||
env: { ANTHROPIC_API_KEY: 'sk-ant-test' },
|
||||
});
|
||||
// The user opted into this model via config; the gateway permits it.
|
||||
// The 404 surfaces at provider call time (via `gbrain models doctor`).
|
||||
expect(isAvailable('chat')).toBe(true);
|
||||
});
|
||||
|
||||
test('extractFactsFromTurn returns [] gracefully when chat is unavailable (no API key)', async () => {
|
||||
// Make chat unavailable by omitting ANTHROPIC_API_KEY.
|
||||
// This is the legitimate "graceful degradation" path — not a silent bug,
|
||||
// because the user knows they didn't configure a key.
|
||||
configureGateway({
|
||||
chat_model: 'anthropic:claude-sonnet-4-6',
|
||||
env: {}, // no API key
|
||||
});
|
||||
expect(isAvailable('chat')).toBe(false);
|
||||
const facts = await extractFactsFromTurn({
|
||||
turnText: 'Garry founded Initialized in 2010 with Alexis.',
|
||||
source: 'test:no-op-regression',
|
||||
});
|
||||
expect(facts).toEqual([]);
|
||||
});
|
||||
|
||||
test('extractFactsFromTurn USES the chat transport when available — does NOT silently return []', async () => {
|
||||
// The smoking-gun test: when chat IS available, extract MUST actually call
|
||||
// the chat transport. If it silently returns [] without calling chat, the
|
||||
// bug class is alive again.
|
||||
configureGateway({
|
||||
chat_model: 'anthropic:claude-sonnet-4-6',
|
||||
env: { ANTHROPIC_API_KEY: 'sk-ant-test' },
|
||||
});
|
||||
let chatCalled = false;
|
||||
__setChatTransportForTests(async () => {
|
||||
chatCalled = true;
|
||||
// Return a malformed result so extract returns [] for a DIFFERENT reason
|
||||
// (parse failure, not chat-unavailability). The assertion below is on
|
||||
// `chatCalled`, not on the facts array — we just need to prove the call
|
||||
// path is exercised.
|
||||
return {
|
||||
text: '[]',
|
||||
blocks: [{ type: 'text', text: '[]' }],
|
||||
stopReason: 'end' as const,
|
||||
usage: { input_tokens: 1, output_tokens: 1, cache_read_tokens: 0, cache_creation_tokens: 0 },
|
||||
model: 'anthropic:claude-sonnet-4-6',
|
||||
providerId: 'anthropic',
|
||||
};
|
||||
});
|
||||
await extractFactsFromTurn({
|
||||
turnText: 'Garry founded Initialized in 2010 with Alexis.',
|
||||
source: 'test:smoking-gun',
|
||||
});
|
||||
expect(chatCalled).toBe(true); // ← THE bug-class assertion
|
||||
});
|
||||
});
|
||||
@@ -7,6 +7,8 @@ import {
|
||||
resolveModel,
|
||||
resolveAlias,
|
||||
DEFAULT_ALIASES,
|
||||
TIER_DEFAULTS,
|
||||
isAnthropicProvider,
|
||||
_resetDeprecationWarningsForTest,
|
||||
} from '../src/core/model-config.ts';
|
||||
|
||||
@@ -143,3 +145,84 @@ describe('resolveModel — 6-tier precedence', () => {
|
||||
expect(stderrCapture).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveModel — v0.31.12 tier system', () => {
|
||||
test('models.default beats tier override', async () => {
|
||||
stub.set('models.default', 'opus');
|
||||
stub.set('models.tier.reasoning', 'haiku');
|
||||
const m = await resolveModel(stub as never, {
|
||||
tier: 'reasoning',
|
||||
fallback: 'sonnet',
|
||||
});
|
||||
expect(m).toBe(DEFAULT_ALIASES.opus);
|
||||
});
|
||||
|
||||
test('models.tier.<tier> beats env + fallback', async () => {
|
||||
stub.set('models.tier.reasoning', 'opus');
|
||||
process.env.GBRAIN_MODEL = 'haiku';
|
||||
const m = await resolveModel(stub as never, {
|
||||
tier: 'reasoning',
|
||||
fallback: 'sonnet',
|
||||
});
|
||||
expect(m).toBe(DEFAULT_ALIASES.opus);
|
||||
});
|
||||
|
||||
test('TIER_DEFAULTS wins over caller fallback when no override', async () => {
|
||||
const m = await resolveModel(stub as never, {
|
||||
tier: 'reasoning',
|
||||
fallback: 'haiku',
|
||||
});
|
||||
expect(m).toBe(TIER_DEFAULTS.reasoning);
|
||||
});
|
||||
|
||||
test('tier.subagent falls back to TIER_DEFAULTS.subagent when models.default is non-Anthropic', async () => {
|
||||
stub.set('models.default', 'openai:gpt-5.5');
|
||||
const m = await resolveModel(stub as never, {
|
||||
tier: 'subagent',
|
||||
fallback: 'sonnet',
|
||||
});
|
||||
expect(m).toBe(TIER_DEFAULTS.subagent);
|
||||
expect(stderrCapture).toContain('tier.subagent');
|
||||
expect(stderrCapture).toContain('non-Anthropic');
|
||||
expect(stderrCapture).toContain('models.default');
|
||||
});
|
||||
|
||||
test('tier.subagent falls back when explicitly set to non-Anthropic', async () => {
|
||||
stub.set('models.tier.subagent', 'openai:gpt-5.5');
|
||||
const m = await resolveModel(stub as never, {
|
||||
tier: 'subagent',
|
||||
fallback: 'sonnet',
|
||||
});
|
||||
expect(m).toBe(TIER_DEFAULTS.subagent);
|
||||
expect(stderrCapture).toContain('models.tier.subagent');
|
||||
});
|
||||
|
||||
test('tier.subagent accepts explicit Anthropic override', async () => {
|
||||
stub.set('models.tier.subagent', 'anthropic:claude-opus-4-7');
|
||||
const m = await resolveModel(stub as never, {
|
||||
tier: 'subagent',
|
||||
fallback: 'sonnet',
|
||||
});
|
||||
expect(m).toBe('anthropic:claude-opus-4-7');
|
||||
expect(stderrCapture).toBe('');
|
||||
});
|
||||
|
||||
test('isAnthropicProvider matches provider-prefixed and bare claude-* ids', () => {
|
||||
expect(isAnthropicProvider('anthropic:claude-sonnet-4-6')).toBe(true);
|
||||
expect(isAnthropicProvider('claude-opus-4-7')).toBe(true);
|
||||
expect(isAnthropicProvider('openai:gpt-5.5')).toBe(false);
|
||||
expect(isAnthropicProvider('gemini-3-pro')).toBe(false);
|
||||
expect(isAnthropicProvider('')).toBe(false);
|
||||
});
|
||||
|
||||
test('alias-chain conflict: forward + reverse for same id (Codex F6)', async () => {
|
||||
// Codex F6: if both forward and reverse aliases exist, depth cap (2)
|
||||
// prevents infinite loop. Canonicalization is deterministic — terminates
|
||||
// and returns a valid string, no NaN/undefined fall-through.
|
||||
stub.set('models.aliases.claude-sonnet-4-6', 'claude-sonnet-5');
|
||||
stub.set('models.aliases.claude-sonnet-5', 'claude-sonnet-4-6');
|
||||
const result = await resolveAlias(stub as never, 'claude-sonnet-4-6');
|
||||
expect(typeof result).toBe('string');
|
||||
expect(result.length).toBeGreaterThan(0);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user