mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
* feat(core): add splitProviderModelId centralizer for pricing-side parsing
New pure helper in src/core/model-id.ts that splits provider:model,
provider/model, and bare model strings into a {provider, model} pair.
Defensive contract: null/undefined/empty/whitespace returns
{provider: null, model: ''}.
Will be wired into the 5 pricing/budget sites in the next commit.
Named splitProviderModelId (not parseModelId) to avoid the in-project
collision with the gateway-side src/core/ai/model-resolver.ts:parseModelId
which has a different bare-name contract.
Pinned by 16 cases in test/model-id.test.ts covering all separator
forms plus defensive + edge inputs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(gateway): accept slash-form provider id in model-resolver
src/core/ai/model-resolver.ts:parseModelId now accepts both
provider:model (colon) and provider/model (slash) forms. Colon wins
when both separators present so OpenRouter nested ids like
openrouter:anthropic/claude-sonnet-4.6 route as
{providerId: 'openrouter', modelId: 'anthropic/claude-sonnet-4.6'}.
Pre-fix: every gateway entry point (chat / embed / rerank) threw
AIConfigError 'missing a provider prefix' on slash form ids. That
meant CLI users running
gbrain brainstorm --judge-model anthropic/claude-sonnet-4-6
would still fail mid-judge with AIConfigError even after pricing
was relaxed to accept slash form. Closes the end-to-end bug class.
Bare names without ANY separator still throw — gateway routing
always needs an explicit provider. Existing tests pinning that
throw (test/ai/capabilities.test.ts:43) stay green.
Pinned by 10 cases in test/ai/model-resolver-slash.test.ts
including a resolveRecipe round-trip that slash and colon forms
land on the same recipe.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor: route 5 pricing/config sites through splitProviderModelId
Five sites had inline ':'-only provider-prefix splits that silently
missed slash-form ids. Centralizing through splitProviderModelId
closes the bug class:
- src/core/anthropic-pricing.ts:estimateMaxCostUsd
- src/core/budget/budget-tracker.ts:lookupPricing (closes the
headline BudgetExhausted no_pricing failure on --max-cost +
slash-form --judge-model)
- src/core/eval-contradictions/cost-tracker.ts:pricingFor
(legacy silent-Haiku fallback preserved per plan D9)
- src/core/minions/batch-projection.ts (deleted bareModel inline
helper; inlined splitProviderModelId at 2 call sites)
- src/core/model-config.ts:isAnthropicProvider (silently fixed
v0.31.12 subagent-guard bypass for slash-form Anthropic ids)
Test gates land together so any bisect step is green:
- NEW test/anthropic-pricing.test.ts (7 cases including structural
regression guard: every ANTHROPIC_PRICING key reachable via all
three forms)
- NEW test/eval-contradictions/cost-tracker-slash.test.ts (6 cases
including legacy-Haiku-fallback pin)
- EXTENDED test/batch-projection.test.ts (slash + double-separator
cases)
- EXTENDED test/model-config.serial.test.ts (2 slash-form
isAnthropicProvider cases)
- EXTENDED test/core/budget/budget-tracker.test.ts (2 slash + colon
reserve() cases)
Behavior changes for slash-prefix ids only; bare and colon ids
unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(brainstorm): scale judge maxTokens with per-model output cap
Replace the hard-coded maxTokens: 4000 with computeJudgeMaxTokens
that scales with idea count and respects each model's actual output
cap.
Pre-fix: any judge call with 36+ ideas produced ~100 tokens/idea of
JSON that got truncated mid-output. parseJudgeJSON threw, orchestrator
surfaced judge_failed: true, all ideas saved unscored. Verified
failure mode on 72-idea fixture: 0/72 passing before, 39/72 after.
Formula: min(modelCap, max(LEGACY_MIN_MAX_TOKENS, ideaCount*150+500))
Named constants extracted at top of judges.ts:
- TOKEN_BUDGET_PER_IDEA = 150 (1.5x headroom over observed ~100/idea)
- TOKEN_BUDGET_ENVELOPE = 500 (JSON wrapper)
- LEGACY_MIN_MAX_TOKENS = 4000 (pre-fix floor preserved for 1-idea)
- MAX_OUTPUT_TOKENS_CEIL = 32_000 (fallback when model unknown)
- ANTHROPIC_OUTPUT_CAPS (per-model: Opus 4.7 = 32K, Sonnet 4.6 /
Haiku 4.5 = 64K, legacy 3.5 = 8K)
When the caller passes no modelOverride, the cap routes through the
gateway's actual configured chat model via getChatModel() so the
formula matches what chat() will use, not whatever the override
hints at. Pre-fix the undefined-override case fell back to 32K even
if the configured default was a legacy 8K model.
Pinned by 16 cases in test/brainstorm/judges-maxtokens.test.ts:
formula at 1/10/36/96/200/300 ideas, per-model cap binding (Haiku 3.5
8K, Opus 4.7 32K, Sonnet 4.6 64K), and integration via runJudge with
a stubbed chatFn that captures ChatOpts.maxTokens.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore: bump version and changelog (v0.41.21.0)
Brainstorm judge fix-wave: closes #1540 end-to-end. parseModelId
centralizer + gateway resolver slash-form acceptance + per-model
maxTokens cap.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs: update project documentation for v0.41.21.0
CLAUDE.md: add v0.41.21.0 annotations to brainstorm/judges + model-config
entries; add new key-files entry for src/core/model-id.ts (the shared
splitProviderModelId centralizer) and src/core/ai/model-resolver.ts
slash-form extension.
README.md: add user-facing callout for the brainstorm judge_failed +
slash-form pricing fix, mirroring the v0.41.19.0 callout shape.
llms-full.txt: regenerated to absorb the CLAUDE.md + README changes
(passes test/build-llms.test.ts drift guard).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
68 lines
3.0 KiB
TypeScript
68 lines
3.0 KiB
TypeScript
/**
|
|
* v0.41.20.0 — pin estimateMaxCostUsd across bare/colon/slash/unknown ids.
|
|
*
|
|
* No prior coverage existed for this helper. The slash-form bug class
|
|
* (#1540) refired here for OpenRouter and CLI `--judge-model` users
|
|
* before this fix; this file pins the centralized parse path so any
|
|
* future refactor of parseModelId or estimateMaxCostUsd can't silently
|
|
* drop slash-form support.
|
|
*/
|
|
|
|
import { describe, test, expect } from 'bun:test';
|
|
import { ANTHROPIC_PRICING, estimateMaxCostUsd } from '../src/core/anthropic-pricing.ts';
|
|
|
|
describe('estimateMaxCostUsd', () => {
|
|
// Sonnet 4.6 = $3 input / $15 output per MTok.
|
|
// 1M input + 0 output → $3.00
|
|
// 0 input + 1M output → $15.00
|
|
|
|
test('bare key claude-sonnet-4-6 → hits pricing', () => {
|
|
const cost = estimateMaxCostUsd('claude-sonnet-4-6', 1_000_000, 0);
|
|
expect(cost).toBeCloseTo(3.0, 5);
|
|
});
|
|
|
|
test('colon-prefixed anthropic:claude-sonnet-4-6 → hits pricing via tail', () => {
|
|
const cost = estimateMaxCostUsd('anthropic:claude-sonnet-4-6', 1_000_000, 0);
|
|
expect(cost).toBeCloseTo(3.0, 5);
|
|
});
|
|
|
|
test('slash-prefixed anthropic/claude-sonnet-4-6 → hits pricing via tail (THE FIX)', () => {
|
|
// Pre-v0.41.20.0: this returned null because the inline split only
|
|
// handled `:`. CLI `--judge-model anthropic/...` + `--max-cost N` then
|
|
// hit BudgetTracker no_pricing fail-closed.
|
|
const cost = estimateMaxCostUsd('anthropic/claude-sonnet-4-6', 1_000_000, 0);
|
|
expect(cost).toBeCloseTo(3.0, 5);
|
|
});
|
|
|
|
test('mixed input + output cost math', () => {
|
|
// 100K input + 50K output for opus 4.7 ($5/$25)
|
|
// = 0.1 * 5 + 0.05 * 25 = 0.5 + 1.25 = 1.75
|
|
const cost = estimateMaxCostUsd('anthropic/claude-opus-4-7', 100_000, 50_000);
|
|
expect(cost).toBeCloseTo(1.75, 5);
|
|
});
|
|
|
|
test('unknown model → returns null (caller warn-once + bypass)', () => {
|
|
expect(estimateMaxCostUsd('mistral:medium', 1_000, 1_000)).toBeNull();
|
|
expect(estimateMaxCostUsd('gpt-5', 1_000, 1_000)).toBeNull();
|
|
});
|
|
|
|
test('OpenRouter nested form returns null — tail is `anthropic/claude-...` which is not a pricing key', () => {
|
|
// Per D2 architecture: parseModelId returns {provider:'openrouter',
|
|
// model:'anthropic/claude-sonnet-4-6'}; lookup on the tail
|
|
// 'anthropic/claude-sonnet-4-6' misses (table has bare 'claude-sonnet-4-6').
|
|
// OpenRouter pricing is intentionally out of scope (TODO #2).
|
|
expect(estimateMaxCostUsd('openrouter:anthropic/claude-sonnet-4-6', 1_000, 1_000)).toBeNull();
|
|
});
|
|
|
|
test('every key in ANTHROPIC_PRICING is reachable via bare/colon/slash form', () => {
|
|
// Regression guard: if someone adds a new entry to ANTHROPIC_PRICING,
|
|
// it should be reachable via all three forms automatically (the route
|
|
// is structural, not per-key).
|
|
for (const key of Object.keys(ANTHROPIC_PRICING)) {
|
|
expect(estimateMaxCostUsd(key, 1_000_000, 0)).not.toBeNull();
|
|
expect(estimateMaxCostUsd(`anthropic:${key}`, 1_000_000, 0)).not.toBeNull();
|
|
expect(estimateMaxCostUsd(`anthropic/${key}`, 1_000_000, 0)).not.toBeNull();
|
|
}
|
|
});
|
|
});
|