mirror of
https://github.com/open-jarvis/OpenJarvis.git
synced 2026-07-31 03:12:16 +00:00
evals: switch the default LLM judge to claude-haiku-4-5
Replaces gpt-5-mini-2025-08-07 as the default judge everywhere: the 147 eval configs plus the hardcoded defaults in the evals CLI, recipe composer, comparison config generator, taubench user-simulator, trial runner and OptimizeConfig. Haiku is roughly 3x faster on the judging path and we are out of OpenAI quota, so gpt-5-mini was failing closed on long eval runs.
This commit is contained in:
@@ -113,7 +113,7 @@ class TestOptimizeConfig:
|
||||
assert cfg.max_trials == 20
|
||||
assert cfg.early_stop_patience == 5
|
||||
assert cfg.optimizer_model == "claude-sonnet-4-6"
|
||||
assert cfg.judge_model == "gpt-5-mini-2025-08-07"
|
||||
assert cfg.judge_model == "claude-haiku-4-5-20251001"
|
||||
|
||||
|
||||
class TestNewEventTypes:
|
||||
|
||||
Reference in New Issue
Block a user