evals: switch the default LLM judge to claude-haiku-4-5

Replaces gpt-5-mini-2025-08-07 as the default judge everywhere: the 147
eval configs plus the hardcoded defaults in the evals CLI, recipe composer,
comparison config generator, taubench user-simulator, trial runner and
OptimizeConfig.

Haiku is roughly 3x faster on the judging path and we are out of OpenAI
quota, so gpt-5-mini was failing closed on long eval runs.
This commit is contained in:
Andrew Park
2026-07-11 11:59:32 -07:00
parent f338d64b21
commit 08fe3a6c98
159 changed files with 165 additions and 165 deletions
+1 -1
View File
@@ -113,7 +113,7 @@ class TestOptimizeConfig:
assert cfg.max_trials == 20
assert cfg.early_stop_patience == 5
assert cfg.optimizer_model == "claude-sonnet-4-6"
assert cfg.judge_model == "gpt-5-mini-2025-08-07"
assert cfg.judge_model == "claude-haiku-4-5-20251001"
class TestNewEventTypes: