Andrew Park 08fe3a6c98 evals: switch the default LLM judge to claude-haiku-4-5 ...

Replaces gpt-5-mini-2025-08-07 as the default judge everywhere: the 147
eval configs plus the hardcoded defaults in the evals CLI, recipe composer,
comparison config generator, taubench user-simulator, trial runner and
OptimizeConfig.

Haiku is roughly 3x faster on the judging path and we are out of OpenAI
quota, so gpt-5-mini was failing closed on long eval runs.

2026-07-11 11:59:32 -07:00

..

a2a

security(server): authenticate WebSocket handshakes and A2A requests (#217)

2026-05-25 23:51:29 +00:00

agents

orchestrator: track the SFT training pipeline and fix ToolOrchestra tracing

2026-07-11 11:55:59 -07:00

analytics

(bug): telemetry fixes (#421)

2026-05-27 09:36:26 -07:00

bench

fix(channels): wire channel→agent handler and fix Telegram send pipeline (#94)

2026-03-20 18:35:18 -07:00

channels

fix(channels): use conversation_id (not channel type) as Discord reply destination (#495)

2026-06-03 19:54:03 -07:00

cli

fix(cli): escape exception text in agents-list error handler (#465)

2026-06-01 10:26:11 -07:00

connectors

fix(connectors): validate credentials before persisting + populate Gmail URLs (#410)

2026-05-25 11:07:35 -07:00

core

fix(config): honor system_prompt.prefix from config.toml (#482)

2026-06-02 20:14:42 -07:00

daemon

fix: resolve ruff I001 import sorting and E501 line length in tests

2026-03-17 00:13:49 +00:00

deploy

security(deploy): stop default deployments shipping an open, unauthenticated server (#221)

2026-05-25 23:56:55 +00:00

deployment

fix: copy package includes in Docker builds, incl. GPU (#450)

2026-06-01 09:23:22 -07:00

engine

perf(serve): parallelize engine discovery + async version check (#470)

2026-06-01 13:17:38 -07:00

evals

ci: fix env-dependent test failures (gated datasets, server extra, router prefix) (#348)

2026-05-15 21:31:13 -07:00

fixtures/docs

Feature/twitter bot (#259)

2026-04-17 15:22:17 -07:00

hardware

fix: detect total RAM on Windows via GlobalMemoryStatusEx

2026-05-20 03:36:55 +00:00

install

fix(install): use requires-python range; bootstrap shims are Python-free (#476, #484) (#492)

2026-06-03 16:59:29 -07:00

integration

fix(channels): wire channel→agent handler and fix Telegram send pipeline (#94)

2026-03-20 18:35:18 -07:00

intelligence

refactor: remove dead shims, stale files, and test duplication (#243)

2026-04-13 15:03:03 -07:00

learning

evals: switch the default LLM judge to claude-haiku-4-5

2026-07-11 11:59:32 -07:00

mcp

fix(mcp): send Authorization: Bearer; auto-load MCP tools in ask/serve (#494)

2026-06-03 18:27:00 -07:00

memory

fix(channels): wire channel→agent handler and fix Telegram send pipeline (#94)

2026-03-20 18:35:18 -07:00

mining

feat(mining): validate Gemma E4B and Qwen 3.6 Pearl artifacts (#328)

2026-05-14 20:11:24 -07:00

operators

refactor(system): decompose JarvisSystem god-object into system/ package (#257)

2026-04-17 11:02:56 -07:00

pearl

feat(mining): validate Gemma E4B and Qwen 3.6 Pearl artifacts (#328)

2026-05-14 20:11:24 -07:00

prompt

feat(prompt): per-invocation persona scope (#380) (#493)

2026-06-03 16:59:47 -07:00

recipes

fix(channels): wire channel→agent handler and fix Telegram send pipeline (#94)

2026-03-20 18:35:18 -07:00

sandbox

fix(channels): wire channel→agent handler and fix Telegram send pipeline (#94)

2026-03-20 18:35:18 -07:00

scheduler

feat: Morning Digest with Voice Mode + unified OAuth + Apple connectors (#174)

2026-04-03 11:05:41 -07:00

sdk

chore: derive __version__ from metadata, bump GHA action versions

2026-04-11 13:19:19 -07:00

security

fix(security): detect Rust at import time + SSRF Python fallback (#467)

2026-06-01 11:57:44 -07:00

server

fix(server): load SOUL.md / USER.md context in streaming chat (#449)

2026-06-01 11:31:19 -07:00

sessions

add test case

2026-03-31 19:25:29 +05:30

skills

fix(skills): implement skill remove and search CLI commands (#240)

2026-04-13 15:03:39 -07:00

speech

refactor: remove dead shims, stale files, and test duplication (#243)

2026-04-13 15:03:03 -07:00

telemetry

style: ruff line-length + import fixes for #340 and #202

2026-05-20 20:22:44 +00:00

templates

fix(channels): wire channel→agent handler and fix Telegram send pipeline (#94)

2026-03-20 18:35:18 -07:00

test_orchestrator_learning

orchestrator: track the SFT training pipeline and fix ToolOrchestra tracing

2026-07-11 11:55:59 -07:00

tools

fix(tools): return labeled page content from web_search (#480)

2026-06-02 19:31:05 -07:00

traces

feat(evals): ToolCall-15, LiveCodeBench, LiveResearchBench + telemetry (#169)

2026-04-01 21:06:20 -07:00

workflow

fix(channels): wire channel→agent handler and fix Telegram send pipeline (#94)

2026-03-20 18:35:18 -07:00

__init__.py

init commit

2026-03-12 17:29:39 +00:00

conftest.py

feat(mining): Pearl mining integration

2026-05-05 19:11:15 -07:00

test_digest_integration.py

feat: Morning Digest with Voice Mode + unified OAuth + Apple connectors (#174)

2026-04-03 11:05:41 -07:00

test_query_orchestrator.py

refactor(system): decompose JarvisSystem god-object into system/ package (#257)

2026-04-17 11:02:56 -07:00