`jarvis eval run --base-url ... --api-key ...` was silently dropped for
jarvis-direct/jarvis-agent (_build_backend only forwarded the flags to
hermes/openclaw) and ignored by terminalbench-native, which hardcoded
api_base="http://localhost:8000/v1". Worse, with --base-url set the
engine-discovery fallback silently substituted ANY healthy local engine
(observed: requested vllm + healthy endpoint at --base-url, got
OllamaEngine@localhost:11434 — the requested URL was never contacted).
Changes:
- _OpenAICompatibleEngine gains an api_key param (Bearer Authorization
header on the httpx client; {ENGINE_ID}_API_KEY env fallback with
hyphen-sanitized names; no header when unset).
- New non-registered OpenAICompatEngine + normalize_openai_base_url()
(strips a single literal trailing "/v1" so request paths don't double).
- SystemBuilder.engine_instance() injects a pre-built engine; build()
health-checks it and fails loudly naming the host instead of falling
back to discovery. Discovery substitution after an explicit -e key now
logs a warning.
- JarvisDirectBackend/JarvisAgentBackend accept base_url/api_key; on
base_url they pin an OpenAICompatEngine to that endpoint with a
fail-fast pre-flight (actionable error naming the URL and probe).
- _build_backend forwards base_url/api_key to first-party backends on
the CLI path; _run_terminalbench_native receives --base-url as
api_base (single /v1 suffix) and exports OPENAI_API_KEY around the
in-process harness run (terminus-2 routes via LiteLLM).
- Suite TOML [backend.external] stays scoped to hermes/openclaw
(suite_mode=True in the suite drivers) — first-party suite semantics
are explicitly deferred. The config-host path is untouched.
- Help text updated on both CLI surfaces; KNOWN_BACKENDS now lists
hermes/openclaw/terminalbench-native.
Fixes the eval-CLI endpoint gap reported by the downstream team.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
