Two test-isolation fixes: (1) an autouse conftest fixture sets OPENJARVIS_NO_UPDATE_CHECK=1 so the CLI's PyPI update-check banner (stderr, merged into CliRunner output) can never pollute JSON/CSV-parsing CLI tests on local runs; CI was already covered by CI=true. (2) test_dense.py's Ollama skip-guard now queries /api/tags and requires nomic-embed-text to be pulled instead of a bare TCP connect, so machines running Ollama without the embed model skip instead of erroring. The probe normalizes all documented OLLAMA_HOST forms (full URL, host:port, bare host) and the Ollama-backed tests construct DenseMemory against that same endpoint rather than the embedder's hard-coded localhost default, with unit tests covering the probe. Related: #645.
get_rust_module() was called outside the try block in GitStatusTool/GitDiffTool/GitLogTool.execute(), so on installs without the compiled openjarvis-rust extension (e.g. plain pip installs, where openjarvis-rust is a uv-only group since #624) the ImportError escaped uncaught instead of degrading. Move the call inside try and fall back to the git CLI via the existing _run_git helper on ImportError, matching the fallback git_log already had. Adds regression tests covering the fallback path for all three tools.
Co-authored-by: Elliot Slusky <elliot@slusky.com>
Wires the previously-dead skills/security.py checks into two places. SkillImporter.import_skill() classifies trust tier before writing to disk, refuses unreviewed skills requesting dangerous capabilities unless confirmed (confirm_dangerous=True, or --yes-dangerous on skill install/sync), and persists tier/capabilities to the .source sidecar. SkillExecutor.run() gains opt-in capability enforcement: allowed_capabilities=None (the default, used by all existing call sites) means no policy; passing a set blocks skills whose required_capabilities are not covered before any step runs. Bulk sync reports refused skills instead of silently skipping them. Both enforcement points are covered by tests.
Two fixes: (1) /v1/chat/completions now builds its injected system prompt via SystemPromptBuilder, so SOUL.md/MEMORY.md/USER.md persona files apply to the OpenAI-compatible endpoint exactly as they do on the managed-agent path; injection still only happens when the client omits a system message. (2) FTS5 query tokens are now split on any non-alphanumeric character, so apostrophes (user's) and quotes can no longer reach the MATCH string unescaped; all-punctuation queries return empty instead of erroring.
faster-whisper's transcribe() was handed the path of a still-open NamedTemporaryFile; on Windows the open handle is exclusive, so PyAV's reopen failed with EACCES and local STT was broken. Switch to delete=False, close before transcribing, and unlink in a finally. The write is wrapped in the file's context manager so the handle closes even if write() raises, and unlink failures are logged at debug.
* fix(knowledge_sql): match write keywords on word boundaries
The read-only guard rejected a query if any of DROP/DELETE/INSERT/UPDATE/
ALTER/CREATE/ATTACH appeared as a bare substring of the uppercased text. That
wrongly blocks valid SELECTs whose column/alias/literal merely contains one --
e.g. "deleted_at" (DELETE), "created_at" (CREATE), "updated_content" (UPDATE).
The knowledge_chunks table actually has deleted_at/created_at columns and the
store's own retrieval filters on "WHERE deleted_at IS NULL", so realistic
read queries were refused. Match on word boundaries with a compiled regex,
mirroring the sibling tool db_query.py. Add a regression test.
* fix(knowledge_sql): ignore string literals in keyword scan, broaden error handling
- Strip single-quoted literals before the forbidden-keyword scan so
SELECTs whose data merely mentions a write keyword (e.g. LIKE
'%delete%') are not rejected.
- Catch sqlite3.Error instead of only OperationalError so multi-
statement strings return a failed ToolResult instead of raising.
- Document created_at/deleted_at in the tool's schema description.
---------
Co-authored-by: Elliot Slusky <elliot@slusky.com>
Refresh the cost-comparison / savings surfaces to current frontier cloud
pricing (per 1M tokens):
- OpenAI: GPT-5.3 ($2/$10) -> GPT-5.6 Sol ($5/$30)
- Anthropic: Claude Opus 4.6 ($5/$25) -> Claude Fable 5 ($10/$50)
- Google: Gemini 3.1 Pro ($2/$12) -> unchanged
Internal provider keys are renamed in lockstep (gpt-5.3 -> gpt-5.6-sol,
claude-opus-4.6 -> claude-fable-5) across the canonical CLOUD_PRICING
dict, the two server-rendered HTML pages, and the frontend color/label
maps so backend, dashboard, and UI stay consistent. Energy/FLOPs
metadata is carried over unchanged. Model catalog and eval configs are
untouched (real model/benchmark entries, not the cost comparison).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The OpenAI-compat and Ollama engines exposed stream()/stream_full() as async def but iterated a synchronous httpx.Client.iter_lines() internally, blocking the single event loop on every inter-token read (serializing concurrent chats; one wedged upstream read froze the whole API). Convert both to a shared AsyncHTTPEngineMixin using httpx.AsyncClient + aiter_lines() with a per-event-loop pooled client and the configured timeout applied; map mid-stream transport errors (RemoteProtocolError/ReadError) to EngineConnectionError via a deliberately narrow set that keeps CancelledError/GeneratorExit propagating; handle non-2xx explicitly (incl. 3xx and a typed EngineContextLengthError for context-window overflow 400s); switch litellm streaming to acompletion; and offload the blocking non-streaming handlers and websocket generate() to asyncio.to_thread. No public API change. Strong MockTransport-based tests, including a pin that the async path never touches the sync client. Complements #618.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CI's lint job ran ruff check but never ruff format --check, letting format drift land silently (79 files had drifted from the pinned ruff 0.15.1). Add the ruff format --check step to ci.yml, reformat the 79 drifted files with the pinned ruff (mechanical only — verified AST-identical to before across all files, no logic changes), and add a Makefile whose test target mirrors the actual CI lane.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(packaging): make openjarvis-rust a uv-only dependency group (unblock pip[desktop])
#615 added openjarvis-rust to the published `desktop` extra so
`uv sync --extra desktop` builds the native PyO3 extension for the desktop
app. But openjarvis-rust is not on PyPI and `[tool.uv.sources]` is stripped
from published wheel metadata, so `pip install openjarvis[desktop]` from PyPI
failed at install trying to resolve openjarvis-rust from PyPI (#584).
Move openjarvis-rust into a uv `desktop-native` dependency group (PEP 735 —
excluded from wheel metadata) and sync it in the desktop app via
`uv sync --group desktop-native`. The extension is still built from the local
path source; only the published metadata changes.
Verified: the built wheel no longer lists openjarvis-rust in any Requires-Dist
(nowhere in the metadata), and uv.lock still resolves it from the local path
source under the group. Adds tests/deployment/test_packaging.py to guard the
split (not in the published extra, present in the group, path source, and the
desktop app syncs the group).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(packaging): sync native group in install paths
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Elliot Slusky <elliot@slusky.com>
Closes#219. Replace synchronous httpx calls in async SendBlue and model-management handlers with awaited httpx.AsyncClient (context-managed close); run Whisper transcription and engine.list_models via asyncio.to_thread so they don't block the event loop; and harden TelemetryStore/aggregator SQLite for concurrency (WAL, synchronous=NORMAL, busy_timeout=5000, plus a write-serializing lock on the shared connection). Adds async-usage assertions and a real 8-thread concurrent-write test. Related: #570 (async httpx, different issue #559).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes#388. Parse Google Calendar all-day events from start.date instead of stamping them with the current time; treat generic next/upcoming calendar-event queries as gcalendar timeline requests that return nearest-future events first (UTC-normalized, instant-aware comparison that handles tz offsets and all-day events); and update the research planner guidance to route such queries with sources=[gcalendar] + a today-onward time_range. Real in-memory KnowledgeStore integration tests cover ordering, tz normalization, all-day inclusion, and source narrowing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes#575. Web Deep Research was hardcoded to OllamaEngine + DEFAULT_PLANNER_MODEL, ignoring the user's configured/active engine and model. Resolve the planner from [deep_research] override -> live app chat engine + selected model -> config defaults -> legacy Ollama, pass the chat picker's model from the frontend into /api/research, record the actual planner engine in telemetry, and refuse to silently fall back to a different engine (raise an actionable error instead). Adds config support and focused tests for resolution and the route. Related: #576 (duplicate).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes#608. Make Message.content officially Optional (str | None) with a Message.text accessor that treats None as empty, and count tool-call IDs/names/arguments, tool-result IDs, and reasoning/thinking metadata in estimate_prompt_tokens — all are replayed into later prompt turns, so they belong in the estimate. Extends estimator and message-type regression tests.
Closes#607. Assistant tool-call turns can carry content=None, which crashed token estimation (len(m.content)) and think-tag stripping. Normalize with 'content or ""', route native OpenHands truncation through the shared estimate_prompt_tokens, and add tests for the estimator, the truncation helper, and an end-to-end tool-call run with None content.
The docs-site leaderboard (docs/javascripts/leaderboard.js) reads the public
Supabase anon key from window.OPENJARVIS_SUPABASE_ANON_KEY, but nothing set it,
so the published leaderboard always rendered "Leaderboard not configured yet".
Add a generated config file (leaderboard-config.js) loaded before
leaderboard.js that supplies the global, and inject its value at docs-build
time from the existing VITE_SUPABASE_ANON_KEY repo secret. The committed
default is empty, so local `mkdocs build` and fork PRs (no secret) degrade
gracefully. The anon key is public by design (Supabase RLS protects the data).
- docs/javascripts/leaderboard-config.js: empty-default global declaration.
- mkdocs.yml: load leaderboard-config.js before leaderboard.js.
- docs.yml: write the config from the secret (read via env, JSON-encoded into a
JS string literal to avoid injection) before `mkdocs build`.
- tests/deployment/test_docs_leaderboard.py: guard the wiring + load order.
Verified with a local `mkdocs build`: the generated config ships in site/ and
loads before leaderboard.js.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes#582. Route fact-store construction through a new FactStoreRegistry (local backend registered by default); align the default facts path with get_config_dir(); wire completed chat exchanges (streamed and non-streamed) through the EventBus so the memory service captures them consistently; reload the local fact store from disk before operations so external clears don't resurrect stale facts; make the affected config/persona/memory/CLI/route tests hermetic; and refresh uv.lock with the current resolver (locks pytest-xdist + transitive deps, drops py3.14 artifacts since the project constrains Python <3.14).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Build and install the mandatory openjarvis_rust wheel in the CPU, NVIDIA, ROCm, and sandbox Docker images. Rust 1.88 (matching the workspace MSRV / rust-toolchain.toml) and maturin are installed only in the builder stage, the module's import is verified during the build, and maturin is removed before the runtime artifacts are copied so build tooling never ships. The frontend leaderboard anon key is an optional empty-by-default build arg (post-#589), so default images cleanly disable the leaderboard. Adds static deployment coverage for the native build path. Closes#584.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TestTraceRecording relied on the ambient ~/.openjarvis/config.toml leaving traces.enabled at its default, so it failed on any machine with traces disabled locally (passing in CI only because the runner has no config file). Pass an explicit traces-enabled config with a tmp db_path so the tests are environment-independent and parallel-safe under pytest -n auto. Relates to #582.
Pin all base images and ollama to fixed versions + @sha256 digests (no floating :latest), run Docker images as an unprivileged openjarvis user (uid 10001), replace the curl|bash NodeSource install with a digest-pinned multi-stage copy, install from the committed uv.lock via uv export --frozen --no-dev (hash-verified, --no-deps), and add systemd sandboxing (NoNewPrivileges, ProtectSystem=strict, PrivateTmp, kernel/SUID protections). Closes#228, #563, #564, #565, #566, #567.
Adds the openjarvis.memory package (LocalFactStore, FactExtractor, background MemoryService), starts/stops it in the jarvis serve and jarvis chat lifecycle, feeds completed non-streaming exchanges to it, adds [memory] config support, and adds jarvis memory list/clear CLI commands. Extraction runs on a background thread and degrades to a no-op on any failure (BrokenPipe, timeouts, unparseable output) so it can never block a reply or crash the host. Disabled by default. Closes#393, #571, #572, #573.
Qwen3 treats /think and /no_think as soft-switch control tokens. On small
models a multi-line prompt makes the model emit one as the sole tool argument
(e.g. {"command": "/no_think"}); OpenJarvis forwards Ollama's native tool_calls
verbatim, so the operative agent executes garbage. Filter control-token-only
tool calls in both the non-streaming generate() and streaming _run_stream()
paths, keeping legitimate calls like {"command": "date"}.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Previously `core/config.py` defined `DEFAULT_CONFIG_DIR = Path.home() /
".openjarvis"` as a by-value module constant imported into ~45 modules, and 34
modules hardcoded `Path.home() / ".openjarvis"` directly. The installer honored
`OPENJARVIS_HOME` but the Python runtime ignored it, producing a split-brain
layout (some modules honored the override, the core config dir did not). Eval
dataset caches also scattered into `~/.cache/<benchmark>`.
This introduces a single env-aware resolver in `openjarvis/core/paths.py` and
routes every state/config/cache path through it. OpenJarvis now keeps ALL of
its state under ONE root, resolved in priority order:
1. $OPENJARVIS_HOME
2. $XDG_DATA_HOME/openjarvis (single nested dir, when XDG_DATA_HOME is set)
3. ~/.openjarvis (default — unchanged, so existing installs are
untouched and no data migration is required)
Implementation:
- New `core/paths.py`: get_config_dir / get_config_path / get_data_dir /
get_cache_dir, with a source-tree rejection guard (fails loudly per
REVIEW.md if the root resolves inside the repo).
- `core/config.py`: DEFAULT_CONFIG_DIR / DEFAULT_CONFIG_PATH are now resolved
via the env-aware resolver at import (real attributes, so existing
monkeypatch.setattr-based tests keep working). All dataclass field defaults
that pointed at ~/.openjarvis converted to default_factory so they honor the
override at instantiation.
- Routed all 34 hardcoders plus several string-literal escapees the original
audit missed: prompt_loader / description_loader (were OPENJARVIS_HOME-only,
no XDG), swebench_harness cache, tools/{memory,skill,user_profile}_manage
defaults, server trace.db fallbacks, doctor_cmd hints.
- spec_search storage/paths now delegates to the unified resolver (gains XDG);
its ConfigurationError is aliased to the core one.
- Eval dataset caches moved from ~/.cache/<name> to <root>/cache/<name>
(~/.cache/huggingface left alone — it is HF's own cache).
- Docs + installer comment + `jarvis config path` to show resolved dirs.
Read-only macOS connectors and OS service files (LaunchAgents/systemd) are
intentionally left untouched.
Fixes#462
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pasting a Google Client ID / Secret never completed OAuth: Drive (and its
Google siblings) accepted the credentials, showed no error, opened no browser,
and never appeared in Data Sources. Root cause is three coupled defects, all
reproduced at the unit level against main with a FastAPI TestClient (no Google
creds, network-free):
(A/B) POST /connect routed a `client_id:client_secret` pair into the
connector's handle_callback, which spawned a daemon thread that popped a
browser and ran its own localhost:8789 callback server. That thread fails
silently in the bundled desktop context (`except Exception: pass`), so the
connector never gained an access_token; /connect returned status "pending"
and the UI's 20x2s poll timed out with no error.
Fix: in POST /connect, an OAuth `client_id:client_secret` pair now persists
the client credentials to every Google credential file and returns an
`oauth_required` directive pointing at the in-process server flow, instead of
the silent background thread. The Google connectors' handle_callback no longer
spawns the browser thread for the pair case — it only persists the creds; the
server's /oauth/start -> /oauth/callback owns the consent round-trip.
(C) The would-be-correct server flow was itself broken: under
`from __future__ import annotations` plus a `Request` import local to the
router factory, FastAPI could not resolve the stringized `request: Request`
annotation. /oauth/start returned HTTP 422 (request mis-bound as a query
param) and /oauth/callback injected None -> AttributeError on
`request.base_url`. Fix: import `Request` at module scope and make the
callback's `request` a required injected dependency.
A malformed/blank client pair now raises HTTP 400 with the provider setup URL
instead of a perpetual silent "pending" (REVIEW.md silent-failure discipline).
Frontend: DataSourcesPage now opens the server OAuth window when /connect
returns `oauth_required`, then polls until connected; connect errors surface the
backend detail; the Drive setup steps document the "Web application" OAuth
client + server-callback redirect URI the in-process flow requires.
Tests (run on the main venv, hermetic — no ~/.openjarvis pollution):
- test_oauth_flow.py: the three handle_callback tests now assert NO browser is
opened and only client creds are persisted (was: assert background flow ran).
- test_connectors_router_oauth.py (new): reproduces + fixes all three defects via
TestClient with mocked token exchange; parametrized over gdrive/gcalendar/
gcontacts/gmail/google_tasks to prove the shared OAuth path is fixed for every
sibling and that a single consent writes the access_token to all six Google
credential files and flips is_connected() to True.
Full tests/connectors suite: 355 passed.
Relationship to PR #510: #510 rewrites all of these files (account-scoped
retrieval) but still carries all three defects. This fix is intentionally scoped
to the OAuth path and does not modify oauth.py, to minimize collision. A
maintainer can either merge this and rebase #510 on top, or port these changes
into #510. See PR body for details.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The OpenAI-compatible POST /v1/chat/completions endpoint — the desktop UI's
chat backend — never injected OpenJarvis's agent.default_system_prompt when the
client omits a system message. The frontend (Chat/InputArea.tsx) posts only
user/assistant turns, so the model answered from its training identity
("I'm Claude", "I am Qwen", ...). The CLI paths ground identity via
SystemPromptBuilder / BaseAgent; the engine-direct server handlers did not.
Fix:
- Add _ensure_identity_prompt(messages, app_config) in server/routes.py: returns
messages unchanged when any has role==SYSTEM, else prepends a SYSTEM message
with the resolved identity prompt (app.state.config.agent.default_system_prompt,
else load_config()), wrapped in try/except that debug-logs on failure (no crash,
no silent swallow per REVIEW.md).
- Apply it after _to_messages() in all three engine-direct handlers:
_handle_stream, _handle_stream_tools, and _handle_direct; thread app.state.config
through. _handle_agent is left untouched (BaseAgent already injects the default).
- Harden AgentConfig.default_system_prompt so distilled models stop claiming to be
Claude/ChatGPT/Gemini and self-identify as OpenJarvis.
Tests (tests/server/test_routes.py, tests/core/test_config.py): identity prompt IS
prepended when no system message is present (stream / direct / tools paths) and is
NOT duplicated when the client supplies one; config wording anchors "OpenJarvis"
and "not Claude". Verified fail-on-unfixed against main (3 inject tests + config
wording test fail there).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(engine): add DeepSeek as a first-class cloud provider
Adds DEEPSEEK_API_KEY support to the cloud engine, wiring DeepSeek's
OpenAI-compatible API (api.deepseek.com/v1) alongside the existing
MiniMax, OpenRouter, Anthropic, and Google providers.
- Add _DEEPSEEK_MODELS list (deepseek-v4-flash, deepseek-v4-pro)
- Add _is_deepseek_model() routing predicate
- Init self._deepseek_client from DEEPSEEK_API_KEY in _init_clients()
- Add _generate_deepseek() and _stream_deepseek() methods
- Wire DeepSeek into generate(), stream(), _stream_full_openai(),
list_models(), and health()
- Add approximate pricing entries for both models
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(engine): strict cloud model routing + deepseek can_serve branch
Builds on the DeepSeek provider (PR #504) with two routing-correctness
fixes to CloudEngine._client_for_model:
1. Add the missing DeepSeek branch so can_serve('deepseek-*') agrees with
list_models()/health() when only DEEPSEEK_API_KEY is set (mirrors the
minimax branch). Without it the engine advertised deepseek models via
list_models() but refused to serve them (the #532 can_serve contract).
2. Fix#335: _client_for_model previously fell through to the OpenAI client
for ANY unrecognized model name, so an OpenAI key (even a dummy
sk-dummy... one) made can_serve('qwen3.5:0.8b') return True. With the
local engine transiently down (classic post-Windows-restart Ollama not
yet up), model-aware get_engine then mis-selected the cloud engine for a
local model and died with "OpenAI client not available". Add a positive
_is_openai_model predicate (gpt-/chatgpt-/o1/o3/o4 + _OPENAI_MODELS) and
return None for unrecognized names, so can_serve declines them. generate()
and stream() keep their OpenAI fall-through, preserving loud failure for an
explicitly-requested unknown cloud model.
Tests: DeepSeek detection/pricing/health/list_models/generate-routing/
can_serve and a #335 regression (can_serve rejects local names with an
OpenAI key; unknown model not served even with all clients set; end-to-end
get_engine does not misroute a local model with a dummy OpenAI key).
Fixes#335
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Jen Huls <me@jenhuls.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
OpenJarvis can run vision-capable local models (gemma3, qwen2.5-vl), but the
CLI had no way to send them a picture -- the Ollama engine only serialized
text. This adds end-to-end image input.
What's new
- `jarvis ask -i/--image <file>` attaches one or more images to the query.
- `jarvis ask -S/--screen` captures the primary monitor (dependency-free on
Windows via .NET; mss/Pillow fallback elsewhere).
- Vision auto-routes to direct-to-engine mode; with an explicit --agent it
warns rather than silently dropping the image.
- Privacy guard: warns before sending an image to a non-local engine,
keeping OpenJarvis local-first by default.
- Context-window default raised 8k -> 16k (JARVIS_NUM_CTX) so an image plus
a conversation fit.
Implementation
- Message.images carries base64 data; messages_to_dicts() forwards it to
Ollama's /api/chat "images" field. Text-only messages are unchanged.
- GuardrailsEngine preserves images when it rewrites a flagged message.
Tests (tests/test_vision.py, 6/6 pass, ruff-clean)
- payload forwarding, text path untouched, num_ctx override, guardrail
image preservation.
Verified on AMD RX 9070 XT (Ollama/Vulkan, 100% GPU) with gemma3:4b:
solid-color image, file image, and live screen capture all described.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
Two failure classes hit by a downstream team:
1) tmux/task-env death (TerminalBenchTaskEnv.__enter__, terminalbench_env.py):
create_session ran inside the spin_up_terminal generator-CM with no
exception safety — a tmux failure leaked the docker compose project
(down deferred to GC, never if the env was retained) and surfaced as an
opaque mid-run death. Now: exception-safe __enter__ with an idempotent
_teardown(), a tmux/asciinema preflight in the container BEFORE the
agent loop (TaskEnvironmentError naming the task image + remedy), and
fail-that-task-cleanly semantics — the failure is recorded as a harness
error (QueryTrace.error_kind="harness_error"), the container is downed,
and the run continues.
2) OpenHands in-container SETUP hang misattributed as a model result:
harness.run() had no bound (terminal-bench runs installed-agent setup
with max_timeout_sec=inf inside the per-trial agent budget), and the
summary conversion read the nonexistent results.trial_results attr and
hardcoded errors=0, folding zero-model-request trials into resolve-rate
as model misses. Now: global_agent_timeout_sec / global_timeout_multiplier
are threaded config -> backend -> Harness kwargs (default 1800 s bound on
SETUP+RUN; configurable per [run]/[[benchmarks]] TOML), and
summarize_benchmark_results() classifies harness/infra failures out of
the accuracy denominator keyed on token usage (zero/missing tokens +
unresolved = the agent never contacted the model), NOT failure_mode —
terminal-bench 0.2.18 leaves failure_mode UNSET on success AND on
genuine misses, so a failure_mode-based check would misflag every real
model miss. Genuine misses (tokens>0, is_resolved=false) stay in the
denominator.
QueryTrace gains error/error_kind (wired through to_dict/from_dict so the
fields actually reach traces.jsonl; backward-compatible loads), the
AgenticRunner flags zero-model-contact traces and TaskEnvironmentError as
harness errors, and export/summary/console output exclude harness errors
from resolve-rate while reporting them loudly.
terminal-bench stays an undeclared dep on purpose: it requires Python
>=3.12 while this project supports >=3.10,<3.14, so an unmarked extra
would break uv lock. pyproject/uv.lock untouched.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`jarvis eval run --base-url ... --api-key ...` was silently dropped for
jarvis-direct/jarvis-agent (_build_backend only forwarded the flags to
hermes/openclaw) and ignored by terminalbench-native, which hardcoded
api_base="http://localhost:8000/v1". Worse, with --base-url set the
engine-discovery fallback silently substituted ANY healthy local engine
(observed: requested vllm + healthy endpoint at --base-url, got
OllamaEngine@localhost:11434 — the requested URL was never contacted).
Changes:
- _OpenAICompatibleEngine gains an api_key param (Bearer Authorization
header on the httpx client; {ENGINE_ID}_API_KEY env fallback with
hyphen-sanitized names; no header when unset).
- New non-registered OpenAICompatEngine + normalize_openai_base_url()
(strips a single literal trailing "/v1" so request paths don't double).
- SystemBuilder.engine_instance() injects a pre-built engine; build()
health-checks it and fails loudly naming the host instead of falling
back to discovery. Discovery substitution after an explicit -e key now
logs a warning.
- JarvisDirectBackend/JarvisAgentBackend accept base_url/api_key; on
base_url they pin an OpenAICompatEngine to that endpoint with a
fail-fast pre-flight (actionable error naming the URL and probe).
- _build_backend forwards base_url/api_key to first-party backends on
the CLI path; _run_terminalbench_native receives --base-url as
api_base (single /v1 suffix) and exports OPENAI_API_KEY around the
in-process harness run (terminus-2 routes via LiteLLM).
- Suite TOML [backend.external] stays scoped to hermes/openclaw
(suite_mode=True in the suite drivers) — first-party suite semantics
are explicitly deferred. The config-host path is untouched.
- Help text updated on both CLI surfaces; KNOWN_BACKENDS now lists
hermes/openclaw/terminalbench-native.
Fixes the eval-CLI endpoint gap reported by the downstream team.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(windows): desktop backend spawn (#531) + model-aware engine selection (#532)
Two runtime bugs found during end-to-end testing on a clean Windows 11
24H2 Azure VM.
#531 - Desktop "Failed to get response": run_jarvis_command spawned the
backend with .output(), which waits for the process to exit. `jarvis
serve` never exits, so the Tauri command hung forever (the Start button
never resolved); and it ran `uv run jarvis` with no cwd, so in a packaged
install -- where the cwd isn't the checkout -- `jarvis` wasn't found and
the server never started. Now: run from find_project_root(), and for
`serve` spawn detached (.spawn()), drain stderr, and poll /health for
readiness (mirrors start_backend); short commands keep .output().
The server layer itself was verified healthy on Windows (/health and
/v1/chat/completions both 200, localhost included) -- the fault was the
Tauri spawn path.
#532 - "OpenAI client not available" after reboot: when the local engine
is down, get_engine's fallback selected CloudEngine because health() is
True if ANY provider client exists -- without checking the resolved
model's provider has a client. A user with e.g. OPENROUTER_API_KEY and a
gpt-* model then hit the OpenAI path with no client. Add
CloudEngine.can_serve(model) (checks the specific provider client via the
same routing generate()/stream() use) + a default can_serve->True on the
base engine, and make get_engine model-aware so it skips an engine that
can't serve the model -- the user falls through to the helpful "no engine
available / start ollama" message instead.
Tests: engine discovery/cloud/model-matrix + cli serve/ask suites pass
(the one ask_e2e failure is a pre-existing version-banner flake, fails
identically on main). The Tauri crate couldn't be compiled locally (no
GTK/webkit sys-libs in this env); relies on CI.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(engine): cover model-aware engine selection + CloudEngine.can_serve (#532)
#533 added a `model` arg to get_engine and a can_serve() gate but shipped no
tests. Add them:
- get_engine skips a healthy engine that can't serve the requested model
(the cloud-fallback-for-unservable-model case behind #532),
- model=None preserves the legacy model-agnostic selection,
- CloudEngine.can_serve gates on the per-provider client (gpt->OpenAI,
claude->Anthropic, ...), verified empirically.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
* security: harden network-exposed surface
Hardening for the network-reachable attack surface, prioritizing fixes
that are strong but do not change working local/loopback defaults.
- auth_middleware: constant-time API key comparison (secrets.compare_digest)
for the HTTP path, and gate /metrics behind auth so operational counters
are not readable unauthenticated. /health stays open.
- webhook_routes: fail closed when a channel's secret/token is unset. Twilio,
BlueBubbles, WhatsApp (verify + inbound), and SendBlue now reject (403)
instead of processing unsigned/unauthenticated input. Constant-time
comparisons for BlueBubbles/SendBlue/WhatsApp verify token.
- http_request: follow redirects manually and re-run the SSRF check on every
hop (capped at 5) so an allowed public URL cannot 30x-redirect to an
internal/metadata address.
- api_routes /v1/memory/index: restrict indexing to OPENJARVIS_WORKSPACE roots
when configured and refuse sensitive files (.env, keys, credentials).
- config.toml: default [server] host to 127.0.0.1 (loopback) with a comment
on how to safely expose to a LAN (0.0.0.0 + API key).
Tests: new fail-closed webhook tests, /metrics auth tests, and SSRF
redirect block/follow tests; updated SendBlue tests for the new
secret-required behavior. Affected suites pass (95 tests), ruff clean.
* fix(http): keep SSRF redirect-following patchable via httpx.request
The manual redirect-following loop used a private httpx.Client, which
bypassed the `http_request.httpx.request` mock seam that consumers' tests
rely on (e.g. the twitter-bot GitHub-issue tests escaped to the real
network and 401'd). Issue each hop via module-level httpx.request with
follow_redirects=False instead — same per-hop SSRF re-check, restored
testability.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(chat): wire SystemPromptBuilder so persona files load in jarvis chat (fixes#458)
* fix(chat): make `--persona none` actually disable persona files
This PR exposes `--persona none`, but SystemPromptBuilder._load_file read
empty paths as "." (Path("") -> ".") and raised IsADirectoryError, so the
documented opt-out crashed. Guard empty path_str so the "none" opt-out
(which _resolve_persona maps to empty file paths) cleanly injects no
persona. Adds an end-to-end regression test (building with persona
"none" must not raise). Also merges current main (branch was stale).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`jarvis serve` constructed every heavy component inline (engine discovery +
instrumentation, telemetry, memory, agent manager, per-agent tools) and then
called `SystemBuilder(config).build()` a second time inside the scheduler
block purely to feed `AgentExecutor.set_system()`. That second build
re-discovered and re-connected the engine, re-instrumented it, re-resolved
tools, re-opened the configured channel and re-created the agent manager —
~30-40s of fully redundant startup work (the headline remaining cost in #263
after engine probes were parallelised and the version check moved off the hot
path in #470).
Fix: assemble the executor's `JarvisSystem` from the components already built
inline instead of rebuilding from scratch. `AgentExecutor` only reads
`engine`, `model`, `config`, `memory_backend`, `tool_executor`,
`session_store` and `channel_backend` off the system; all are wired here. The
memory backend is now constructed just before the scheduler block (it was
built later) so the executor's system can reference it, and the primary
agent's resolved tool list is reused to build the scheduler's `ToolExecutor`
(preserving the MCP-discovered-tool pool the executor reads via
`tool_executor._tools`). `skill_manager` / the learning orchestrator are only
consumed by the orchestrator's `system.ask()` path, which the executor never
invokes, so they are intentionally omitted.
Tests: new `tests/cli/test_serve_single_build.py` patches
`SystemBuilder.build` and asserts it is never called during `jarvis serve`
startup, and that the executor still receives a system exposing
`tool_executor` / `session_store` / `memory_backend` (plus engine/model/config).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The channel `send()` contract was inconsistent across adapters. Almost
every adapter (Discord, Slack, email, WhatsApp, ...) treats the first
positional `channel` arg as the real DESTINATION id and `conversation_id`
as an optional reply/thread reference. Telegram alone treated
`conversation_id` as the destination (`chat_id = conversation_id or
channel`).
`JarvisSystem._on_channel_message` hard-coded the Telegram-shaped mapping
for ALL channels: `send(cm.channel, reply, conversation_id=cm.conversation_id)`.
Since inbound `ChannelMessage`s carry the channel TYPE label in `.channel`
("discord") and the real destination id in `.conversation_id`, this sent
the literal "discord" as the Discord channel id (HTTP 400
NUMBER_TYPE_COERCE, #515) and passed the channel id as a Discord
`message_reference` (MESSAGE_REFERENCE_UNKNOWN_MESSAGE, #516).
Fix: define and document ONE canonical contract on `BaseChannel.send` —
positional `channel` = destination id, `conversation_id` = inbound message
id used as a reply reference — and dispatch it from `_on_channel_message`
as `send(cm.conversation_id, reply, conversation_id=cm.message_id)`,
matching the already-fixed `ChannelAgent` path (#495/#459). Telegram's
`send()` is brought into line (destination = `channel`, with a
`reply_to_message_id` reply ref and a legacy `conversation_id`-only
fallback) so it keeps working unchanged.
The DiscordChannel `_gateway_loop` ChannelMessage shape locked by #495 is
untouched; `tests/agents/test_channel_agent.py` passes unchanged. The
stale `test_serve_channel_wiring.py` assertions (which encoded the old
buggy mapping from #94) are updated, and regression tests are added for
Discord (real channel id + correct message_reference), Telegram (chat id
+ reply ref + legacy fallback), and per-channel dispatch in
`_on_channel_message`.
Fixes#515Fixes#516
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#463 made the OpenAI-compatible engine wrap upstream HTTP errors (incl.
404) in EngineConnectionError with an actionable message, but
test_invalid_model_404 still asserted the raw httpx.HTTPStatusError, so it
broke on main once #463 landed (#463 was a stale fork PR with no CI, so it
wasn't caught pre-merge). Expect EngineConnectionError now, asserting the
httpx.HTTPStatusError is preserved as the chained cause. Whole tests/engine
suite is green again.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Memory tools degraded silently and misleadingly when the mandatory
`openjarvis_rust` extension was absent from the *serving* venv:
- `POST /v1/memory/store` returned HTTP 200 `{"status":"stored","note":
"no backend available"}` and stored nothing (silent data loss).
- `POST /v1/memory/index` returned a generic "No memory backend available",
and the desktop frontend discarded the server `detail` and threw a blanket
"Failed to index path", blaming the path instead of the real cause.
- `GET /v1/memory/config` reported `backend_type: sqlite` even though no
backend could be constructed.
Root cause: `SQLiteMemory.__init__` calls `get_rust_module()` (which raises
ImportError by design — the Rust ext is mandatory, no Python fallback), and
`_get_memory_backend` swallowed that ImportError and returned `None`,
conflating "native extension missing" (a hard install error) with "memory
intentionally disabled" (benign). A chunking floor also silently dropped whole
short documents, and the installer never verified the extension imported from
the serving venv before writing its success marker.
Fix (no fake Python fallback — the Rust ext stays mandatory by design):
- Add `MemoryBackendUnavailable` + `RUST_MISSING_HINT` in tools/storage/_stubs.
`SQLiteMemory.__init__` translates the bridge ImportError into this clear,
actionable error ("run `uv run maturin develop ...`").
- `_get_memory_backend` distinguishes the two cases: a missing native ext
raises HTTP 503 with the actionable hint; a benign unconfigured backend
still returns `None` (graceful path preserved for search/stats).
- `/store` now returns 503 instead of a 200 silent no-op.
- `/config` reports `available: false` + `detail` instead of falsely claiming
a healthy `backend_type`.
- `/index` adds a `note` when `chunks_indexed == 0` so "indexed" never
silently means "stored nothing".
- chunk_text no longer drops an entire short document below `min_chunk_size`
(the floor only discards tiny *trailing* fragments now).
- Frontend `storeMemory`/`indexMemoryPath` surface the server `detail` instead
of blanket strings; `MemoryConfig` gains optional `available`/`detail`.
(Left the pre-existing `backend` vs `backend_type` mismatch untouched.)
- build-extension.sh verifies `import openjarvis_rust` succeeds in the serving
venv before writing the `extension-built` marker.
Regression tests: tests/server/test_api_routes.py::TestMemoryRustMissing mocks
`get_rust_module` to raise ImportError and asserts /store (503, not 200 no-op),
/index (actionable detail, not "Failed to index path"), and /config
(available:false) all surface the clear error; tests/memory/test_chunking.py
asserts short-only docs are kept while tiny trailing fragments are still
filtered.
Fixes#502
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: forward tools through OpenRouter engine
The OpenRouter chat completion path built the request from only
`model`, `messages`, `max_tokens`, and `temperature`. `tools` and
`tool_choice` passed via `kwargs` were silently dropped, so
function definitions never reached the model. Symptom on a
managed deep_research agent: the model answered every query from
its own prior knowledge and never invoked `knowledge_search`,
`knowledge_sql`, etc.
The same path also discarded `choice.message.tool_calls` from the
response — when a model did return a tool call (verified directly
against OpenRouter with `google/gemma-4-31b-it:free` and
`nvidia/nemotron-3-ultra-550b-a55b:free`), the agent loop never
saw it.
This patch:
- forwards `tools` and `tool_choice` into the OpenAI-compatible
request in both `_generate_openrouter` (sync) and
`_stream_openrouter` (async stream),
- extracts `tool_calls` from the response in `_generate_openrouter`
in the same shape used by the OpenAI / Anthropic paths.
Verified end-to-end against a `deep_research` managed agent using
an OpenRouter preset with Gemma 4 31B + Nemotron 3 Ultra fallback:
before, the agent stated "I don't have access to your vault";
after, it calls `knowledge_search`, cites results, and produces a
structured answer.
* test(engine): regression test for OpenRouter tool forwarding
Asserts the OpenRouter path forwards tools/tool_choice to the
OpenAI-compatible API and parses tool_calls back into the result (#511).
Verified: passes on the fix, fails (KeyError 'tools') against pre-fix main.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(chat): honor model config in managed agent chat (#477)
* test(server): regression test for managed-agent engine resolution
_make_lightweight_system must resolve the user's configured engine
(intelligence.preferred_engine, else engine.default) via get_engine,
not a hardcoded OllamaEngine (#477/#514). Asserts the key passed to
get_engine (captured before the system is built); runs under the server
extra (fastapi). Verified: passes on the fix, fails (KeyError) against the
pre-fix hardcoded-Ollama code.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(learning): exclude padding tokens from SFT loss
* test(learning): add regression tests for SFT padding-loss masking
Cover the fix in both trainers (#521):
- OrchestratorSFTDataset.__getitem__: labels are -100 at padded positions,
equal to input_ids elsewhere, and input_ids is not mutated.
- LoRATrainer._train_step: the labels passed to the model are masked at
padded positions (captured via an injected model), with input_ids intact.
Both are torch-gated (pytest.importorskip / skipif HAS_TORCH), matching the
project's existing torch test gating, so they skip cleanly in the default CI
env. Verified locally with CPU torch: both PASS against the fix and FAIL
against the pre-fix code.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The two eval-dataset suites that download real corpora from the
HuggingFace Hub at runtime were running in the default CI lane. When the
Hub was unreachable or rate-limited they failed and reddened `main` even
though no code changed — confirmed by #506 (docs-only) failing on merge
while its own PR run passed an hour earlier. They also dominated CI
wall-time (~33 min of downloads + retry backoff on the failing run).
Add a `hub` pytest marker, apply it to both suites via module-level
`pytestmark`, and exclude it from the default CI lane
(`-m "not live and not cloud and not hub"`). The tests stay runnable on
demand with `pytest -m hub`.
The ADP provider swallows per-config download errors and returns 0
records on a network failure, so a Hub outage surfaced there as
`assert 1 <= 0` (not an exception) — it could not be made non-flaky by
exception handling alone, only by gating.
Coverage holds: removing these from CI drops total from 60.92% to
~60.73% (paranoid worst case 60.18%), still above the 60% gate.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(leaderboard): correct telemetry pipeline and outlier handling
The public leaderboard at /leaderboard showed a clear bimodal Wh/token
distribution: most users at ~3-5 J/token, ~30% inflated by 1000-4000×.
A 5-agent investigation workflow + 4-agent verification pass traced the
inflation to a cluster of related bugs across telemetry, server, and
display layers. This PR fixes the in-repo half. Backfilling existing
Supabase data is a separate follow-up.
Bug 1 — Dual telemetry recording (`server/routes.py`)
=====================================================
`_handle_direct` wrapped the engine with `instrumented_generate`
unconditionally. When `app.state.engine` was already an
`InstrumentedEngine` (the common case when telemetry is wired in),
BOTH layers published `TELEMETRY_RECORD` — once from the inner
`InstrumentedEngine.generate`, once from the outer wrapper. Every
chat-completion request was counted twice in the leaderboard pipeline.
Fix: detect the InstrumentedEngine and unwrap to `._inner` before
passing to the wrapper so only one layer fires.
Bug 2 — KV-cache fallback over-counts multi-turn (`server/savings.py`)
=====================================================================
`compute_savings` falls back from `prompt_tokens_evaluated` to
`prompt_tokens` when the KV-cache-aware count is missing. But routes.py
aggregates by summing each turn's full prompt — which counts the system
prompt N times for an N-turn conversation. The fallback inflated FLOPs
and energy by N×.
Fix: use 0 (conservative under-count) when the evaluated count is
missing rather than falling back to the inflated `prompt_tokens` sum.
Bug 3 — TelemetryRecord lacked methodology versioning
=====================================================
There was no per-record version tag, so legacy (pre-fix) and current
records were silently aggregated together in the public leaderboard
even though they used different methodologies.
Fix: add `token_counting_version: Optional[int]` field to the
`TelemetryRecord` dataclass + a nullable column to the SQLite schema
(with idempotent migration); the constant moves from `server.savings`
to `core.types` to avoid the server→telemetry layering. New records
write the current version; pre-fix rows remain NULL. The aggregator
gains a `current_methodology_only=True` flag that filters NULL rows out
of leaderboard sums — local dashboards leave it off so historical
aggregates still render.
Bug 4 — Leaderboard JS displayed missing telemetry as legit zeros
=================================================================
Rows with significant token counts but `energy_wh_saved = 0` and
`flops_saved = 0` rendered as `0.00 Wh / 0 FLOPs` — visually identical
to a user who genuinely did almost nothing. The headline totals also
included these rows.
Fix: new `isMissingTelemetry()` detector renders missing-telemetry
energy/FLOPs cells as `—` (with a tooltip explaining why); a new
`.lb-missing` CSS class differentiates the placeholder visually.
Bug 5 — Outlier filter was too generous
=======================================
`MAX_ENERGY_WH_PER_TOKEN = 10` left a 10,000× margin that admitted
every Group B row even though those values are physically impossible
(would imply a space-heater per token). Same for the FLOPs cap at
`1e17`.
Fix: tighten to `0.5` Wh/token (still 500× over a typical consumer GPU)
and `1e15` FLOPs/token (still 10,000× over typical). This removes
existing pre-fix corrupt rows from the public view without touching
Supabase.
Tests
=====
New regression tests (all pass, ruff clean):
- `tests/server/test_routes.py::test_instrumented_engine_unwrapped_to_avoid_dual_telemetry`
— pins the bug 1 fix; asserts exactly ONE TELEMETRY_RECORD event per
request when the engine is already an InstrumentedEngine.
- `tests/server/test_savings.py` (NEW file, 3 tests) — pins the bug 2
fix (FLOPs not inflated via fallback) and the cost-side invariant
(dollar savings still use full prompt_tokens because cloud providers
bill per input token even when local KV cache hit).
- `tests/telemetry/test_aggregator.py::TestMethodologyFilter` (3 tests)
— default behaviour includes legacy rows (local dashboard parity);
`current_methodology_only=True` excludes them; the summary surface
honors the same filter.
Unrelated test note
===================
`tests/telemetry/test_energy_wiring.py::TestTelemetryStatsEnergy::test_export_includes_energy_fields`
and `::TestEndToEndPipeline::test_ask_to_export_with_energy` are flaky
on this branch but ALSO flaky on `main` (confirmed via stash + the
banner-only run). Root cause: `_version_check.py:132-135` prints the
"new version of OpenJarvis is available" banner to stdout outside the
throttle window, contaminating `json.loads(result.output)` in those two
tests. Reproducible with `OPENJARVIS_NO_UPDATE_CHECK=1` → all pass.
That's a separate bug (the banner shouldn't write to the same stream
as machine-readable output); not addressing it here to keep this PR
focused on the leaderboard pipeline.
What this PR explicitly does NOT do
====================================
- Backfill ~30% of Supabase rows that already shipped with inflated
values from pre-fix clients. That requires Supabase write access and
is the natural follow-up — see PR comments for proposed dry-run audit
query and the additive `methodology_version` column migration.
- Distinguish which past submissions came from buggy vs correct clients
retroactively (no `app_version` tag on existing submissions).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* test(evals): fix test_energy_scales_linearly under leaderboard fix
CI on #498 failed in the slow `test` job:
tests/evals/test_use_case_benchmarks.py::TestSavings::
test_energy_scales_linearly — ZeroDivisionError: float division by zero
Root cause: the test (and `test_energy_wh_matches_direct_formula`)
called `compute_savings(N, 0)` without `prompt_tokens_evaluated`. Under
the OLD buggy fallback, an unset evaluated count silently became N,
energy scaled linearly with N, and the test passed by accident. Under
this branch's conservative fix the fallback is 0, FLOPs collapse to 0,
and the `p10.energy_wh / p1.energy_wh` ratio divides 0 by 0.
The test's own docstring says "evaluated tokens (KV-cache model)" — it
was always meant to exercise the explicit-evaluated path, the API
call just didn't match. Pass `prompt_tokens_evaluated` explicitly so
the test now expresses the invariant it claims to.
Same one-line fix for `test_energy_wh_matches_direct_formula` plus an
explicit `flops > 0` sanity assertion — that test was passing trivially
with `0 == 0` after the conservative fallback fix, masking whether the
formula was actually being exercised.
No production code change in this commit. Verified locally: all 6
TestSavings tests pass; the 5 new regression tests added on this branch
still pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: krypticmouse <herumbshandilya123@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Closes#459. Reported by @jasonftl with a precise smoking gun: incoming ChannelMessage has channel="discord" (TYPE label) and conversation_id=<numeric channel id>, but the reply code was using cm.channel as the destination — Discord saw /channels/discord/messages and 404'd silently, blackholing every reply.
The bug was two field-mappings in channel_agent.py:_process_message (the happy path + the exception path):
self._channel.send(
msg.channel, # WRONG — TYPE label
reply,
conversation_id=msg.conversation_id, # WRONG — channel id used as msg-ref-id
)
The existing DiscordChannel.send() contract — proved by the existing test_send_with_conversation_id test — is:
- first positional `channel` = native destination ID (Discord channel id)
- `conversation_id` kwarg = native message ID for reply threading
Fix: swap both fields to the correct ones from ChannelMessage:
self._channel.send(
msg.conversation_id, # Discord channel id
reply,
conversation_id=msg.message_id, # message id for threading
)
Plus a defensive guard in discord_channel.py: when channel is empty, refuse fast with a clear warning instead of POSTing to /channels//messages and silently 404'ing.
3 new tests, 38 affected pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes#461. Reported and empirically validated by @swilliams76360.
Two bugs prevented authenticated MCP servers (e.g. Home Assistant) from working with OpenJarvis:
1. StreamableHTTPTransport never sent Authorization: Bearer <token> — constructor didn't accept a token kwarg and _build_headers() never set the header. Authenticated MCP servers always returned 401.
2. jarvis ask and jarvis serve never iterated config.tools.mcp.servers — only loaded tools from ToolRegistry. MCP tools were silently dropped on every CLI invocation.
The reporter's 3-file fix was correct; the workflow investigation surfaced a 4th file (agent_manager_routes.py:695, identical broken code) and an adversarial-review catch (MCP clients in _build_tools would be GC'd on function return, closing transports mid-request — fixed by stashing on agent._mcp_clients).
Edits:
- transport.py: token kwarg + Authorization header (skips on empty/None — avoids malformed "Bearer " that triggers confusing 400s).
- mcp/loader.py (NEW): shared load_mcp_tools_from_config helper returning (tools, clients). Caller MUST hold the clients reference.
- builder.py + agent_manager_routes.py: extract cfg.get("token"), forward to transport.
- cli/ask.py: _run_agent calls the loader, dedupes by spec.name (registry wins), stashes clients on agent._mcp_clients.
- cli/serve.py: same pattern in main-agent AND channel-agent paths; mcp_clients initialised before the accepts_tools branch so the post-instantiation reference is always valid.
22 new tests (transport + loader + discovery updates), 179 total cli/server/mcp tests pass on this branch.
Adversarial review interrogated 10 angles — slotted-class attr safety, MCPConfig duck-typing, config.tools.mcp AttributeError risk, dedup precedence, token leak via str(exc), logger scope in serve.py, _mcp_clients shadowing, _channel_mcp_clients lifetime, empty-token future-compat, lazy-import cost shift. Nine non-issues; the tenth (theoretical token leak via httpx exception str()) assessed as low actual risk because the token is a header value, not URL-embedded.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds --persona NAME / --persona none and a [memory_files].persona_name config field, resolving ~/.openjarvis/personas/<name>/{SOUL,MEMORY,USER}.md. Default (empty) preserves today's global-persona behavior exactly. Includes a path-traversal guard on persona names. Resolution lives in SystemPromptBuilder._resolve_persona so all callers benefit. Squad-derived: Ada (Qwen3.6-27B) authored the spec independently from the issue+code; Lucy (Qwen3-Coder-Next, 80B-A3B / ~3B active) implemented it; cross-function param threading completed in the test phase. 21 existing tests pass; +8 new persona-scope tests.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Consolidates two reports that overlap in scope:
- #476 (@senki): install.sh hardcoded `--python 3.11` but
pyproject.toml declares `requires-python = ">=3.10,<3.14"`. The
installer should track the project's allowed range, not pin a
conservative-three-years-ago version.
- #484 (@sanjayravit): install.sh crashed on hosts with no
python3/python on PATH because `mark_done`, `beacon`, and
`get_anon_id` all called $PY_CMD via inline Python heredocs.
This PR closes both gaps with five surgical edits to install.sh
(all behavior-preserving for the existing happy path; the bats
tests prove it):
1. get_anon_id — replace `python3 -c uuid` with POSIX
`/dev/urandom + od` + bash substring expansion. Same UUID v4
shape; works on Python-less hosts.
2. beacon — replace the 40-line Python heredoc with curl + a
shell-built JSON payload. All inputs are from controlled
sources (event ∈ fixed vocabulary, stage from stage_label(),
numeric ids/codes from validated arithmetic, anon_id from a
fresh UUID) so no general-purpose JSON escaping is needed.
`|| true` is load-bearing — PostHog 5xx must never abort an
install via the ERR trap.
3. mark_done — replace `python3 -c json.load+update+dump` with
awk that regenerates the file from scratch. Idempotent: if
the key is already marked, return early. Robust against
prior format drift. wsl key is always rewritten last so a
later FORCE_WSL=1 re-run correctly updates it.
4. parse_requires_python — new helper. Greps the project's
requires-python field, handles inclusive (`<=3.13`) and
exclusive (`<3.14`) upper bounds correctly, falls back to
3.11 if pyproject can't be parsed (the previous hardcoded
value — safe under the existing 3.10-3.13 range).
5. create_venv — call parse_requires_python instead of
hardcoding 3.11. The existing uv-managed-Python fallback
(from #444) still kicks in if the host doesn't have the
target version installed.
Adversarial review caught two real bugs before commit:
- HIGH: parse_requires_python's exclusive-bound regex would
also match the digits after `<=` (inclusive bound) and then
incorrectly subtract 1 — producing 3.12 from `<=3.13`. Fixed
by checking the inclusive form first.
- MEDIUM: the new "no Python on PATH" bats test silently skips
symlinking `pgrep` on hosts where it isn't at /usr/bin or
/bin (some minimal BusyBox configurations). Added an explicit
"no matching process" fallback so start_ollama's check works.
New bats coverage:
- `install succeeds with no system Python on PATH (#484)` —
builds a PATH that excludes python3/python and exercises the
full install. Asserts state file is written and contains
greppable step keys.
- `mark_done is idempotent — second mark of same key doesn't
duplicate` — re-runs the install and verifies install_uv
appears exactly once in install-state.json.
- `create_venv picks newest in requires-python range, not
hardcoded 3.11 (#476)` — asserts uv was called with
`--python 3.13` (the upper minor of `>=3.10,<3.14`).
The git stub now includes `requires-python = ">=3.10,<3.14"`
in its fake pyproject.toml so create_venv has something
realistic to parse.
@sanjayravit — your PR #484 motivated this consolidation;
closing that one as superseded with credit.
Co-authored-by: krypticmouse <herumbshandilya123@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
