* security: harden network-exposed surface
Hardening for the network-reachable attack surface, prioritizing fixes
that are strong but do not change working local/loopback defaults.
- auth_middleware: constant-time API key comparison (secrets.compare_digest)
for the HTTP path, and gate /metrics behind auth so operational counters
are not readable unauthenticated. /health stays open.
- webhook_routes: fail closed when a channel's secret/token is unset. Twilio,
BlueBubbles, WhatsApp (verify + inbound), and SendBlue now reject (403)
instead of processing unsigned/unauthenticated input. Constant-time
comparisons for BlueBubbles/SendBlue/WhatsApp verify token.
- http_request: follow redirects manually and re-run the SSRF check on every
hop (capped at 5) so an allowed public URL cannot 30x-redirect to an
internal/metadata address.
- api_routes /v1/memory/index: restrict indexing to OPENJARVIS_WORKSPACE roots
when configured and refuse sensitive files (.env, keys, credentials).
- config.toml: default [server] host to 127.0.0.1 (loopback) with a comment
on how to safely expose to a LAN (0.0.0.0 + API key).
Tests: new fail-closed webhook tests, /metrics auth tests, and SSRF
redirect block/follow tests; updated SendBlue tests for the new
secret-required behavior. Affected suites pass (95 tests), ruff clean.
* fix(http): keep SSRF redirect-following patchable via httpx.request
The manual redirect-following loop used a private httpx.Client, which
bypassed the `http_request.httpx.request` mock seam that consumers' tests
rely on (e.g. the twitter-bot GitHub-issue tests escaped to the real
network and 401'd). Issue each hop via module-level httpx.request with
follow_redirects=False instead — same per-hop SSRF re-check, restored
testability.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Memory tools degraded silently and misleadingly when the mandatory
`openjarvis_rust` extension was absent from the *serving* venv:
- `POST /v1/memory/store` returned HTTP 200 `{"status":"stored","note":
"no backend available"}` and stored nothing (silent data loss).
- `POST /v1/memory/index` returned a generic "No memory backend available",
and the desktop frontend discarded the server `detail` and threw a blanket
"Failed to index path", blaming the path instead of the real cause.
- `GET /v1/memory/config` reported `backend_type: sqlite` even though no
backend could be constructed.
Root cause: `SQLiteMemory.__init__` calls `get_rust_module()` (which raises
ImportError by design — the Rust ext is mandatory, no Python fallback), and
`_get_memory_backend` swallowed that ImportError and returned `None`,
conflating "native extension missing" (a hard install error) with "memory
intentionally disabled" (benign). A chunking floor also silently dropped whole
short documents, and the installer never verified the extension imported from
the serving venv before writing its success marker.
Fix (no fake Python fallback — the Rust ext stays mandatory by design):
- Add `MemoryBackendUnavailable` + `RUST_MISSING_HINT` in tools/storage/_stubs.
`SQLiteMemory.__init__` translates the bridge ImportError into this clear,
actionable error ("run `uv run maturin develop ...`").
- `_get_memory_backend` distinguishes the two cases: a missing native ext
raises HTTP 503 with the actionable hint; a benign unconfigured backend
still returns `None` (graceful path preserved for search/stats).
- `/store` now returns 503 instead of a 200 silent no-op.
- `/config` reports `available: false` + `detail` instead of falsely claiming
a healthy `backend_type`.
- `/index` adds a `note` when `chunks_indexed == 0` so "indexed" never
silently means "stored nothing".
- chunk_text no longer drops an entire short document below `min_chunk_size`
(the floor only discards tiny *trailing* fragments now).
- Frontend `storeMemory`/`indexMemoryPath` surface the server `detail` instead
of blanket strings; `MemoryConfig` gains optional `available`/`detail`.
(Left the pre-existing `backend` vs `backend_type` mismatch untouched.)
- build-extension.sh verifies `import openjarvis_rust` succeeds in the serving
venv before writing the `extension-built` marker.
Regression tests: tests/server/test_api_routes.py::TestMemoryRustMissing mocks
`get_rust_module` to raise ImportError and asserts /store (503, not 200 no-op),
/index (actionable detail, not "Failed to index path"), and /config
(available:false) all surface the clear error; tests/memory/test_chunking.py
asserts short-only docs are kept while tiny trailing fragments are still
filtered.
Fixes#502
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(chat): honor model config in managed agent chat (#477)
* test(server): regression test for managed-agent engine resolution
_make_lightweight_system must resolve the user's configured engine
(intelligence.preferred_engine, else engine.default) via get_engine,
not a hardcoded OllamaEngine (#477/#514). Asserts the key passed to
get_engine (captured before the system is built); runs under the server
extra (fastapi). Verified: passes on the fix, fails (KeyError) against the
pre-fix hardcoded-Ollama code.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(leaderboard): correct telemetry pipeline and outlier handling
The public leaderboard at /leaderboard showed a clear bimodal Wh/token
distribution: most users at ~3-5 J/token, ~30% inflated by 1000-4000×.
A 5-agent investigation workflow + 4-agent verification pass traced the
inflation to a cluster of related bugs across telemetry, server, and
display layers. This PR fixes the in-repo half. Backfilling existing
Supabase data is a separate follow-up.
Bug 1 — Dual telemetry recording (`server/routes.py`)
=====================================================
`_handle_direct` wrapped the engine with `instrumented_generate`
unconditionally. When `app.state.engine` was already an
`InstrumentedEngine` (the common case when telemetry is wired in),
BOTH layers published `TELEMETRY_RECORD` — once from the inner
`InstrumentedEngine.generate`, once from the outer wrapper. Every
chat-completion request was counted twice in the leaderboard pipeline.
Fix: detect the InstrumentedEngine and unwrap to `._inner` before
passing to the wrapper so only one layer fires.
Bug 2 — KV-cache fallback over-counts multi-turn (`server/savings.py`)
=====================================================================
`compute_savings` falls back from `prompt_tokens_evaluated` to
`prompt_tokens` when the KV-cache-aware count is missing. But routes.py
aggregates by summing each turn's full prompt — which counts the system
prompt N times for an N-turn conversation. The fallback inflated FLOPs
and energy by N×.
Fix: use 0 (conservative under-count) when the evaluated count is
missing rather than falling back to the inflated `prompt_tokens` sum.
Bug 3 — TelemetryRecord lacked methodology versioning
=====================================================
There was no per-record version tag, so legacy (pre-fix) and current
records were silently aggregated together in the public leaderboard
even though they used different methodologies.
Fix: add `token_counting_version: Optional[int]` field to the
`TelemetryRecord` dataclass + a nullable column to the SQLite schema
(with idempotent migration); the constant moves from `server.savings`
to `core.types` to avoid the server→telemetry layering. New records
write the current version; pre-fix rows remain NULL. The aggregator
gains a `current_methodology_only=True` flag that filters NULL rows out
of leaderboard sums — local dashboards leave it off so historical
aggregates still render.
Bug 4 — Leaderboard JS displayed missing telemetry as legit zeros
=================================================================
Rows with significant token counts but `energy_wh_saved = 0` and
`flops_saved = 0` rendered as `0.00 Wh / 0 FLOPs` — visually identical
to a user who genuinely did almost nothing. The headline totals also
included these rows.
Fix: new `isMissingTelemetry()` detector renders missing-telemetry
energy/FLOPs cells as `—` (with a tooltip explaining why); a new
`.lb-missing` CSS class differentiates the placeholder visually.
Bug 5 — Outlier filter was too generous
=======================================
`MAX_ENERGY_WH_PER_TOKEN = 10` left a 10,000× margin that admitted
every Group B row even though those values are physically impossible
(would imply a space-heater per token). Same for the FLOPs cap at
`1e17`.
Fix: tighten to `0.5` Wh/token (still 500× over a typical consumer GPU)
and `1e15` FLOPs/token (still 10,000× over typical). This removes
existing pre-fix corrupt rows from the public view without touching
Supabase.
Tests
=====
New regression tests (all pass, ruff clean):
- `tests/server/test_routes.py::test_instrumented_engine_unwrapped_to_avoid_dual_telemetry`
— pins the bug 1 fix; asserts exactly ONE TELEMETRY_RECORD event per
request when the engine is already an InstrumentedEngine.
- `tests/server/test_savings.py` (NEW file, 3 tests) — pins the bug 2
fix (FLOPs not inflated via fallback) and the cost-side invariant
(dollar savings still use full prompt_tokens because cloud providers
bill per input token even when local KV cache hit).
- `tests/telemetry/test_aggregator.py::TestMethodologyFilter` (3 tests)
— default behaviour includes legacy rows (local dashboard parity);
`current_methodology_only=True` excludes them; the summary surface
honors the same filter.
Unrelated test note
===================
`tests/telemetry/test_energy_wiring.py::TestTelemetryStatsEnergy::test_export_includes_energy_fields`
and `::TestEndToEndPipeline::test_ask_to_export_with_energy` are flaky
on this branch but ALSO flaky on `main` (confirmed via stash + the
banner-only run). Root cause: `_version_check.py:132-135` prints the
"new version of OpenJarvis is available" banner to stdout outside the
throttle window, contaminating `json.loads(result.output)` in those two
tests. Reproducible with `OPENJARVIS_NO_UPDATE_CHECK=1` → all pass.
That's a separate bug (the banner shouldn't write to the same stream
as machine-readable output); not addressing it here to keep this PR
focused on the leaderboard pipeline.
What this PR explicitly does NOT do
====================================
- Backfill ~30% of Supabase rows that already shipped with inflated
values from pre-fix clients. That requires Supabase write access and
is the natural follow-up — see PR comments for proposed dry-run audit
query and the additive `methodology_version` column migration.
- Distinguish which past submissions came from buggy vs correct clients
retroactively (no `app_version` tag on existing submissions).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* test(evals): fix test_energy_scales_linearly under leaderboard fix
CI on #498 failed in the slow `test` job:
tests/evals/test_use_case_benchmarks.py::TestSavings::
test_energy_scales_linearly — ZeroDivisionError: float division by zero
Root cause: the test (and `test_energy_wh_matches_direct_formula`)
called `compute_savings(N, 0)` without `prompt_tokens_evaluated`. Under
the OLD buggy fallback, an unset evaluated count silently became N,
energy scaled linearly with N, and the test passed by accident. Under
this branch's conservative fix the fallback is 0, FLOPs collapse to 0,
and the `p10.energy_wh / p1.energy_wh` ratio divides 0 by 0.
The test's own docstring says "evaluated tokens (KV-cache model)" — it
was always meant to exercise the explicit-evaluated path, the API
call just didn't match. Pass `prompt_tokens_evaluated` explicitly so
the test now expresses the invariant it claims to.
Same one-line fix for `test_energy_wh_matches_direct_formula` plus an
explicit `flops > 0` sanity assertion — that test was passing trivially
with `0 == 0` after the conservative fallback fix, masking whether the
formula was actually being exercised.
No production code change in this commit. Verified locally: all 6
TestSavings tests pass; the 5 new regression tests added on this branch
still pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: krypticmouse <herumbshandilya123@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* fix(server): load SOUL.md / USER.md context in streaming chat
* refactor+test: extract _build_managed_system_prompt + cover #431
The streaming persona fix was inline and untestable without a live
engine. Extract it into _build_managed_system_prompt (matching this
module's extract-and-unit-test pattern for the streaming helpers) and
add regression tests:
- SOUL.md persona is injected into the streaming system prompt (#431),
- the agent's own template is preserved,
- output matches a directly-constructed SystemPromptBuilder (parity with
the CLI/ask path — the whole point of the fix).
Behavior unchanged from the original PR; this only makes it testable and
locks in CLI parity.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Jon Saad-Falcon <41205309+jonsaadfalcon@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When a client streams (stream:true) with explicit `tools`, the server
routed to the agent stream bridge, which ignored request_body.tools, ran
the agent's own tool loop, and word-split filler content into fake token
deltas — dropping the caller's tool_calls. This is the streaming analog
of #414 (whose non-streaming fix was #454).
Now stream+tools bypasses the agent and streams the model's raw
function-calling decision via engine.stream_full(), emitting OpenAI-shape
tool_calls deltas and a tool_calls finish_reason. Adds tool_calls to
DeltaMessage and removes the now-dead _handle_agent_stream.
Verified end-to-end on Ollama (qwen3.5:4b) plus a unit regression test.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes#414.
Root cause: routes.py:151 unconditionally routed non-streaming /v1/chat/completions through _handle_agent when an agent was registered. _handle_agent calls agent.run(input_text) which IGNORES request_body.tools entirely, runs the agent's own internal tool loop with its own (different) tool spec, and returns only result.content — never result.tool_calls. The "Understood. If you have another request..." filler is not hardcoded anywhere in OpenJarvis (the cloud_router.py:126 "Understood." is a different Gemini-only injection). It's the model's actual generic response when the agent re-prompts it without the user's intended tools.
Fix: one conditional. Skip _handle_agent when request_body.tools is present — the client is asking for raw OpenAI-compat function-calling, so route to _handle_direct which preserves tool_calls. Plus a forward-looking comment documenting this as an intentional trade-off so a future maintainer doesn't naively remove the guard.
Streaming path left intact (its asymmetry — "use agent_stream WHEN tools present" — is intentional per the existing comment at lines 143-145; reporter's repro is non-streaming).
Two regression tests:
- test_with_tools_bypasses_agent: mocks engine+agent, asserts tool_calls survives, agent.run is NOT called.
- test_without_tools_still_uses_agent: pins existing behavior for the no-tools path.
Reported by @gilbert-barajas — the side-by-side curl repro made the triage tractable.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`_stream_managed_agent` had diverged from the canonical cli/ask.py path and
lost three behaviours. All three are fixed via small extracted, unit-tested
helpers:
- #382: cross-request history replay dropped stored `tool_calls`, so the model
never saw its own prior tool use and fabricated tool output on turn 2+.
`_replay_history_messages` now reconstructs the assistant tool-use message
plus matching tool-result messages (synthesised, consistent tool_call_ids).
- #386: only temperature/max_tokens reached the engine. `_sampler_kwargs`
forwards repetition_penalty / top_p / top_k / min_p / frequency_penalty /
presence_penalty when set in the agent config (opt-in; default agents send
nothing extra). Fixes degenerate repetition loops on local models with no
repetition_penalty.
- #395: tools were built with a bare `tool_cls()`, so memory_* / channel_* /
llm tools loaded with no backend and failed on every call.
`_instantiate_managed_tool` injects backend / channel / engine the same way
cli/ask.py::_build_tools does.
Verified empirically: replay emits user → assistant(tool_calls) → tool(result)
with matching ids; sampler extraction forwards only set keys; DI gives memory
tools a backend and llm the engine/model. New tests in
tests/server/test_managed_agent_streaming.py (helpers are pure, so verifiable
without a live engine). 38 passed locally incl. existing route tests.
Closes#382Closes#386Closes#395
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`AuthMiddleware` is a BaseHTTPMiddleware and never intercepts WebSocket
upgrade requests, so `/v1/chat/stream` and `/v1/agents/events` accepted any
connection — leaking all agent events/message content and allowing
unauthenticated inference even when an API key was configured for HTTP. The
A2A JSON-RPC server likewise dispatched every request without auth.
- Add `websocket_authorized(websocket, expected_key)` (constant-time compare)
and check it in both WS handlers BEFORE `accept()`, closing with code 1008
on failure. Token is read from `?token=` (browsers can't set WS headers) or
an `Authorization: Bearer` header. `create_app` now exposes the key via
`app.state.api_key`; when empty, auth is disabled, matching the HTTP
middleware's local-default behavior (so loopback dev is unchanged).
- A2AServer gains an optional `auth_token`: `handle_request(token=...)`
rejects with JSON-RPC -32001 before dispatch when configured, advertises
`{"schemes": ["bearer"]}` on the agent card, and stays open when unset.
Added `A2AConfig.auth_token`.
Verified empirically against the real mounted endpoints via TestClient: no
token / wrong token are rejected at the handshake (WebSocketDisconnect),
correct token streams normally, and no-key configs still connect freely.
Closes#217
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tests were storing digests with local time but get_today() filters
by UTC date, causing failures when local date != UTC date.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CI installs `--extra dev` but not `--extra server`, so fastapi is
unavailable. All other server tests use pytest.importorskip("fastapi")
to skip gracefully — this file was missing the guard.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- personal_deep_research.toml: add schedule_value="" for template test compat
- test_deep_research_tools_wiring: skip when fastapi not installed in CI
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add deep_research_agent parameter to ChannelBridge.__init__ and update
_handle_chat to try DeepResearchAgent first, falling back to system.ask()
when no research agent is configured.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a new POST /{connector_id}/sync endpoint alongside the existing GET
sync-status endpoint. The new endpoint validates the connector is registered
and connected, then runs SyncEngine.sync() and returns chunks_indexed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Merge main into fix/ssrf-check, keeping both the auto-recover
logic for error-state agents and the async streaming support
for the send_message endpoint.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements create_connectors_router() with GET /connectors, GET
/connectors/{id}, POST /connectors/{id}/connect, POST
/connectors/{id}/disconnect, and GET /connectors/{id}/sync endpoints.
Includes 6 passing tests covering list, detail, 404, connect, disconnect,
and sync status.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add `stream: bool` parameter to `POST /v1/managed-agents/{id}/messages`.
When `stream=true`, the agent processes the message synchronously and
returns an SSE stream (OpenAI-compatible format) with token-by-token
response, tool result events, and usage metadata.
This enables real-time voice assistants and chat UIs to receive agent
responses as they are generated, rather than polling for completion.
- Extend SendMessageRequest with `stream` field (default: false)
- Add _stream_managed_agent() helper using asyncio.to_thread()
- Build AgentContext from conversation history for multi-turn support
- Emit tool_results as named SSE events
- Persist agent response in DB after streaming completes
- Add 6 new tests covering streaming behavior
- Update agents.md documentation with streaming examples
- Acquire tick BEFORE spawning thread to prevent race condition on concurrent Run Now clicks
- Auto-recover agents in error/needs_attention state when Run Now is clicked
- Auto-recover error-state agents when receiving immediate messages
- End tick on system build failure to avoid stuck running state
- Add test verifying concurrent start_tick raises ValueError
* feat: model catalogue with download/delete and auto-pull Qwen3.5
- Desktop boot: start server immediately with fallback model (qwen3:0.6b),
then pull preferred model (qwen3.5:4b) and remaining Qwen3.5 variants
that fit in RAM in the background. No more broken "Select model" state.
- Backend: add POST /v1/models/pull and DELETE /v1/models/{name} endpoints
so the frontend can trigger model downloads and deletions via Ollama.
- Frontend: redesign CommandPalette (Cmd+K) with two tabs — "Installed"
shows pulled models with select/delete, "Download Models" shows a
catalogue of popular models plus a custom model input field.
- Fix ollama_has_model() to use exact tag matching instead of prefix
matching, preventing false positives.
* fix: streaming, model switching, second-largest default, and tests
- Streaming: use direct engine streaming for non-tool requests so tokens
arrive in real-time instead of being batched by the agent bridge.
Add error handling to _handle_stream so engine errors surface as
content chunks instead of silent failures.
- Model selection: pick the second-largest Qwen3.5 model that fits
(leaves headroom for OS/apps) instead of the absolute largest.
- Model switching: abort in-flight stream when the user changes models
mid-generation, preventing stale-model errors. Improve error messages
in catch blocks.
- Tests: add tests/server/test_model_management.py with 11 tests
covering model pull/delete endpoints, streaming error resilience,
direct-engine streaming bypass, and model listing. All 100 server
tests pass.
The SecurityHeadersMiddleware ran before CORSMiddleware (Starlette
executes middleware in LIFO order) and added headers to OPTIONS
preflight requests. The Content-Security-Policy: default-src 'self'
header told the browser to reject cross-origin connections, so fetch()
from the Tauri webview (https://tauri.localhost) to the API server
(http://127.0.0.1) was blocked — causing "Failed to get response" on
every chat message in the desktop app.
Two fixes:
- Skip security headers on OPTIONS requests so CORS preflight works
- Remove Content-Security-Policy from API responses — it is a
document-level browser policy irrelevant to JSON API responses and
breaks any cross-origin API consumer
