Commit Graph
248 Commits
Author SHA1 Message Date
Jon Saad-FalconandClaude Opus 4.6 35249b4cb4 fix: markdown rendering in Interact, separate instruction section
1. Agent responses in Interact tab now render as markdown (headers,
   bold, lists, etc.) instead of raw text with ** and ##
2. Instruction section is its own box above Configuration in Overview
3. Edit button is inline next to "Instruction" heading
4. User messages stay as plain text (no markdown needed)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 13:33:42 -07:00
Jon Saad-FalconandClaude Opus 4.6 037b40a937 fix: wizard hover highlight, card layout, editable schedule & instruction
1. Template cards highlight with accent border on hover
2. Text aligned top-left, emoji inline with title
3. Schedule is now an editable dropdown (not hardcoded display)
4. Overview tab shows current instruction with Edit button
5. Instruction is editable and saved to agent config

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 13:09:41 -07:00
Jon Saad-FalconandGitHub b55b93a9c5 feat: simplify agent wizard, wire tools, add smart defaults (#149)
feat: simplify agent wizard, wire tools, add smart defaults
2026-03-27 12:54:30 -07:00
Jon Saad-Falcon 5a37e9c774 fix: remove unused TestClient import in recommended model tests 2026-03-27 12:48:31 -07:00
Jon Saad-Falcon 5c942f41b3 chore: remove spec and plan docs from branch 2026-03-27 12:42:47 -07:00
Jon Saad-FalconandClaude Opus 4.6 cbabcd4a8f feat: rewrite agent wizard — 2-step flow with smart defaults
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 12:36:07 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 e4466146ce feat: add /v1/recommended-model endpoint
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 12:27:24 -07:00
Jon Saad-FalconandClaude Opus 4.6 7366681f62 feat: wire tools from agent config to executor via ToolRegistry
Resolve tool names from config["tools"] into actual tool instances
using ToolRegistry, inject runtime deps, and pass them to the agent
constructor instead of hardcoded tools=[].

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 11:32:57 -07:00
Jon Saad-FalconandGitHub b82575ea5b Merge pull request #147 from open-jarvis/feat/mcp-streaming-tool-calls
feat: fix external MCP server integration and add streaming tool-call support
2026-03-27 11:30:03 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 ccea853e5e feat: expand system_prompt_template with instruction on agent creation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 11:28:18 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 043c86b321 feat: add system_prompt_template, icon, update defaults in agent templates
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 11:25:41 -07:00
Jon Saad-FalconandGitHub b4e70a5a7d Merge pull request #148 from open-jarvis/fix/leaderboard-outlier-filter
fix: filter outlier entries from leaderboard instead of recomputing
2026-03-27 11:22:09 -07:00
Jon Saad-FalconandClaude Opus 4.6 049d6ff543 docs: add agent wizard simplification implementation plan
7-task plan covering template TOMLs, manager expansion, executor tool
wiring, recommended-model endpoint, frontend API, wizard rewrite,
and integration testing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 11:21:01 -07:00
krypticmouseandClaude Opus 4.6 8899662fc5 fix: line-too-long lint error in agent_manager_routes.py
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:19:09 +00:00
Jon Saad-FalconandClaude Opus 4.6 264998b36c fix: filter outlier entries from leaderboard instead of recomputing
Replace the recompute approach from #146 with per-token outlier
detection.  Entries whose energy, FLOPs, or dollar savings exceed
generous per-token thresholds (~1000x legitimate values) are hidden
from the leaderboard entirely.  All displayed values come directly
from the database — no rewriting of submitted data.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 11:17:31 -07:00
Jon Saad-FalconandClaude Opus 4.6 728896ae86 docs: add agent wizard simplification design spec
Sub-project A spec covering: 2-step wizard, smart template defaults,
rich system prompt templates, tool wiring fix, recommended model
endpoint, and Advanced settings collapse.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 11:15:37 -07:00
b39dbedcc4 feat: fix external MCP server integration and add streaming tool-call support
Rebased and cleaned-up version of PR #113 by @mricharz, resolved against
current main (including Codex engine, Gemini thought_signature, and
agent manager fixes merged since the original PR).

MCP Transport & Client:
- StreamableHTTPTransport with session tracking, SSE parsing, timeouts
- MCPClient.initialize() sends proper MCP handshake (protocolVersion,
  capabilities, clientInfo) + notifications/initialized
- Fix StdioTransport constructor: command=[command] + args
- MCPRequest.to_dict() with notification support (id=None)

External MCP Discovery:
- _discover_external_mcp supports both url (HTTP) and command (stdio)
- Per-server include_tools / exclude_tools filtering
- MCP clients persisted on JarvisSystem for runtime lifetime

Streaming Tool-Call Support (stream_full):
- StreamChunk dataclass in _stubs.py (content, tool_calls, finish_reason, usage)
- Default stream_full() on InferenceEngine ABC wraps stream() for backward compat
- _OpenAICompatibleEngine.stream_full() with SSE parsing
- CloudEngine: _stream_full_openai (OpenAI/OpenRouter/MiniMax/Codex routing)
               _stream_full_anthropic (event-based → OpenAI delta format)
- InstrumentedEngine, MultiEngine: stream_full delegation
- GuardrailsEngine: stream_full with post-hoc security scanning (FIXED:
  original PR bypassed output scanning — now accumulates and scans like stream())

Other improvements:
- _prepare_anthropic_messages() extracted to eliminate duplication
- Default tool_choice=auto when tools are provided (OpenAI compat engines)
- MCP tool injection into managed agent streaming path
- Documentation: docs/user-guide/mcp-external-servers.md

Tests: ~59 new tests across 8 test files, all passing.

Closes PR #113

Co-Authored-By: mricharz <mricharz@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:09:54 +00:00
Jon Saad-FalconandGitHub eadcc3bb60 Merge pull request #146 from open-jarvis/fix/leaderboard-anti-gaming
fix: recompute leaderboard energy/FLOPs from tokens, cap dollar savings
2026-03-27 11:03:19 -07:00
Jon Saad-FalconandClaude Opus 4.6 77b5567e47 fix: recompute leaderboard energy/FLOPs from tokens, cap dollar savings
Energy and FLOPs are now derived from total_tokens on the leaderboard
display using Claude Opus 4.6 constants, rather than trusting submitted
DB values. This prevents gaming (e.g. TotallyNoire submitting 14B Wh
from 15M tokens). Dollar savings are clamped at the theoretical max
($25/1M tokens) on both the leaderboard and frontend submission side.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 10:58:36 -07:00
Jon Saad-FalconandGitHub 957ce83955 Merge pull request #145 from open-jarvis/docs/macos-installation-guide
docs: add macOS installation guide for llama.cpp
2026-03-27 10:46:55 -07:00
Jon Saad-FalconandGitHub b8b3369ad1 Merge pull request #144 from open-jarvis/fix/migrate-savings-data
fix: add SQL migration to recompute historical leaderboard savings
2026-03-27 09:17:55 -07:00
0ce10bd97b docs: add macOS installation guide for llama.cpp
Comprehensive step-by-step guide covering Homebrew, uv, Rust, llama.cpp,
model download, Python 3.12 pin (PyO3 compat), and common pitfalls.

Cherry-picked from PR #131 by @gridworks — cleaned up to include only
the docs content (removed duplicate files, binary artifacts, and
unrelated lockfile changes from the original PR).

Co-Authored-By: gridworks <5502067+gridworks@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 16:17:17 +00:00
Jon Saad-FalconandClaude Opus 4.6 d7d0125454 fix: add SQL migration to recompute historical leaderboard savings
Companion to #143 which fixed the frontend to use Claude Opus 4.6 as
the sole baseline. This migration recomputes existing Supabase rows
using the exact closed-form: new = T/3.8M + 10*old/19, derived from
the original triple-provider formula.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 09:13:31 -07:00
Jon Saad-FalconandGitHub bfdc71d31d Merge pull request #142 from open-jarvis/feat/codex-engine-support
feat: add Codex cloud engine for ChatGPT Plus/Pro subscribers
2026-03-26 19:06:46 -07:00
Jon Saad-FalconandGitHub 44626b0ef8 Merge pull request #143 from open-jarvis/fix/leaderboard-savings-single-provider
fix: use Claude Opus 4.6 as sole baseline for leaderboard savings
2026-03-26 19:04:23 -07:00
Jon Saad-FalconandClaude Opus 4.6 c807666bd5 fix: cost format $0.0000, compact overview layout
- Remove cent sign from cost display, use $X.XXXX format
- Compact stat cards (horizontal icon+value layout)
- Tighter config grid spacing with bolder labels
- Reduce padding and gaps throughout overview tab

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 19:00:39 -07:00
Jon Saad-FalconandClaude Opus 4.6 fc369b9ab0 fix: use Claude Opus 4.6 as sole baseline for leaderboard savings
Dollar savings were previously summed across all three cloud providers
(GPT-5.3 + Claude Opus 4.6 + Gemini 3.1 Pro), inflating the reported
number by ~3x. Now uses only Claude Opus 4.6 pricing as the baseline,
with an asterisk footnote on the leaderboard explaining this.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 18:59:40 -07:00
krypticmouseandClaude Opus 4.6 63ae942116 feat: add Codex cloud engine for ChatGPT Plus/Pro subscribers
Adds `codex/` prefixed model support using the OpenAI Responses API —
the same protocol used by zeroclaw and other Codex-compatible tools.

Live-tested against gpt-5-mini-2025-08-07 with:
- Generate (non-streaming): confirmed working
- System prompt → instructions mapping: confirmed working
- SSE streaming: confirmed working (9 chunks)
- End-to-end via `jarvis ask`: confirmed working

Implementation:
- Default endpoint: api.openai.com/v1/responses (standard API key)
- Override via OPENAI_CODEX_BASE_URL for ChatGPT OAuth tokens
  (e.g. chatgpt.com/backend-api/codex)
- Auth via OPENAI_CODEX_API_KEY env var
- Responses API format: input array, instructions field, output_text extraction
- Handles reasoning+message output blocks correctly
- SSE streaming parses response.output_text.delta events

Models: codex/gpt-4o, codex/gpt-4o-mini, codex/o3-mini,
        codex/gpt-5-mini, codex/gpt-5-mini-2025-08-07

Usage:
  export OPENAI_CODEX_API_KEY="your-api-key-or-oauth-token"
  jarvis ask "Hello" --model codex/gpt-5-mini-2025-08-07

Closes #134

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 01:54:36 +00:00
Jon Saad-FalconandClaude Opus 4.6 c1e02a236a feat: show model in agent overview, split tokens, editable model
Overview tab:
- Show Intelligence (model name) with click-to-change dropdown
- Split "Total Tokens" into "Input Tokens" and "Output Tokens"
- Model can be switched for existing agents via dropdown

Backend:
- Add input_tokens/output_tokens columns to managed_agents
- Track prompt_tokens and completion_tokens separately in executor
- Disable Ollama thinking by default (think:false) to prevent
  empty responses from token exhaustion

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 18:32:25 -07:00
Jon Saad-FalconandGitHub 813d704201 Merge pull request #140 from open-jarvis/fix/pinchbench-tool-args-and-sessions
fix(evals): pipe tool arguments to transcript + multi-session support
2026-03-26 18:29:36 -07:00
Jon Saad-FalconandClaude Opus 4.6 8590bc5140 fix: disable Ollama thinking by default, fix token tracking
Root cause of empty responses: Qwen3.5's extended thinking mode
consumes all tokens (4096) on hidden <think> tags, leaving zero
visible content. The /no_think text tag was unreliable.

Fix: pass think=false in the Ollama API payload, which properly
disables thinking at the API level. Drops token usage from ~4096
to ~5-200 per response and eliminates empty content.

Also:
- Fix token tracking to read total_tokens from metadata (was looking
  for tokens_used which is never set)
- Remove the /no_think system prompt hack (superseded by API param)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 18:20:05 -07:00
Jon Saad-FalconandGitHub f61d690791 Merge pull request #141 from open-jarvis/feat/security-scan-enhancements
feat: enhance security scan with DNS check, JSON output, and API endpoint
2026-03-26 18:01:30 -07:00
4d05475ec4 feat: enhance security scan with DNS check, JSON output, Rich UI, and API endpoint
Incorporates the useful new features from PR #135 (by @gridworks) on top
of the existing PrivacyScanner implementation:

- Add DNS configuration check (macOS, via scutil --dns)
- Add --json flag to `jarvis scan` for machine-readable output
- Add --no-scan flag to `jarvis init` to skip the post-init audit
- Expand remote-access process list (ngrok, tailscaled, cloudflared, ZeroTier)
- Upgrade `jarvis scan` output from plain text to Rich table
- Add GET /v1/security/scan API endpoint
- Add tests for all new features (30 tests, all passing)

Closes #133

Co-Authored-By: gridworks <5502067+gridworks@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 01:00:38 +00:00
Jon Saad-FalconandClaude Opus 4.6 11d1643261 feat: live agent progress visibility in Interact tab
Add current_activity field to managed agents that the executor updates
at each phase of a tick (loading model, delivering messages, generating
response, retrying, finalizing). The frontend polls this every 2s and
displays the live status instead of static "Agent is thinking...".

Backend:
- Add current_activity column to managed_agents (migration)
- Add _set_activity helper to AgentExecutor
- Update activity at: start_tick, model load, message delivery,
  generation, retry, finalize
- Clear activity on end_tick

Frontend:
- InteractTab polls both messages and agent status in parallel
- Shows current_activity text with pulsing indicator
- Falls back to "Agent is thinking..." if activity is empty

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 17:54:08 -07:00
Jon Saad-FalconandGitHub 7f336d7679 Merge pull request #130 from jbergant/fix/qwen35-model-catalog-mlx-129
fix: correct Qwen3.5 model sizes and MLX repos in catalog
2026-03-26 17:50:44 -07:00
Jon Saad-FalconandClaude Opus 4.6 1531ad656c fix(evals): pipe tool arguments to transcript + multi-session support
Two fixes for the last 4 failing PinchBench tasks:

1. Tool arguments (tasks 05, 07, 13): Arguments were lost in the
   pipeline — ToolExecutor stored them but system.py stripped them
   from the tool_results dict, so the LLM judge saw write_file({})
   instead of the actual content. Now pipe arguments through:
   _stubs.py → system.py → scorer transcript.

2. Multi-session tasks (task 22): Parse `sessions` field from task
   frontmatter, use first session's prompt as record.problem, and
   execute remaining sessions sequentially within the workspace
   context in _process_one().

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 00:43:22 +00:00
Jon Saad-FalconandClaude Opus 4.6 756e28b109 fix: create OllamaEngine directly, skip health checks that stall ticks
get_engine() probes all engines with health checks, which can load
different models and interfere with in-flight Ollama requests, causing
intermittent empty responses. Create a plain OllamaEngine directly
from config instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 17:34:51 -07:00
Jon Saad-FalconandClaude Opus 4.6 885b2caf99 fix: suppress Qwen3.5 thinking mode, retry on empty content, add logging
- Append /no_think to system prompt to prevent Qwen3.5 from consuming
  all tokens on extended thinking and producing empty visible output
- Retry once if agent returns empty content
- Add debug logging to _make_lightweight_system for engine diagnostics

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 17:23:18 -07:00
aab743a6c7 fix(engine): support Gemini thought_signature for multi-turn tool calling (#139)
Gemini 3.1+ reasoning models require a thought_signature field in
function_call parts when replaying conversation history. Without it,
the API returns 400 INVALID_ARGUMENT on every multi-turn tool call.

Changes:
- CloudEngine: capture thought_signature from Gemini responses and
  store in _thought_sigs dict keyed by tool_call id
- CloudEngine: replay thought_signature when building function_call
  parts for Gemini conversation history
- native_openhands: thread thought_signature through via side dict
  (ToolCall uses slots, can't add dynamic attributes)
- Add PinchBench eval configs for Claude Opus 4.6, Gemini 3.1 Pro,
  Nemotron-3-Super, Qwen 122B, and Qwen 35B

Impact: Gemini 3.1 Pro PinchBench score 4% → 78%

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 17:18:48 -07:00
Jon Saad-FalconandClaude Opus 4.6 8b358620da fix: use fresh engine for agent ticks, not wrapped server engine
The server's engine is wrapped in MultiEngine → InstrumentedEngine →
GuardrailsEngine. When reused from a background thread for agent ticks,
this chain returns empty content. Create a fresh OllamaEngine for each
tick instead, which reliably returns model output.

Also fixes Run Now endpoint to use the same lightweight system approach.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 16:47:00 -07:00
Jon Saad-FalconandClaude Opus 4.6 808ec6eb5c feat: change default startup model to qwen3.5:4b
Update STARTUP_MODEL from 2b to 4b for better quality on first launch.
Also update preferred_model() to prefer STARTUP_MODEL when it fits,
rather than always picking the third-largest model. This gives a
consistent default across machines while still falling back to
RAM-appropriate sizing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 16:17:57 -07:00
Jon Saad-FalconandGitHub e2f4d9f2f4 fix: use server's model for agent ticks, add logging (#138)
fix: use server's model for agent ticks, add logging
2026-03-26 16:04:31 -07:00
Jon Saad-FalconandClaude Opus 4.6 19fe061696 fix: use server's model for agent ticks, add executor logging, simplify send UI
Root cause: _run_tick and _immediate_tick called SystemBuilder().build()
which picks the first model from Ollama (qwen3.5:35b) instead of the
model the server was started with (e.g. qwen3.5:9b). A 0.6B query was
running on a 35B model, causing 5+ minute stalls.

Fix: reuse the server's engine/model from app.state via a lightweight
system facade instead of rebuilding the full JarvisSystem.

Also:
- Add detailed logging to AgentExecutor (model, pending messages,
  timing, content length, errors with tracebacks)
- Add logging to immediate tick lifecycle
- Remove Queue button from Interact tab (single Send button)
- Enter key now sends immediately instead of queueing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 16:03:51 -07:00
Jon Saad-FalconandGitHub 6cc055f25e fix: agent Interact tab UX and immediate message handling (#137)
fix: agent Interact tab UX and immediate message handling
2026-03-26 15:10:57 -07:00
Jon Saad-FalconandClaude Opus 4.6 a212652dc0 fix: agent Interact tab UX — immediate messages, ordering, scroll
Backend:
- Immediate-mode messages now trigger a background tick so the agent
  actually processes and responds (previously they were just stored)

Frontend (Interact tab):
- Reverse message order so newest appear at bottom near the input box
- Filter out agent responses with empty content (blank bubbles)
- Add "Agent is thinking..." indicator with pulsing dot while processing
- Show timestamps instead of raw mode/status labels
- Poll for new messages every 3s so responses appear automatically
- Only auto-scroll to bottom on initial tab load, not on every poll
  update (prevents hijacking the user's scroll position)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 15:08:08 -07:00
Jon Saad-FalconandClaude Opus 4.6 3b4eeded0e fix: normalize TOML arrays for property setters, not just dataclass fields
_apply_toml_section only normalized TOML arrays to comma-separated strings
for real dataclass fields, but backward-compat property setters like
reward_weights also expect string input. When a user's config.toml had
an array value for a property-backed attribute, the raw list was passed
to the setter which called .split(",") on it, causing:

  'list' object has no attribute 'split'

This also hardens serve.py against the same issue when reading
config.agent.tools.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 13:46:09 -07:00
Jon Saad-FalconandGitHub db2f20e25b Delete instr.md 2026-03-26 12:42:54 -07:00
Jon Saad-FalconandGitHub e5ff7b3695 Delete MagicMock/load_config().security.audit_log_path directory 2026-03-26 12:42:16 -07:00
Jon Saad-FalconandGitHub 36878e0263 feat: agent testing, channel adapters, bug fixes (#89)
feat: agent testing, channel adapters, bug fixes
2026-03-26 11:29:42 -07:00
Jon Saad-FalconandClaude Opus 4.6 1358d946c1 merge: resolve conflicts with main (streaming + auto-recover)
Merge main into fix/ssrf-check, keeping both the auto-recover
logic for error-state agents and the async streaming support
for the send_message endpoint.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 11:29:33 -07:00