Add the orchestration layer that ties together the LLM optimizer, trial
runner, and persistence into a propose-evaluate-analyze loop with early
stopping and recipe export. 36 new tests pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement LLMOptimizer that uses a cloud LLM to propose and analyze
OpenJarvis configurations, inspired by DSPy's GEPA approach with
textual trace feedback rather than just scalar rewards. 46 tests pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add TrialRunner to bridge TrialConfig to the eval framework (EvalRunner),
TraceJudge for LLM-as-judge scoring of agent traces, and FeedbackCollector
for aggregating explicit, thumbs, and judge-driven feedback signals.
68 tests pass covering all three modules.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Synthesize reusable benchmarks from interaction traces with feedback
scores. Mines high-quality traces, groups by query class, picks the
best reference per class, and exposes results through DatasetProvider
and LLM-judge Scorer so EvalRunner can evaluate against personal
workflow patterns.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add the optimize module with core data types (SearchDimension, SearchSpace,
TrialConfig, TrialResult, OptimizationRun) and search space builder
(build_search_space, DEFAULT_SEARCH_SPACE) covering all 5 pillars. Includes
TrialConfig.to_recipe() mapping and SearchSpace.to_prompt_description()
for LLM-readable rendering. 66 tests pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduce ResultTracker ABC with pluggable experiment tracking for the
eval framework, enabling per-sample streaming to W&B and summary-row
appending to Google Sheets — critical for the NeurIPS paper sweep.
- ResultTracker ABC: on_run_start/on_result/on_summary/on_run_end
- WandbTracker: per-sample wandb.log with sample/ prefix, flattened
MetricStats in run summary, reinit=True for suite mode
- SheetsTracker: summary-only (no per-sample API calls), 29-column
canonical row, idempotent header, service account + ADC auth
- EvalRunner: trackers wired into lifecycle, all calls try/except
wrapped so tracker failures never abort an eval run
- CLI: 7 new flags (--wandb-project/entity/tags/group,
--sheets-id/worksheet/creds) on both jarvis eval run and
python -m openjarvis.evals run
- Config: tracker fields parsed from TOML [run] section, propagated
through expand_suite() for suite mode
- pyproject.toml: eval-wandb and eval-sheets optional dependency groups
- 9 new tests (lifecycle, crash resilience, mock W&B/Sheets)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Define the abstract interface for the speech subsystem: SpeechBackend ABC
with transcribe/health/supported_formats methods, TranscriptionResult and
Segment dataclasses for structured transcription output.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add SpeechConfig dataclass with backend, model, language, device, and
compute_type fields. Wire it into JarvisConfig and load_config() so
[speech] TOML sections are loaded automatically.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace vllm.py, sglang.py, llamacpp.py, mlx.py, lmstudio.py with
openai_compat_engines.py using dynamic class creation and registration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Delete the entire src/openjarvis/memory/ directory (11 files) which
was a pure re-export shim pointing to openjarvis.tools.storage.*.
Update all imports across src/ and tests/ to use the canonical
openjarvis.tools.storage imports directly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move the standalone evals framework from the project root into the
openjarvis package. Rewrite all ~50+ import statements from 'from evals.'
to 'from openjarvis.evals.' across the package, CLI, and tests. Remove
the evals-specific pyproject.toml (no longer a standalone package).
Update ruff per-file-ignores paths and fix line-length violations
introduced by the longer import paths.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add std_latency to LatencyBenchmark metrics. New _render_stats_table
in bench_cmd.py detects stats-pattern keys (mean_X, p50_X, min_X,
max_X, std_X, p95_X) and renders Avg/Median/Min/Max/Std/P95 columns.
Falls back to simple key-value for non-stats metrics.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New log_config.py with setup_logging() configuring the openjarvis
logger (WARNING default, DEBUG on --verbose, ERROR on --quiet).
RotatingFileHandler (5MB, 3 backups) enabled in verbose mode.
Flags added to root CLI group and forwarded via click context.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New hints.py with hint_no_config(), hint_no_engine(), hint_no_model().
Wire hint_no_engine into ask.py EngineConnectionError handlers to
show actionable suggestions when engine is unreachable.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
5-step guided setup: detect hardware, write config, check engine,
verify model, run test query. Skips config step if already present
unless --force is used. Exits with helpful message on engine failure.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add StepTypeStats dataclass with full descriptive statistics per step
type (count, avg/median/min/max/std duration, total energy, and
avg/median/min/max/std input/output tokens)
- Add total_energy_joules, total_generate_energy_joules, and
step_type_stats fields to TraceSummary
- Compute per-step-type aggregations in summary() method
- Add 3 tests covering energy totals, step-type stats, and dataclass fields
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add tokens_per_joule column to TelemetryStore schema, INSERT, and migration
- Add avg_tokens_per_joule field to ModelStats and EngineStats
- Add AVG(tokens_per_joule) to per_model_stats() and per_engine_stats() SQL
queries using _safe_col() pattern for backward compatibility
- Add 3 tests for store/retrieve, multi-record aggregation, and engine stats
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Computes completion_tokens / energy_joules in both generate() and
stream() methods. Zero when energy or tokens are zero.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduces the Operator subsystem — persistent agents that run on a schedule
with automatic state management. Built on a new OperativeAgent (extends
ToolUsingAgent) with built-in session persistence, memory state recall, and
tick-aware lifecycle. Includes OperatorManager for lifecycle management,
TOML manifest format, 4 bundled operators, 9 CLI commands, and 39 tests.
New: OperativeAgent, operators package (types/loader/manager),
operators CLI, OperatorsConfig, OPERATOR_TICK_START/END events.
Modified: system.py (system_prompt/operator_id passthrough),
scheduler.py (operator metadata forwarding), config.py, events.py.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Rewrite .github/workflows/desktop.yml: 2-job pipeline (validate + build-and-release)
with rolling desktop-latest pre-release on push to main and stable desktop-v* releases
- Add UpdateChecker component: checks for updates on startup + every 30 min,
background download with progress bar, one-click relaunch
- Configure Tauri updater: endpoints pointing to desktop-latest release, pubkey placeholder
- Add tauri-plugin-process for relaunch support (Cargo.toml, lib.rs, package.json)
- Add macOS Entitlements.plist for notarization (network + file access, no sandbox)
- Add scripts/bump-desktop-version.sh for atomic version bumps across 3 config files
- Add desktop/README.md with dev setup, auto-update architecture, signing docs
- Update .gitignore for desktop/node_modules, dist, target
- Configure macOS minimumSystemVersion, Windows timestampUrl
- Include all Phase 14-21 work: agent hardening, RBAC, taint tracking, workflows,
skills, knowledge graph, sessions, A2A, MCP templates, WASM sandbox, TUI dashboard,
production tools, CLI expansion, API expansion, learning productionization,
Tauri desktop app, and 10 new channels
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Pull shared boilerplate (event emission, message building, generation,
think-tag stripping) into BaseAgent concrete helpers and ToolUsingAgent
intermediate base class with tool-call loop
- Add NativeReActAgent and NativeOpenHandsAgent as clean implementations
built on the new base classes
- Simplify SimpleAgent, OrchestratorAgent, ReActAgent, OpenHandsAgent,
and RLMAgent to use inherited helpers instead of duplicated logic
- Remove CustomAgent (superseded by BaseAgent subclassing)
- Add backward-compat tests, base agent tests, native agent tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove --router CLI flag from `jarvis ask`; use config fallback chain
(default_model -> first engine model -> fallback_model) instead
- Simplify SDK _resolve_model() to match the same fallback chain
- Reclassify OrchestratorSFTPolicy and OrchestratorGRPOPolicy as
IntelligenceLearningPolicy (they fine-tune LM weights, not agent logic)
- Add MCPServer.get_tools() for SystemBuilder integration
- Refactor SystemBuilder._resolve_tools() to discover tools via MCPServer
with dependency injection, replacing manual per-tool-name imports
- Update CLI router tests to test the new fallback chain
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tests: GuardrailsEngine.stream() async tests, OpenClawChannelBridge listener_loop
tests, channel CLI command tests, FileReadTool sensitive file blocking, ingest_path
sensitive file filtering, SecurityConfig/ChannelConfig config tests. Documentation:
new user guides, architecture pages, and API references for Security and Channels
modules; updated CLAUDE.md, configuration docs, CLI reference, and mkdocs.yml nav.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add URL detection, normalization, and direct fetching to web_search tool
(handles standalone URLs, embedded URLs, arxiv PDF→abs conversion)
- Add URL pre-expansion in OpenHands agent to bypass tool loop for URL queries
- Add input truncation (_truncate_if_needed) to prevent context window overflow
- Fix stream_bridge ToolResult field access (content/latency_seconds vs output/latency_ms)
- Add XML tool call format parser alongside Action:/Action Input: format
- Reduce default max_turns from 10→3 to cap worst-case response time
- Add empty response fallback text in frontend useChat hook
- Add channels system, security module, and model catalog updates
- Add 30+ new tests for URL fetching, truncation, and agent behavior
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ReAct and OpenHands agents, WebSearch and CodeInterpreter tools,
full MCP protocol layer (server/client/transport), Gemini cloud engine
support, 12 new model specs (4 local MoE + 8 cloud), trace system,
and comprehensive test coverage across all dimensions (hardware, engine,
memory, agents, tools, MCP, integration).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use importlib.import_module + patch.object to fix 19 broken tests
(module shadowing issue in cli/__init__.py)
- Add 400 retry without tools in _openai_compat.py for vLLM servers
without --enable-auto-tool-choice
- Update NOTES.md with session 2 findings
🤖 Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <noreply@cognition.ai>
