Extend the eval framework to compute per-sample hardware efficiency
metrics when model hardware parameters are provided in TOML config:
- ModelConfig gains param_count_b, active_params_b, gpu_peak_tflops,
gpu_peak_bandwidth_gb_s, num_gpus fields
- RunConfig gains metadata dict, populated by expand_suite() from
model hardware params
- EvalRunner._process_one() computes IPW (Intelligence Per Watt),
IPJ (Intelligence Per Joule), MFU, and MBU per sample
- All telemetry fields (energy, power, GPU util, throughput, MFU, MBU,
IPW, IPJ) written to JSONL output and summary JSON
- RunSummary includes MetricStats (mean/median/min/max/std) for all
telemetry metrics plus total_energy_joules
- GLM-4.7-Flash eval config enriched with A100 SXM hardware params
- 146 eval tests pass (26 new tests for telemetry, efficiency metrics,
metadata flow, MetricStats helpers)
- User guide and API docs updated with new fields and output format
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rewrites the main docs/index.md to reflect the current five-pillar
structure (Intelligence, Agents, Tools, Engine, Learning) with
accurate descriptions of each. Updates project status to v1.5
Phase 10 complete, seven agent types, 1800+ tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The openclaw_protocol, openclaw_transport, openclaw_plugin, and
openclaw_bridge modules are not installed in CI. Replace their
mkdocstrings auto-doc directives with note admonitions linking
to the architecture docs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Reflects the new agent hierarchy (BaseAgent helpers, ToolUsingAgent
intermediate base, NativeReActAgent, NativeOpenHandsAgent, RLMAgent,
OpenHandsAgent SDK) across all architecture, user-guide, extending,
contributing, roadmap, and API reference pages. Removes CustomAgent
references and adds Mermaid diagrams for new agent types.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Pull shared boilerplate (event emission, message building, generation,
think-tag stripping) into BaseAgent concrete helpers and ToolUsingAgent
intermediate base class with tool-call loop
- Add NativeReActAgent and NativeOpenHandsAgent as clean implementations
built on the new base classes
- Simplify SimpleAgent, OrchestratorAgent, ReActAgent, OpenHandsAgent,
and RLMAgent to use inherited helpers instead of duplicated logic
- Remove CustomAgent (superseded by BaseAgent subclassing)
- Add backward-compat tests, base agent tests, native agent tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove --router CLI flag from `jarvis ask`; use config fallback chain
(default_model -> first engine model -> fallback_model) instead
- Simplify SDK _resolve_model() to match the same fallback chain
- Reclassify OrchestratorSFTPolicy and OrchestratorGRPOPolicy as
IntelligenceLearningPolicy (they fine-tune LM weights, not agent logic)
- Add MCPServer.get_tools() for SystemBuilder integration
- Refactor SystemBuilder._resolve_tools() to discover tools via MCPServer
with dependency injection, replacing manual per-tool-name imports
- Update CLI router tests to test the new fallback chain
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Evaluation framework (evals/): benchmarking system for measuring accuracy
across four categories — Chat (WildChat), Reasoning (SuperGPQA), RAG (FRAMES),
and Agentic (GAIA). Two backends: jarvis-direct (engine-level) and jarvis-agent
(agent-level with tool calling), both supporting local and cloud models.
Datasets adapted from IPW, scorers include exact match, LLM letter extraction,
and LLM-as-judge. Parallel execution via ThreadPoolExecutor with incremental
JSONL output. CLI: python -m evals {run,run-all,summarize,list}. 57 tests pass.
SVG fix: center logo content within viewBox by wrapping icon+text in a
translate(90,0) group, eliminating the left-shift visible in the README.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tests: GuardrailsEngine.stream() async tests, OpenClawChannelBridge listener_loop
tests, channel CLI command tests, FileReadTool sensitive file blocking, ingest_path
sensitive file filtering, SecurityConfig/ChannelConfig config tests. Documentation:
new user guides, architecture pages, and API references for Security and Channels
modules; updated CLAUDE.md, configuration docs, CLI reference, and mkdocs.yml nav.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add URL detection, normalization, and direct fetching to web_search tool
(handles standalone URLs, embedded URLs, arxiv PDF→abs conversion)
- Add URL pre-expansion in OpenHands agent to bypass tool loop for URL queries
- Add input truncation (_truncate_if_needed) to prevent context window overflow
- Fix stream_bridge ToolResult field access (content/latency_seconds vs output/latency_ms)
- Add XML tool call format parser alongside Action:/Action Input: format
- Reduce default max_turns from 10→3 to cap worst-case response time
- Add empty response fallback text in frontend useChat hook
- Add channels system, security module, and model catalog updates
- Add 30+ new tests for URL fetching, truncation, and agent behavior
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add React/TypeScript Chat UI frontend with sidebar, conversation management,
model selector, streaming indicator, and savings panel
- Wire up TelemetryStore in `jarvis serve` so TELEMETRY_RECORD events are
persisted to SQLite (matching the pattern from `jarvis ask`)
- Switch agents (orchestrator, react, openhands, openclaw_plugin) and the
direct route handler to use `instrumented_generate()` so token counts are
recorded and the /v1/savings endpoint returns real data
- Restyle savings panel model display with bordered card boxes (green accent
for local, orange for cloud) instead of plain text rows
- Add SGLang engine backend, dashboard routes, savings computation, and
agent-stream bridge for SSE streaming
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sets up a complete documentation website with 7 navigable sections (Home, Getting
Started, User Guide, Architecture, API Reference, Deployment, Development), light/dark
mode, search, code copy, and Mermaid diagram support. API reference pages use
mkdocstrings to auto-generate docs from source docstrings. GitHub Actions workflow
deploys to GitHub Pages on push to main.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ReAct and OpenHands agents, WebSearch and CodeInterpreter tools,
full MCP protocol layer (server/client/transport), Gemini cloud engine
support, 12 new model specs (4 local MoE + 8 cloud), trace system,
and comprehensive test coverage across all dimensions (hardware, engine,
memory, agents, tools, MCP, integration).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use importlib.import_module + patch.object to fix 19 broken tests
(module shadowing issue in cli/__init__.py)
- Add 400 retry without tools in _openai_compat.py for vLLM servers
without --enable-auto-tool-choice
- Update NOTES.md with session 2 findings
🤖 Generated with [Devin](https://cli.devin.ai/docs)
Co-Authored-By: Devin <noreply@cognition.ai>
Establishes OpenJarvis as a modular AI assistant backend with five
composable pillars (Intelligence, Memory, Agents, Inference, Learning).
Documents the architecture, supported backends, and phased development plan.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
