Wire new display flags through the run command to print_full_results.
--compact renders a single dense table; --trace-detail enables full
per-step trace listing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add print_accuracy_panel, print_latency_table, print_energy_table,
print_trace_summary, print_compact_table, and print_full_results
orchestrator. Keep existing print_metrics_table for backward compat.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add total_input_tokens, total_output_tokens, avg_power_watts to
_compute_summary() and include in RunSummary constructor
- Add _output_path and _traces_dir as proper slots on RunSummary (fixes
pre-existing bug: slots=True prevented dynamic attribute setting)
- Update _summary_to_dict to serialize new fields
- Fix pre-existing test_metric_stats_to_dict to check p90/p95/p99 keys
- Add 2 new tests: total token counts and avg power in summary
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add trace_steps and trace_energy_joules to EvalResult
- Add avg_power_watts, total_input_tokens, total_output_tokens, and
trace_step_type_stats to RunSummary
- Add 3 tests covering new field defaults and setters
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Rewrite .github/workflows/desktop.yml: 2-job pipeline (validate + build-and-release)
with rolling desktop-latest pre-release on push to main and stable desktop-v* releases
- Add UpdateChecker component: checks for updates on startup + every 30 min,
background download with progress bar, one-click relaunch
- Configure Tauri updater: endpoints pointing to desktop-latest release, pubkey placeholder
- Add tauri-plugin-process for relaunch support (Cargo.toml, lib.rs, package.json)
- Add macOS Entitlements.plist for notarization (network + file access, no sandbox)
- Add scripts/bump-desktop-version.sh for atomic version bumps across 3 config files
- Add desktop/README.md with dev setup, auto-update architecture, signing docs
- Update .gitignore for desktop/node_modules, dist, target
- Configure macOS minimumSystemVersion, Windows timestampUrl
- Include all Phase 14-21 work: agent hardening, RBAC, taint tracking, workflows,
skills, knowledge graph, sessions, A2A, MCP templates, WASM sandbox, TUI dashboard,
production tools, CLI expansion, API expansion, learning productionization,
Tauri desktop app, and 10 new channels
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extend the eval framework to compute per-sample hardware efficiency
metrics when model hardware parameters are provided in TOML config:
- ModelConfig gains param_count_b, active_params_b, gpu_peak_tflops,
gpu_peak_bandwidth_gb_s, num_gpus fields
- RunConfig gains metadata dict, populated by expand_suite() from
model hardware params
- EvalRunner._process_one() computes IPW (Intelligence Per Watt),
IPJ (Intelligence Per Joule), MFU, and MBU per sample
- All telemetry fields (energy, power, GPU util, throughput, MFU, MBU,
IPW, IPJ) written to JSONL output and summary JSON
- RunSummary includes MetricStats (mean/median/min/max/std) for all
telemetry metrics plus total_energy_joules
- GLM-4.7-Flash eval config enriched with A100 SXM hardware params
- 146 eval tests pass (26 new tests for telemetry, efficiency metrics,
metadata flow, MetricStats helpers)
- User guide and API docs updated with new fields and output format
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Evaluation framework (evals/): benchmarking system for measuring accuracy
across four categories — Chat (WildChat), Reasoning (SuperGPQA), RAG (FRAMES),
and Agentic (GAIA). Two backends: jarvis-direct (engine-level) and jarvis-agent
(agent-level with tool calling), both supporting local and cloud models.
Datasets adapted from IPW, scorers include exact match, LLM letter extraction,
and LLM-as-judge. Parallel execution via ThreadPoolExecutor with incremental
JSONL output. CLI: python -m evals {run,run-all,summarize,list}. 57 tests pass.
SVG fix: center logo content within viewBox by wrapping icon+text in a
translate(90,0) group, eliminating the left-shift visible in the README.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
