admin/OpenJarvis
1
0
Fork 0
mirror of https://github.com/open-jarvis/OpenJarvis.git synced 2026-07-31 03:12:16 +00:00
Code Issues Packages Projects Releases Wiki Activity
37 Commits 11 Branches 77 Tags
2aebcd7d770d3daa4da61e1ddcf9fb6c9fe83e69
Commit Graph
6 Commits
Author SHA1 Message Date
Jon Saad-FalconandClaude Opus 4.6 2aebcd7d77 feat: Phase 23 — Differentiated functionalities 
Trace-driven learning pipeline:
- TrainingDataMiner: extract SFT/routing/agent pairs from traces
- LoRATrainer: fine-tune local models from trace-derived data
- AgentConfigEvolver: rewrite agent configs from trace analysis
- LearningOrchestrator: coordinate mine→train→evolve cycle, wired into SystemBuilder

Eval framework (15 real IPW benchmarks):
- Datasets: SuperGPQA, GPQA, MMLU-Pro, MATH-500, Natural Reasoning, HLE,
  SimpleQA, WildChat, IPW, GAIA, FRAMES, SWE-bench, SWEfficiency,
  TerminalBench, TerminalBench Native
- Scorers: MCQ extraction, LLM-judge, exact match, structural validation
- CLI: jarvis eval list|run|compare|report

Composable abstractions:
- Recipe system: TOML composition of all 5 pillars (3 built-in recipes)
- Agent templates: 15 pre-configured TOML manifests with system prompts
- Bundled skills: 20 ready-to-use TOML skill manifests
- Operator recipes: researcher (4h), correspondent (5min), sentinel (2h)

102 files changed, ~11,500 lines added. 3241 tests pass (44 skipped).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-02 05:34:46 +00:00
Jon Saad-FalconandClaude Opus 4.6 24972e3e52 Add Phase 12+13: energy measurement, install polish, PWA, cross-hardware 
Phase 12 — Energy Measurement Upgrade:
- EnergyMonitor ABC with multi-vendor support (NVIDIA hw counters,
  AMD amdsmi, Apple zeus-ml, CPU RAPL sysfs)
- EnergyBatch batch-level energy-per-token accounting
- SteadyStateDetector CV-based thermal equilibrium detection
- EnergyBenchmark with warmup phase
- InstrumentedEngine prefers EnergyMonitor over legacy GpuMonitor
- Telemetry store/aggregator extended with energy fields

Phase 13 — Install, Hosting, Cross-Hardware:
- jarvis doctor diagnostic command (8 checks, --json output)
- jarvis init post-setup guidance with engine-specific next steps
- README Quick Start section
- MLX engine backend (Apple Silicon → mlx recommendation)
- AMD VRAM/multi-GPU detection via rocm-smi
- PyTorch MPS device selection in orchestrator trainers
- PWA support (vite-plugin-pwa, service worker, manifest, icons)
- Server static file serving fix for PWA files
- Dockerfile.gpu.rocm + docker-compose.gpu.rocm.yml for ROCm
- Eval framework display module and efficiency metrics

2244 tests pass, 37 skipped.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-26 20:09:07 +00:00
Jon Saad-FalconandClaude Opus 4.6 9f5e97eed9 Add eval framework efficiency metrics: MFU, MBU, IPW, IPJ 
Extend the eval framework to compute per-sample hardware efficiency
metrics when model hardware parameters are provided in TOML config:

- ModelConfig gains param_count_b, active_params_b, gpu_peak_tflops,
  gpu_peak_bandwidth_gb_s, num_gpus fields
- RunConfig gains metadata dict, populated by expand_suite() from
  model hardware params
- EvalRunner._process_one() computes IPW (Intelligence Per Watt),
  IPJ (Intelligence Per Joule), MFU, and MBU per sample
- All telemetry fields (energy, power, GPU util, throughput, MFU, MBU,
  IPW, IPJ) written to JSONL output and summary JSON
- RunSummary includes MetricStats (mean/median/min/max/std) for all
  telemetry metrics plus total_energy_joules
- GLM-4.7-Flash eval config enriched with A100 SXM hardware params
- 146 eval tests pass (26 new tests for telemetry, efficiency metrics,
  metadata flow, MetricStats helpers)
- User guide and API docs updated with new fields and output format

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-25 21:17:53 +00:00
Jon Saad-FalconandClaude Opus 4.6 9ec402ab4a Enrich agent tool awareness, normalize engine tool_calls, add telemetry and eval config 
- Add build_tool_descriptions() shared builder for enriched agent system
  prompts (NativeReAct, NativeOpenHands, RLM, Orchestrator structured mode)
- Normalize tool_calls to flat {id, name, arguments} across CloudEngine
  (OpenAI/Anthropic/Google), LiteLLM, and Ollama
- Add Anthropic tool_use extraction and input_schema conversion
- Add Google function_call extraction
- Make ReAct/OpenHands parsing case-insensitive
- Add telemetry efficiency, GPU monitor, and vLLM metrics modules
- Add TOML-based eval suite config system
- Update documentation and changelog

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-25 01:58:18 +00:00
Jon Saad-FalconandClaude Opus 4.6 323d7ff032 Add TOML config system for eval suites, pillar-aligned config, and documentation 
- Eval config: TOML-based suite configs defining models x benchmarks matrix,
  loaded via --config flag. Includes load_eval_config(), expand_suite(),
  7 config dataclasses, 3 example configs, and 61 new tests.
- Pillar-aligned config: generation params in IntelligenceConfig, nested
  engine/learning configs, agent objective/system_prompt/context_from_memory,
  structured learning sub-policies, TOML migration layer.
- Documentation: evaluations user guide, evals API reference, updated
  mkdocs.yml navigation, updated architecture docs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-24 03:34:05 +00:00
Jon Saad-FalconandClaude Opus 4.6 bd49383201 Add evaluation framework and center README logo SVGs 
Evaluation framework (evals/): benchmarking system for measuring accuracy
across four categories — Chat (WildChat), Reasoning (SuperGPQA), RAG (FRAMES),
and Agentic (GAIA). Two backends: jarvis-direct (engine-level) and jarvis-agent
(agent-level with tool calling), both supporting local and cloud models.
Datasets adapted from IPW, scorers include exact match, LLM letter extraction,
and LLM-as-judge. Parallel execution via ThreadPoolExecutor with incremental
JSONL output. CLI: python -m evals {run,run-all,summarize,list}. 57 tests pass.

SVG fix: center logo content within viewBox by wrapping icon+text in a
translate(90,0) group, eliminating the left-shift visible in the README.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 23:48:43 +00:00