admin/OpenJarvis
1
0
Fork 0
mirror of https://github.com/open-jarvis/OpenJarvis.git synced 2026-07-30 10:52:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
168 Commits 11 Branches 77 Tags
c504e50ccdd02f36cf2fa125b9cd6adf1126f139
Commit Graph
7 Commits
Author SHA1 Message Date
krypticmouse c504e50ccd Migrate new modules to rust, more compile time abs, remove python fallback 2026-03-06 20:37:44 -08:00
Jon Saad-FalconandClaude Opus 4.6 50ca80dfa4 merge: resolve conflicts with main (keep both benchmark sets) 
Merge origin/main into feat/savings-meter-benchmarks-comparison.
Both sides added new benchmarks to cli.py — keep all:
- Our branch: use-case benchmarks (email_triage, morning_brief, etc.)
- Main: agentic benchmarks (loghub, ama-bench, workarena, etc.)

dataset.py: keep both create_task_env/verify_requirements and iter_episodes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 04:38:26 +00:00
Jon Saad-FalconandClaude Opus 4.6 d89bbcce52 feat: upgrade eval pipeline with agentic runner, telemetry session, and savings meter 
Add 9 capabilities to match IPW pipeline:

Eval Pipeline:
- AgenticRunner for multi-turn agent execution with per-turn trace decomposition
- QueryTrace/TurnTrace data model for agentic workload telemetry
- EventRecorder for thread-safe agent event collection
- TerminalBenchTaskEnv for Docker-based task execution
- Cost computation via engine/cloud.py PRICING table
- Rich export: JSONL, HF Arrow, summary JSON, artifacts manifest
- CLI: --agentic, --concurrency, --query-timeout flags

Telemetry:
- TelemetrySession with background-sampling ring buffer (Python fallback)
- Phase metrics: prefill/decode energy split at TTFT boundary
- ITL percentile tracking (p50/p90/p95/p99)
- FLOPs estimation and MFU computation
- EnergyMonitor.snapshot() method

Rust Performance Layer:
- Ring buffer with binary search O(log n) window queries
- Trapezoidal energy integration
- Phase metrics, ITL stats, FLOPs estimation in Rust
- PyO3 bindings for all new telemetry modules (50 Rust tests)

Savings Meter & Benchmarks:
- Use-case benchmark datasets (coding, email, research, knowledge, morning brief)
- Savings dashboard component with cost comparison visualization
- Cloud cost calculator and comparison server routes
- Use-case eval configs for multiple agent/engine combinations

Tests: 80 new tests (3779 total pass, 37 skipped, 0 failures)
Lint: ruff check src/ tests/ — all checks passed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 04:22:55 +00:00
Jon Saad-FalconandClaude Opus 4.6 3040f468b6 feat: integrate Exo, Nexa, Uzu, and Apple FM inference engines 
Add four new OpenAI-compatible inference engines across the full stack:

Rust backend:
- Config structs with serde defaults (ExoEngineConfig, NexaEngineConfig,
  UzuEngineConfig, AppleFmEngineConfig) in openjarvis-core
- Factory constructors on OpenAICompatEngine (exo, nexa, uzu, apple_fm)
- Engine enum variants with delegate_engine! macro dispatch
- Discovery and resolution in get_engine_static()
- PyO3 bridge support in PyEngine

Python frontend:
- Data-driven engine class registration in openai_compat_engines.py
- Config dataclasses with backward-compat host properties
- Discovery host map entries
- DEFAULT_SEARCH_SPACE updated with new engine backends
- Apple FM shim (FastAPI wrapper for python-apple-fm-sdk, macOS only)

Config & tests:
- TOML config sections for all four engines
- Unit tests for factory methods, enum variants, and search space

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 02:00:10 +00:00
krypticmouse b03a0662bb Use Rust utils in Python SDK 2026-03-04 19:15:50 -08:00
krypticmouse 77026f6803 Add Generics, reduce dynamic dispatch, python integration 2026-03-04 18:13:45 -08:00
krypticmouse 4adfc84d11 First Migration crates 2026-03-03 19:54:00 -08:00