Jon Saad-Falcon and Claude Opus 4.6
86869e1e27
feat(rust): update PyO3 bindings, Python bridge, and Rust CI
...
- Add PyO3 wrappers for NativeOpenHandsAgent, MonitorOperativeAgent
- Add PyO3 wrappers for FAISSMemory, ColBERTMemory, HybridMemory
- Add native engine variants (vllm_native, sglang_native, llamacpp_native)
- Add PyO3 wrappers for OptimizationStore and LLMOptimizer
- Update _rust_bridge.py with JSON converters for new types
- Add Rust CI job (clippy + test) to GitHub Actions workflow
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-07 01:08:37 +00:00
Jon Saad-Falcon and Claude Opus 4.6
50ca80dfa4
merge: resolve conflicts with main (keep both benchmark sets)
...
Merge origin/main into feat/savings-meter-benchmarks-comparison.
Both sides added new benchmarks to cli.py — keep all:
- Our branch: use-case benchmarks (email_triage, morning_brief, etc.)
- Main: agentic benchmarks (loghub, ama-bench, workarena, etc.)
dataset.py: keep both create_task_env/verify_requirements and iter_episodes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-06 04:38:26 +00:00
Jon Saad-Falcon and Claude Opus 4.6
d89bbcce52
feat: upgrade eval pipeline with agentic runner, telemetry session, and savings meter
...
Add 9 capabilities to match IPW pipeline:
Eval Pipeline:
- AgenticRunner for multi-turn agent execution with per-turn trace decomposition
- QueryTrace/TurnTrace data model for agentic workload telemetry
- EventRecorder for thread-safe agent event collection
- TerminalBenchTaskEnv for Docker-based task execution
- Cost computation via engine/cloud.py PRICING table
- Rich export: JSONL, HF Arrow, summary JSON, artifacts manifest
- CLI: --agentic, --concurrency, --query-timeout flags
Telemetry:
- TelemetrySession with background-sampling ring buffer (Python fallback)
- Phase metrics: prefill/decode energy split at TTFT boundary
- ITL percentile tracking (p50/p90/p95/p99)
- FLOPs estimation and MFU computation
- EnergyMonitor.snapshot() method
Rust Performance Layer:
- Ring buffer with binary search O(log n) window queries
- Trapezoidal energy integration
- Phase metrics, ITL stats, FLOPs estimation in Rust
- PyO3 bindings for all new telemetry modules (50 Rust tests)
Savings Meter & Benchmarks:
- Use-case benchmark datasets (coding, email, research, knowledge, morning brief)
- Savings dashboard component with cost comparison visualization
- Cloud cost calculator and comparison server routes
- Use-case eval configs for multiple agent/engine combinations
Tests: 80 new tests (3779 total pass, 37 skipped, 0 failures)
Lint: ruff check src/ tests/ — all checks passed
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-06 04:22:55 +00:00
Jon Saad-Falcon and Claude Opus 4.6
3040f468b6
feat: integrate Exo, Nexa, Uzu, and Apple FM inference engines
...
Add four new OpenAI-compatible inference engines across the full stack:
Rust backend:
- Config structs with serde defaults (ExoEngineConfig, NexaEngineConfig,
UzuEngineConfig, AppleFmEngineConfig) in openjarvis-core
- Factory constructors on OpenAICompatEngine (exo, nexa, uzu, apple_fm)
- Engine enum variants with delegate_engine! macro dispatch
- Discovery and resolution in get_engine_static()
- PyO3 bridge support in PyEngine
Python frontend:
- Data-driven engine class registration in openai_compat_engines.py
- Config dataclasses with backward-compat host properties
- Discovery host map entries
- DEFAULT_SEARCH_SPACE updated with new engine backends
- Apple FM shim (FastAPI wrapper for python-apple-fm-sdk, macOS only)
Config & tests:
- TOML config sections for all four engines
- Unit tests for factory methods, enum variants, and search space
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-06 02:00:10 +00:00
krypticmouse
b03a0662bb
Use Rust utils in Python SDK
2026-03-04 19:15:50 -08:00
krypticmouse
77026f6803
Add Generics, reduce dynamic dispatch, python integration
2026-03-04 18:13:45 -08:00
krypticmouse
4adfc84d11
First Migration crates
2026-03-03 19:54:00 -08:00
