Jon Saad-Falcon and Claude Opus 4.6
a9518e1574
refactor: move evals/ into src/openjarvis/evals/ as proper subpackage
...
Move the standalone evals framework from the project root into the
openjarvis package. Rewrite all ~50+ import statements from 'from evals.'
to 'from openjarvis.evals.' across the package, CLI, and tests. Remove
the evals-specific pyproject.toml (no longer a standalone package).
Update ruff per-file-ignores paths and fix line-length violations
introduced by the longer import paths.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-02 21:06:08 +00:00
Jon Saad-Falcon and Claude Opus 4.6
2aebcd7d77
feat: Phase 23 — Differentiated functionalities
...
Trace-driven learning pipeline:
- TrainingDataMiner: extract SFT/routing/agent pairs from traces
- LoRATrainer: fine-tune local models from trace-derived data
- AgentConfigEvolver: rewrite agent configs from trace analysis
- LearningOrchestrator: coordinate mine→train→evolve cycle, wired into SystemBuilder
Eval framework (15 real IPW benchmarks):
- Datasets: SuperGPQA, GPQA, MMLU-Pro, MATH-500, Natural Reasoning, HLE,
SimpleQA, WildChat, IPW, GAIA, FRAMES, SWE-bench, SWEfficiency,
TerminalBench, TerminalBench Native
- Scorers: MCQ extraction, LLM-judge, exact match, structural validation
- CLI: jarvis eval list|run|compare|report
Composable abstractions:
- Recipe system: TOML composition of all 5 pillars (3 built-in recipes)
- Agent templates: 15 pre-configured TOML manifests with system prompts
- Bundled skills: 20 ready-to-use TOML skill manifests
- Operator recipes: researcher (4h), correspondent (5min), sentinel (2h)
102 files changed, ~11,500 lines added. 3241 tests pass (44 skipped).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-02 05:34:46 +00:00
