Commit Graph
5 Commits
Author SHA1 Message Date
0cac61d3bb docs(evals): rewrite evaluations guide to match the real CLI surface; add openjarvis-eval alias (#534)
docs/user-guide/evaluations.md documented a standalone "openjarvis-evals"
package, a "uv sync --extra eval" install, and an "openjarvis-eval" console
script — a layout from commit bd493832 that was never an ancestor of main.
Rewrite the page against the real surface (jarvis eval / python -m
openjarvis.evals), document all 40 registered benchmark keys and 4 backends,
fix the judge-model default, correct run-all semantics, and split the option
reference into the jarvis-eval subset and the module CLI's research-only
options.

Add the one-line [project.scripts] alias
openjarvis-eval = "openjarvis.evals.cli:main" (the click group the module
CLI already dispatches to) so the long-documented command name works again.

The page was a complete orphan: add it (and the equally orphaned
benchmarks.md) to the mkdocs nav and link it from docs/index.md.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 13:50:17 -07:00
Jon Saad-FalconandGitHub eacf34e500 spec_search: drop CLI stubs, rename, and ship paper-aligned tutorial + configs (#346) 2026-05-15 14:17:23 -07:00
Jon Saad-FalconandGitHub 44815e8544 feat(evals): add TerminalBench V2.1 benchmark (#345) 2026-05-14 20:23:14 -07:00
Jon Saad-FalconandGitHub f5695845b7 feat(distillation): M1→M2→M3 spec-level distillation pipeline + hill-climb optimizer (#273) 2026-04-20 19:18:10 -07:00
Jon Saad-Falconandkrypticmouse 8798e2ee4f init commit 2026-03-12 17:29:39 +00:00