mirror of
https://github.com/open-jarvis/OpenJarvis.git
synced 2026-07-30 10:52:15 +00:00
Jon Saad-FalconandClaude Opus 4.6
2606c283e9
Introduce ResultTracker ABC with pluggable experiment tracking for the eval framework, enabling per-sample streaming to W&B and summary-row appending to Google Sheets — critical for the NeurIPS paper sweep. - ResultTracker ABC: on_run_start/on_result/on_summary/on_run_end - WandbTracker: per-sample wandb.log with sample/ prefix, flattened MetricStats in run summary, reinit=True for suite mode - SheetsTracker: summary-only (no per-sample API calls), 29-column canonical row, idempotent header, service account + ADC auth - EvalRunner: trackers wired into lifecycle, all calls try/except wrapped so tracker failures never abort an eval run - CLI: 7 new flags (--wandb-project/entity/tags/group, --sheets-id/worksheet/creds) on both jarvis eval run and python -m openjarvis.evals run - Config: tracker fields parsed from TOML [run] section, propagated through expand_suite() for suite mode - pyproject.toml: eval-wandb and eval-sheets optional dependency groups - 9 new tests (lifecycle, crash resilience, mock W&B/Sheets) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>