mirror of
https://github.com/open-jarvis/OpenJarvis.git
synced 2026-07-30 10:52:15 +00:00
Jon Saad-FalconandClaude Opus 4.6
9f5e97eed9
Extend the eval framework to compute per-sample hardware efficiency metrics when model hardware parameters are provided in TOML config: - ModelConfig gains param_count_b, active_params_b, gpu_peak_tflops, gpu_peak_bandwidth_gb_s, num_gpus fields - RunConfig gains metadata dict, populated by expand_suite() from model hardware params - EvalRunner._process_one() computes IPW (Intelligence Per Watt), IPJ (Intelligence Per Joule), MFU, and MBU per sample - All telemetry fields (energy, power, GPU util, throughput, MFU, MBU, IPW, IPJ) written to JSONL output and summary JSON - RunSummary includes MetricStats (mean/median/min/max/std) for all telemetry metrics plus total_energy_joules - GLM-4.7-Flash eval config enriched with A100 SXM hardware params - 146 eval tests pass (26 new tests for telemetry, efficiency metrics, metadata flow, MetricStats helpers) - User guide and API docs updated with new fields and output format Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>