2 Commits

Include renames

Author	SHA1	Message	Date
Jon Saad-FalconandClaude Opus 4.6	9f5e97eed9	Add eval framework efficiency metrics: MFU, MBU, IPW, IPJ ... Extend the eval framework to compute per-sample hardware efficiency metrics when model hardware parameters are provided in TOML config: - ModelConfig gains param_count_b, active_params_b, gpu_peak_tflops, gpu_peak_bandwidth_gb_s, num_gpus fields - RunConfig gains metadata dict, populated by expand_suite() from model hardware params - EvalRunner._process_one() computes IPW (Intelligence Per Watt), IPJ (Intelligence Per Joule), MFU, and MBU per sample - All telemetry fields (energy, power, GPU util, throughput, MFU, MBU, IPW, IPJ) written to JSONL output and summary JSON - RunSummary includes MetricStats (mean/median/min/max/std) for all telemetry metrics plus total_energy_joules - GLM-4.7-Flash eval config enriched with A100 SXM hardware params - 146 eval tests pass (26 new tests for telemetry, efficiency metrics, metadata flow, MetricStats helpers) - User guide and API docs updated with new fields and output format Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-25 21:17:53 +00:00
Jon Saad-FalconandClaude Opus 4.6	323d7ff032	Add TOML config system for eval suites, pillar-aligned config, and documentation ... - Eval config: TOML-based suite configs defining models x benchmarks matrix, loaded via --config flag. Includes load_eval_config(), expand_suite(), 7 config dataclasses, 3 example configs, and 61 new tests. - Pillar-aligned config: generation params in IntelligenceConfig, nested engine/learning configs, agent objective/system_prompt/context_from_memory, structured learning sub-policies, TOML migration layer. - Documentation: evaluations user guide, evals API reference, updated mkdocs.yml navigation, updated architecture docs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-24 03:34:05 +00:00