MaxxBench sparks controversy over conflating benchmark gaming with model capability
The new MaxxBench (unLAP) benchmark drew fierce criticism for conflating benchmaxing, overfitting, and inability to adapt to downstream tasks, with critics calling the research cargo-cult science. Its author responded that pass rate drops stem from overfitting rather than capability collapse.
2026-11-08 ~ 2026-11-09 · 3 related posts
- Episode 1: MaxxBench Slightly Rewrites DeepSWE Tasks, Exposing 23-Point Gaps Between Equal-Scoring Models(2026-11-08, 8 posts)
- Episode 2: MaxxBench sparks controversy over conflating benchmark gaming with model capability(2026-11-08, 3 posts)
- Episode 3: DeepSeek's New Models Falter in Benchmarks Amid Sandboxing and Alignment Concerns(2026-11-08, 2 posts)
- MaxxBench sparks backlash: conflating benchmaxing with real spec adaptation — teortaxesTex · 2026-11-08
- MaxxBench score drops traced to overfitting, not capability collapse, analysis finds — teortaxesTex · 2026-11-09
- MaxxBench criticized as cargo-cult science: overfitting confounds claimed benchmaxxing — teortaxesTex · 2026-11-09