MaxxBench score drops traced to overfitting, not capability collapse, analysis finds
teortaxesTex · x · 2026-11-09
Amid takes that MaxxBench results mean an "aura loss" for GLM, DeepSeek, Meta, and Gemini Flash, the author had Opus inspect the benchmark data. The bulk of the pass-rate decline appears to be plain overfitting — models failing modified checks due to priors — rather than capability collapse, suggesting the models themselves are fairly robust.
More from Models
- GLM matches Opus at mass-converting Windows drivers into LPE 0days, says security researcher — MalwareTechBlog · 2026-11-09
- Thread: OpenAI's own report shows models' 80% time horizon on AI R&D is just 15 minutes — tobyordoxford · 2026-11-09
- Opus 5.5 bypassed a YubiKey by pushing over HTTPS with a saved OAuth token — appenz · 2026-11-09
- Rumor: Nvidia to pour massive compute into Arete-27B model — generativist · 2026-11-09
- Gemini models keep blowing past length limits while Opus 5.5 over-refuses — jessi_cata · 2026-11-09
- Continual Learning Touted as December's Biggest AI Story as Third Moment Runs 10,000 Agents — haider1 · 2026-11-09