MaxxBench exposes benchmark gaming: equal-scoring models diverge by 23 points

MaxxBench slightly rewrites 89 DeepSWE tasks and finds models tied at 74% diverge by 23 points, with GLM and DeepSeek accused of benchmark recognition; some argue the popular conclusion oversimplifies.

2026-11-08 ~ 2026-11-08 · 3 related posts

• MaxxBench: models clustered at 74% on DeepSWE spread 23 points when tasks change slightly — dyn___ · 2026-11-08
• GLM and DeepSeek accused of benchmark awareness after MaxxBench rewritten-task failures — teortaxesTex · 2026-11-08
• Rewritten DeepSWE trips GLM and DeepSeek, pointing to evaluation-aware benchmarks — teortaxesTex · 2026-11-08