Why Do Benchmark Scores Rise Every Release? Reddit Debates Closed Evals
doomadah · reddit · 2026-10-01
A Reddit user asks why benchmark scores almost always go up with every model release. Some releases are clearly real improvements (e.g. the initial Fable release), but for others the community consensus is that little improved — or things got worse — yet they still bench above the previous generation: Opus 5's initial release benched above Fable, and GPT Sol 6.1 above Astra or 5.6.
The author floats two explanations: misplaced perception, or providers having a way to iterate on benchmark results without genuinely improving real-world performance — noting some benchmarks are proprietary and closed-source. The post asks anyone with insider knowledge to explain how this loop actually works.
Related event: Community Questions Reliability of AI Benchmark Scores(3 posts)→
More from Models
- AI's clever quantization move is just sensitivity analysis, an old pruning trick — code_star · 2026-11-16
- AI progress doubled since GTA VI's first trailer; GPT-4 Turbo now ranks 379th — ArtificialAnlys · 2026-11-16
- Tencent's Hy4 Flash builds a 3D game with shield mechanics in 63 minutes, beating Hy4's 90 — ShunyuYao12 · 2026-11-16
- Hy4 Flash 3D voxel demo outpaces DeepSeek-V4.1 in same-prompt comparison — ShunyuYao12 · 2026-11-16
- Yacine MTB claims training LLMs is 'easy', says he's shocked by how simple it is — yacineMTB · 2026-11-16
- MiniMax M3.1 Flash lands on OpenRouter, with Inferact as first provider — Xianbao_QIAN · 2026-11-16