Reddit builds benchmark release-date timeline showing evals go stale after 3-4 months
Express_Quail_1493 · reddit · 2026-10-28
A Reddit user compiled release dates for benchmarks on Artificial Analysis to combat benchmaxxing. Key takeaway: a benchmark becomes unreliable about 3-4 months after release as models overfit to it, and owners usually respond by shipping newer versions.
Recent evals in the timeline include Code2Video-AA and CritPt v2 (Oct 26, 2026), Cognition's FrontierCode 2.0 (Oct 27), AA-PersonalityBench and HealthBench Professional (Oct 19), AA-3D (Oct 12), Harvey LAB-AA v1.1 (Oct 8, adding hallucination checks), HLE-Diamond (Sep 22), GDPval-AA v2.1 (Sep 19). Older ones include Terminal-Bench 4.0, AA-Omniscience (Nov 2025), IFBench (Jul 2025), and Terminal-Bench Hard.
Bottom line: check a benchmark's age before trusting leaderboard results; version iteration is currently the only countermeasure.
More from Models
- Grok 4.7 Sweeps Top 3 on VulcanBench Frontier v4, Beating Opus 5.5 and Sonnet 5.5 — XFreeze · 2026-10-28
- Rumors: OpenAI and Anthropic internal models gain reasoning via RL in games and simulated environments — imjustnewatai · 2026-10-28
- Users notice ChatGPT almost always ends replies with "One question:" — Angaisb_ · 2026-10-28
- Community wishlist for next Gemma: better multilingual, OCR, fewer refusals — MaziyarPanahi · 2026-10-28
- Mystery model Space Bunny revealed as early MiniMax M3.1 Flash, free on EvoX Agent Oct 28–30 — lxfater · 2026-10-28
- Accusations fly that Artificial Analysis rankings systematically underrate open-weight models — aiamblichus · 2026-10-28