SatBench: 63% of lab-cited AI benchmarks already saturated, and solving is speeding up
emollick · x · 2026-11-15
Ethan Mollick built SatBench, a "benchmark of benchmarks" tracking how fast AI benchmarks cited by major labs since 2023 get saturated (best score reaching 85%).
Key findings:
- 109 of 173 lab-cited benchmarks (63%) are now solved as of Nov 2026, and saturation is accelerating each year
- The entire 2023 cohort is saturated; median time to solve a 2025 benchmark is 14 months vs 20 for 2023/24
- Share solved within a year of release: 17% (2023) → 30% (2024) → 43% (2025); 2026 benchmarks hit 32% within 6 months
- OpenAI has saturated the most tests (20); Anthropic leads in 2026 (11)
- Top holdouts: Humanity's Last Exam (74%), MMMU-Pro, SciCode, IFBench
- Extrapolation: today's set fully saturates 2028; with new benchmarks still arriving, 2030
Bottom line: AI is improving faster than our measurement systems can keep up.
Related event: SatBench Tracks AI Benchmark Saturation: 63% of 173 Tests Maxed Out(2 posts)→
More from Models
- Gemini math lead reveals inside story: evals, post-training recipes, scaled runs — thtrieu_ · 2026-11-15
- Blogger saves $53 in 43 hours after switching from Claude Code to Gemini 4 Carbon — mazzaTalk · 2026-11-15
- LFM2.5-24B-A2B praised for local agent use: 37 tok/s on an AMD 780M iGPU — jeremyckahn · 2026-11-15
- Will Depue's Blackwell 'forensic evidence' claim on DeepSeek hits snag: V4.1-Flash not cracked — teortaxesTex · 2026-11-15
- nanoGPT Speedrun Gains Mostly From Environment Design; Newer Models Beat Human Record — eliebakouch · 2026-11-15
- Scott Reed: robotaxi is amazing, but pedestrian detection still needs work — scott_e_reed · 2026-11-15