Ethan Mollick's SatBench: 63% of 173 AI benchmarks cited by labs are now saturated
emollick · x · 2026-11-15
Ethan Mollick released SatBench, an open dataset (CC0-1.0) that measures how quickly major AI benchmarks become saturated.
- Of 173 benchmarks cited by AI labs since 2023, 63% are now effectively beaten;
- Saturation is accelerating year over year — AI is improving faster than our measurement systems can track;
- Companion resource: The Benchmark Almanac.
Implication: leaderboards increasingly lag the frontier, so benchmark-based model comparisons are becoming less reliable.
Related event: SatBench: 63% of AI Benchmarks Cited by Labs Are Saturated(3 posts)→
More from Models
- Claude Opus 5.5 caught making a basic grammar mistake: 'Not much dramatic' — Pizzahunter3 · 2026-11-15
- Redditor proposes a 'normies benchmark': 95% task success equals good enough — AdDizzy8160 · 2026-11-15
- Gemini math lead reveals inside story: evals, post-training recipes, scaled runs — thtrieu_ · 2026-11-15
- Blogger saves $53 in 43 hours after switching from Claude Code to Gemini 4 Carbon — mazzaTalk · 2026-11-15
- SatBench: 63% of lab-cited AI benchmarks already saturated, and solving is speeding up — emollick · 2026-11-15
- LFM2.5-24B-A2B praised for local agent use: 37 tok/s on an AMD 780M iGPU — jeremyckahn · 2026-11-15