SatBench: AI benchmarks still arrive faster than models solve them — 7-9 new ones a month in 2026
mattbeane · x · 2026-11-15
Matt Beane contributed a "first derivatives" analysis to SatBench (forked from Ethan Mollick), which treats the 173 most-cited AI benchmarks as questions on one big exam, solved once the best reported score hits 85%. The chart shows new benchmarks keep arriving faster than AI solves them: 1.5 released vs 0.8 solved per month in 2023, 3.7 vs 1.2 in 2024, 5.0 vs 2.8 in 2025, and an estimated 7-9 vs 4.4-4.7 in 2026 (to Nov 14). The apparent 2026 reversal is only a counting artifact — benchmarks aren't counted until 10 lab launch posts name them, a months-long delay corrected via backtests (errors within 5-8%). Data comes from the Benchmark Almanac's TypeScript layer, incl. Epoch AI's Benchmarking Hub. The repo also analyzes benchmark correlations, clusters, and which benchmarks labs have stopped reporting.
More from Models
- Tesla FSD 15's real-time view isn't 3D rendering — it's the AI model's own synthetic perception — mark_k · 2026-11-22
- Gemini Black Friday half-price: JP-region trick gets Pro annual plan for $91.54 — huangyun_122 · 2026-11-22
- Microsoft's USB4 RDMA lets mini PCs pool memory to run Qwen3-235B — JFPuget · 2026-11-22
- Google AI Mode now enforces usage limits, users surprised by rate-limit popup — elvano124 · 2026-11-22
- Sol 6.2 now refuses reverse engineering that Sol 6.1 happily did for a month — cardak98 · 2026-11-22
- Anthropic's record ~$81B IPO, GPT-6.2 pricing, Qwen4-Max and Copilot Autopilot: week in review — ksraj1001 · 2026-11-22