No benchmark is hard anymore: Fable 5.2 tops SimpleQA, arbitrary jumps ahead
teortaxesTex · x · 2026-11-15
Andrew Curran notes new Fable 5.2 dominates Epoch's SimpleQA Verified and expects two big signals next week: Anthropic's internal progress report (with updated RSI numbers) and OpenAI's GPT-6.2 Astra, which must answer Fable both on benchmarks and on the softer gap of people enjoying working with it. teortaxesTex agrees on the big picture but says he no longer cares about any specific "hard" benchmark—SimpleQA, CritPT, robotics, painting—no defined hill is hard to climb anymore, and arbitrary jumps could come in any domain, in any order. The attached chart shows SimpleQA's question category distribution (Science and tech 858, Politics 709, Art 550, etc.).
Related event: Fable 5.2 tops SimpleQA leaderboard(2 posts)→
More from AGI Musings
- What Happens When AI Beats Most People at Most Work? — IagoInTheLight · 2026-11-15
- Fed survey: AI-adopting firms' revenue growth outlook net gap hits 48.2 pts, 2x+ non-adopters — AccBalanced · 2026-11-15
- AI Circles Lament Nick Land Posting Through Takeoff While Bostrom and Omohundro Stay Silent — abhiadesai · 2026-11-15
- Where the check stops: anchors, holders, and versioned oversight at the human-agent boundary — open_is_free · 2026-11-15
- Can AI invent a new competitive game genre in 4 hours? 18 prototypes so far — gabriel1 · 2026-11-15
- Rohit Krishnan: defining the messiness AI can't do would let us hillclimb it away — BethMayBarnes · 2026-11-15