Only 372 hits out of 4,000 problems: analysts probe failure rates in OpenAI's math output
basedjensen · x · 2026-10-07
Commenting on OpenAI's 722 math manuscripts, one analyst notes that 372 successful result families out of 4,000 attempted problems seems like a low hit rate — though only about three hours of thinking compute was spent per problem. He suggests it would be far more informative to see the failure cases and which families of results proved most resistant to the model, rather than only the successes.
More from Models
- 'Compression of the RH proof has begun': AI circles buzz over unverified Riemann claim — teortaxesTex · 2026-11-22
- GLM 5.3 and DeepSeek V4.1 Flash resold at 60% off: the economics of open-source giveaways — lxfater · 2026-11-22
- Baseline model test: no persona, no forced formatting, no safety bypasses observed — Sauers_ · 2026-11-22
- 2-day Gemini stress test: $4.99 Google AI Pro nearly maxed, orchestrator UI revealed — lipeng0820 · 2026-11-22
- Timed eval task asks users to deliberately make Claude fail at real work in 12 minutes — Icy-Measurement10 · 2026-11-22
- Tesla FSD 15's real-time view isn't 3D rendering — it's the AI model's own synthetic perception — mark_k · 2026-11-22