VulcanBench: Grok 4.5 High Leads, Max Effort Doesn't Equal Better Accuracy
elonmusk · x · 2026-08-22
VulcanBench is a benchmark suite of 100% real engineering tasks that tests across different effort levels. The current leaderboard for Eval Suite 3 shows Grok 4.5 High at the top, followed closely by Fable 5 Low. The author reveals that running models at Max effort often doesn't buy more accuracy but simply costs more and consumes more tokens, taking longer to process.
Related event: Grok 4.5 Tops VulcanBench Real-World Engineering Benchmark(2 posts)→
More from Models
- Nathan Lambert slams Meta's Muse Open License as 'trash' — natolambert · 2026-10-16
- Google launches Gemini 4 Argon to Ultra and Pro subscribers, touting physics and 3D skills — ShikharMurty · 2026-10-16
- Google gives X users early access to test Argon, not just companies — lyraxana · 2026-10-16
- Grok Voice 3.0 Tops Design Arena Speech Leaderboard, Up 7.4 Points — XFreeze · 2026-10-16
- Robotics LLMs tested on 16 tasks: GPT-6 Astra hits 17.2% success vs Fable's 9.1% — pabbeel · 2026-10-16
- Meta open-sources Muse Spark: top US open model, scores just below DeepSeek — bindureddy · 2026-10-16