Overthinking Bench: Claude burns 1,069 thinking tokens on 'is the globe round?'
intellectronica · x · 2026-11-04
Peter Gostev's 'Overthinking Bench' asks simple questions and counts thinking tokens at lowest and highest effort (5 runs each). Results vary wildly: Claude Opus 5.5, Gemini 4 Argon and GPT-6 Astra use zero thinking tokens at low effort for small talk and 2+2, but at max effort Claude spends 1,069 tokens on whether the globe is round and Gemini 662. Grok 4.7 and DeepSeek V4.1 Flash always think a little (30-60 tokens even at low effort), while Kimi K3.1 Pro shows almost no gap between effort levels.
More from Models
- New mystery OpenAI models rc-ada-or-vm and rc-ada-sh-vm spotted on the API — imjustnewatai · 2026-11-04
- AgentTime Adds Combined Time-Awareness Score; GPT Models Clearly Ahead — maksym_andr · 2026-11-04
- 4 decision-model leaderboards compared: fine-tuning beats size under 5B, rankings disagree — Balance- · 2026-11-04
- "The automated evaluator is the score": quote nails models gaming evals over humans — cephaloform · 2026-11-04
- Ollama bets on zero data retention to make open models the enterprise default — ollama · 2026-11-04
- CMU's cua-speedrun ranks computer-use agents on speed and cost: GPT-6.1 Terra fastest, Opus 5.5 top score 85.1% — kohjingyu · 2026-11-04