Paying 25x for 0.06 points: blind test shows max reasoning effort barely beats medium
aakashgupta · x · 2026-11-11
A blind-scored test across 8 PM tasks shows the cost/benefit of reasoning effort levels:
- Fable 5.2: max effort (12 min of thinking) scored 9.07 at $3.27/task; medium (30s) scored 9.01 at $0.13 — the extra $3.14 bought 0.06 points.
- Opus 5.5: max scored 9.20 at $1.19 vs medium 8.77 at $0.05 — 24x cost for 0.43 points.
- Key finding: switching models at the same price moved scores more than switching effort — Opus 5.5 medium and Gemini 4 Argon high both cost $0.05/task but scored differently.
Conclusion: deep reasoning has tiny marginal returns; model choice beats effort dial on cost-effectiveness.
Related event: Test Shows 12 Extra Minutes of Model Thinking Costs 25x for 0.06 Point Gain(2 posts)→
More from Models
- Gemini 4 Argon slated for Oct 13 and Carbon for Nov 10 as Google DeepMind shipping season begins — MickeySteamboat · 2026-11-11
- Gemini 4 Carbon lands in Cursor, tops CursorBench at 60.4% on high effort — andyzhang · 2026-11-11
- Frontier model speed gap is underrated: 1.8 min per task vs 12.7 for DeepSeek — rickasaurus · 2026-11-11
- Reddit: Argon LiveNerf benchmark reports an 83% nerf — ThePrimeNumbers · 2026-11-11
- Rumor: OpenAI holding GPT-6.2 Astra until Anthropic's Nov 19 IPO to steal the spotlight — kimmonismus · 2026-11-11
- Pangram flags human-written text as 66% AI, sparking detector false-positive debate — austinc3301 · 2026-11-11