ARC-AGI Insight: More Reasoning Tokens Lead to Wrong Hypotheses; GPT-5.6 Max is Cheaper
GregKamradt · x · 2026-08-08
Greg Kamradt highlighted an analysis from ARC Prize explaining the counter-intuitive "zigzag" cost curve in reasoning models. Research reveals that higher reasoning performance often comes at a lower cost.
ARC officials explain that models running in Max reasoning mode actually consume fewer tokens overall than High mode. The reasons include:
- High mode often burns through tokens elaborating on an incorrect hypothesis, eventually failing to produce a usable prediction.
- Max mode correctly combines fundamental rules (like color conservation) earlier, hitting the right answer without unnecessary exploration.
For instance, on one ARC-AGI-2 task, High mode spent 98k tokens building a complex but wrong theory, whereas Max mode solved it correctly with only 76k tokens. This dynamic also underscores the outstanding cost-to-performance ratio demonstrated by DeepSeek.
More from Models
- GYRO quant of Qwen3.8-Flash-Next tested: near Q3 quality at much smaller size — PlasticRevenue4601 · 2026-10-16
- Open source is so back: dozens of open models dropped within 48 hours — beffjezos · 2026-10-16
- Gemini users hit usage caps on 3.1 Pro and 4 Argon, resets not until Oct 16 — R3d-Gr33n-Blu3 · 2026-10-16
- Gemini desktop app can't accept images — an image model that can't take images — Rebl · 2026-10-16
- Apollo Research: GLM-5.3 unprompted reasons about its actions becoming training data — _lewtun · 2026-10-16
- Opus 5.5 turns heads with surprisingly polished vertical video demo — FinanceYF5 · 2026-10-16