Meta Paper: Quantization Causes Overthinking in Reasoning Models, Fixable with Word Penalties
rohanpaul_ai · x · 2026-08-09
A new paper from Meta highlights that post-training quantization in reasoning models introduces noise, causing them to frequently second-guess themselves even after reaching the correct answer.
This quantization noise makes models more likely to generate words like "wait" or "but" at uncertain steps, reopening the reasoning process. Aggressive quantization was found to increase overthinking failures by up to 52%.
Tested across math, coding, and science tasks on models ranging from 1.5B to 32B parameters, the researchers demonstrated that applying a small decoding penalty to about 50 hesitation words can reduce reasoning length by 12% to 23% while maintaining or even improving accuracy.
More from Models
- willcb: Opus 5.5 doesn't smell bigger, it smells like they fixed the data — willcb · 2026-10-16
- Unverified claim: Apple and Meta's 816B-parameter Muse Spark Open runs on two Mac Studios — TansuYegen · 2026-10-16
- Gemini 4 model card: model eliminated months of trial-and-error on bioweapons research — Puzzleheaded-King584 · 2026-10-16
- Every AI picks 7: Anthropic engineer's rule for escaping model taste convergence — rubenhassid · 2026-10-16
- Brain teaser: strip all attention, MLP and embedding matrices from a Llama 3 — how many trainable params remain? — Sauers_ · 2026-10-16
- Lean v4.35.0 ships verified kernels — one written and proven consistent by Claude — geoffreyirving · 2026-10-16