LoopCD: training-free contrastive decoding lifts looped transformers, AIME 61.9%→73.3%
arankomatsuzaki · x · 2026-10-02
A new Apple research paper introduces LoopCD, a training-free contrastive decoding framework for Looped Transformers.
Core idea: looped models reuse a shared block across recurrent passes; each pass yields an intermediate representation decodable for the same next token, but standard decoding discards earlier states. Recurrence inherently supplies aligned weak-and-strong prediction pairs — usable as contrastive guidance with no auxiliary models.
Two variants:
- LoopCD-Logits: contrasts final vs. earlier predictions in logit space with one extra output pass;
- LoopCD-Hidden: contrasts in hidden-state space with zero output overhead.
Results:
- Ouro-2.6B-Thinking's AIME 2024 pass@1 jumps from 61.88% to 73.33% (Logits);
- Huginn's HumanEval pass@1 rises from 22.56% to 31.71% (Hidden);
- Halving the recurrent loops still matches or beats full-depth baselines, cutting forward FLOPs by 22.5%–48.2%.
Paper: arXiv 2610.02185, "Decoding Looped Transformers Better for (Almost) Free".
Related event: Apple's LoopCD: Training-Free Decoding Boosts Looped Transformers(4 posts)→
More from Research
- Stanford's Le Cong to talk AI co-scientists for RNA design at 14th mRNA Health Conference — MengdiWang10 · 2026-11-19
- New paper shows DiT latents can be linearly probed to predict pixels, skipping the VAE decoder — kwangmoo_yi · 2026-11-19
- SAIL 2027 opens abstract submissions for AI in clinical care, deadline Jan 15 — zakkohane · 2026-11-19
- The academia LLM paradox: everyone writes with AI, nobody wants to read or be judged by it — wandedob · 2026-11-19
- Baseten hosts All-Gather, its first research conference on where AI's next gains will come from — baseten · 2026-11-19
- Interpretability solved—then what? Najoung Kim proposes filtering for recurrent constructs — najoungkim · 2026-11-19