LoopCD: training-free contrastive decoding lifts looped transformers, AIME 61.9%→73.3%
arankomatsuzaki · x · 2026-10-02
A new Apple research paper introduces LoopCD, a training-free contrastive decoding framework for Looped Transformers.
Core idea: looped models reuse a shared block across recurrent passes; each pass yields an intermediate representation decodable for the same next token, but standard decoding discards earlier states. Recurrence inherently supplies aligned weak-and-strong prediction pairs — usable as contrastive guidance with no auxiliary models.
Two variants:
- LoopCD-Logits: contrasts final vs. earlier predictions in logit space with one extra output pass;
- LoopCD-Hidden: contrasts in hidden-state space with zero output overhead.
Results:
- Ouro-2.6B-Thinking's AIME 2024 pass@1 jumps from 61.88% to 73.33% (Logits);
- Huginn's HumanEval pass@1 rises from 22.56% to 31.71% (Hidden);
- Halving the recurrent loops still matches or beats full-depth baselines, cutting forward FLOPs by 22.5%–48.2%.
Paper: arXiv 2610.02185, "Decoding Looped Transformers Better for (Almost) Free".
More from Research
- HIDE Benchmark Exposes Memory Gaps in Robotic Manipulation Under Partial Observability — Yansong Shi · 2026-10-02
- InterEvolve Evolves Reward Programs at Test Time to Teach Humanoid Robots New Skills — UIUC-CS · 2026-10-02
- Smaller Models Make Better Rejects: Study Rethinks Preference Distillation from 7B to 72B — LinkedIn · 2026-10-02
- HeteroFold Enables Prefill-Free Cross-Family KV Cache Transfer, 10.7x Faster at 32K Context — UniversityofSouthernCalifornia · 2026-10-02
- Nature cover: 6.3M nuclei sequenced to build largest human prefrontal cortex cell atlas — jiqizhixin · 2026-10-02
- Detect LLM hallucinations in 1.3µs on CPU — but 120B models hallucinate with unanimous false certainty — More_Slide5739 · 2026-10-02