Apple's LoopCD boosts looped Transformers 11+ points while cutting FLOPs up to 48%
apple · hf · 2026-10-02
Apple released LoopCD, a training-free contrastive decoding framework for looped Transformers.
- Key idea: earlier recurrent loops compute less, so they naturally form weak-and-strong prediction pairs—guidance comes free, without auxiliary models
- Two variants: LoopCD-Logits (one extra output pass, logit-space contrast) and LoopCD-Hidden (hidden-state contrast, zero output overhead)
- Results: raises Ouro-2.6B-Thinking's AIME 2024 pass@1 from 61.88% to 73.33% and Huginn's HumanEval pass@1 from 22.56% to 31.71%
- Gains allow halving the number of recurrent loops while matching or beating full-depth baselines, cutting forward FLOPs by 22.5%-48.2%
Related event: Apple's LoopCD: Training-Free Decoding Boosts Looped Transformers(4 posts)→
More from Research
- LEV: Looped decision model lifts 2.6B accuracy 63.2% to 76.2% with adaptive compute — jiqizhixin · 2026-11-19
- Aurora2-400M-A20M: a sparse MoE pretrained from scratch in 36 hours on one RTX Pro 6000 — Tall_Abrocoma_3533 · 2026-11-19
- Researcher calls METR/Anthropic '2x speedup' RSI analysis statistically dubious — jessi_cata · 2026-11-19
- Linear Probes Could Reveal Which Persona a Base Model Is Playing, Researchers Suggest — cephaloform · 2026-11-19
- ETH's Anymal robot completes 10km autonomous alpine hike with VLM-guided ROAM system — ChongZzZhang · 2026-11-19
- OpenVax codebase grows from 78K to 565K lines in 15 months — iskander · 2026-11-19