Apple's LoopCD boosts looped Transformers 11+ points while cutting FLOPs up to 48%
apple · hf · 2026-10-02
Apple released LoopCD, a training-free contrastive decoding framework for looped Transformers.
- Key idea: earlier recurrent loops compute less, so they naturally form weak-and-strong prediction pairs—guidance comes free, without auxiliary models
- Two variants: LoopCD-Logits (one extra output pass, logit-space contrast) and LoopCD-Hidden (hidden-state contrast, zero output overhead)
- Results: raises Ouro-2.6B-Thinking's AIME 2024 pass@1 from 61.88% to 73.33% and Huginn's HumanEval pass@1 from 22.56% to 31.71%
- Gains allow halving the number of recurrent loops while matching or beating full-depth baselines, cutting forward FLOPs by 22.5%-48.2%
Related event: Apple Releases LoopCD: Training-Free Decoding Boosts Looped Transformers(3 posts)→
More from Research
- Marin's 535B MoE hero run hits ~27% MFU on 11 NVL72 racks: expert parallelism deep dive — dlwh · 2026-10-03
- Harvard-led team unveils brain imaging 60x faster than fMRI, tracking activity in ~100ms — melnykowycz · 2026-10-02
- RExBench: best coding agent implements research extensions only 33% of the time — najoungkim · 2026-10-02
- Retrieval-augmented episodic memory narrows LLMs' lexical frequency gap in syntax tests — najoungkim · 2026-10-02
- EMPIRIC teaches robots missing physics as code, solving all 25 tasks where baselines get 14-16 — tomssilver · 2026-10-02
- What the AI pain paper actually found, according to someone who read it — Drukarshar · 2026-10-02