DiscoLoop: A Residual Design to Break Looped Transformers' Implicit Reasoning Bottleneck
jasondeanlee · x · 2026-10-09
Frontier labs are rumored to be adopting looped Transformers for implicit reasoning, but can they truly master it? Using mechanistic interpretability, the authors identify a bottleneck that keeps looped Transformers from perfect implicit reasoning, explain how looping helps and why it struggles to generalize, and introduce DiscoLoop, a residual design that improves implicit reasoning. Long thread with details.
Related event: Berkeley's DiscoLoop Fixes Generalization Bottleneck in Looped Transformers(2 posts)→
More from Research
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23
- Perceptic launches PercepticOS, an AI workspace for 'scientific superintelligence' — nathanbenaich · 2026-10-23
- EPFL's Bosselut to talk long context as test-time learning at ELLIS seminar — andre_t_martins · 2026-10-23
- DRATS: Adaptive Task Sampling Boosts Worst-Task Performance in Multi-Task RL — Cohere · 2026-10-23