Looped Transformers 'overthink': InfiLoop paper shows loops keep burning compute after answers settle
jm_alexia · x · 2026-10-19
A new paper, InfiLoop, finds that looped Transformers suffer from "latent overthinking" — like reasoning LLMs burning tokens on solved problems, they keep spending loops after the answer has settled, with much of that computation redundant or even harmful.
- The fix isn't "loop less" but "loop smart": the residual connection turns out to be the key mechanism for looped models.
- Understanding and correcting this behavior recovers the performance that extra loops were supposed to add.
Paper and thread linked by the authors.
Related event: Looped Transformers suffer "latent overthinking," paper finds(2 posts)→
More from Research
- Anthropic Risk Report shows distillation degrades agentic coding: SWE-bench drops detailed — maksym_andr · 2026-10-20
- Probe-Guided Fine-Tuning for Alignment: ELLIS Team Open-Sources LoRAs and Detectors — maksym_andr · 2026-10-20
- Forward-only pretraining matches backprop: 3.704 val in 2.5 min on 8×H100, no autograd — tensorqt · 2026-10-20
- NeurIPS 2026 paper 'Intelligence per Watt' measures on-device AI efficiency across 1M queries — HazyResearch · 2026-10-20
- Cohere Labs talk: Goodfire's Matthew Kowal on interpretability across the model design pipeline — Cohere_Labs · 2026-10-20
- NeurIPS paper: AI models can provably hide arbitrary capabilities in encrypted weight backdoors — MoonL88537 · 2026-10-20