Looped Transformers aren't magic, just a deeper network, researcher argues
savvyRL · x · 2026-09-03
Researcher savvyRL pushes back on the Looped Transformers hype: it's not magical, doesn't add real recurrence to an inherently feedforward network, and the same idea was already explored by Dehghani et al. in 2019. The effect is simply making the model deeper, equivalent to stacking more layers from the start.
Related event: Looped Transformer Debate: Just Deeper Networks or Real Recurrence?(8 posts)→
More from Research
- Opus-assisted open design: a $31k low-field head MRI sketched in hours — DreamFly_13 · 2026-10-18
- semiconductor.diy: open-source guides to make real chips in a garden shed — jwt0625 · 2026-10-18
- Work in progress: geometric structure of GPG projections as Markov chain coarsening — michaelchchoi · 2026-10-18
- Backprop-free pretraining gets 2.3x speedup as Noise method crushes Dust record on 8 GPUs — inductionheads · 2026-10-18
- q3dviewer: open-source Qt+OpenGL tool for painter-like point cloud editing — 4310sy · 2026-10-18
- Smarter Every Day's Destin to defend PhD dissertation on impact fusion — CatAstro_Piyush · 2026-10-18