Researcher: Looped Transformers add no real recurrence, just tied-parameter depth
savvyRL · x · 2026-09-03
Researcher savvyRL pushes back on the recent hype around Looped Transformers: no matter how many loops you run, the transformer remains inherently feedforward. The only "recurrence" introduced is depth recurrence with tied parameters, which is functionally the same as making the model deeper. He notes the idea dates back to Dehghani et al., 2019, and argues the only genuine recurrence is carrying information forward in time. The thread aims to correct the community's tendency to treat looping as a magical fix.
Related event: Looped Transformer Debate: Just Deeper Networks or Real Recurrence?(8 posts)→
More from Models
- Google's Business Standard plan says "Gemini across Workspace" but still can't test AI — ChrisRackauckas · 2026-10-18
- Paying Google AI subscriber blocked: custom-domain Workspace accounts can't access AI Studio — ChrisRackauckas · 2026-10-18
- Running Ling 3.1 Flash (560B MoE, 25B Active) Across 4 DGX Sparks via llama.cpp RPC — BerryBlush_Rogers691 · 2026-10-18
- OpenAI API lists aurora-blue/green/red-alpha checkpoints, reportedly a response to Fable 5.5 — ChrisGPT · 2026-10-18
- Users press OpenAI on long-delayed Adult Mode a year after it was promised — ___Patrice___ · 2026-10-18
- Polymarket opens betting market on who releases the next Gemini Argon model — Polymarket · 2026-10-18