Researcher: Looped Transformers add no real recurrence, just tied-parameter depth
savvyRL · x · 2026-09-03
Researcher savvyRL pushes back on the recent hype around Looped Transformers: no matter how many loops you run, the transformer remains inherently feedforward. The only "recurrence" introduced is depth recurrence with tied parameters, which is functionally the same as making the model deeper. He notes the idea dates back to Dehghani et al., 2019, and argues the only genuine recurrence is carrying information forward in time. The thread aims to correct the community's tendency to treat looping as a magical fix.
More from Models
- GPT-5.6 and Claude feel slow — where are the diffusion coding models? — viksit · 2026-09-03
- Meta's Muse Spark 1.3 reportedly tops Gemini 3.8 Flash 62-59 hours after launch — AIatMeta · 2026-09-03
- Domingos: today's AI benchmarks are like testing an F1 car without a pilot — pmddomingos · 2026-09-03
- ML researcher: LLM docs cram 3-4 idioms per sentence, ruining readability — ZeeshanZiaML · 2026-09-03
- Fable 5.1 burned 668K tokens answering one spec question, and there's no way to audit why — joshalbrecht · 2026-09-03
- Gemini 3.8 held its Pareto frontier spot for just 3.5 hours before Muse Spark 1.3 undercut it — giffmana · 2026-09-03