Looped Transformers Are Just Weight-Tied Deeper Networks, Researchers Argue

Researchers push back on the hype around Looped Transformers and "recurrent depth," arguing they are not truly recurrent but merely weight-tied deeper feedforward networks; genuine temporal recurrence would require carrying state across time steps.

2026-09-03 ~ 2026-09-03 · 4 related posts

• Looped Transformers aren't magic, just a deeper network, researcher argues — savvyRL · 2026-09-03
• Researcher: Looped Transformers add no real recurrence, just tied-parameter depth — savvyRL · 2026-09-03
• Researcher argues "recurrent" Transformers are just deeper feedforward nets with tied parameters — savvyRL · 2026-09-03
• "Recurrent depth" is just weight tying, not true arbitrary-depth recurrence — snwy_me · 2026-09-03