Researcher argues "recurrent" Transformers are just deeper feedforward nets with tied parameters
savvyRL · x · 2026-09-03
savvyRL reiterates that no amount of looping makes a Transformer truly recurrent: the added "recurrence" is just depth recurrence with tied parameters, equivalent to making the model deeper; real recurrence requires carrying state forward in time. Cites their papers "The Topological Trouble With Transformers" (arxiv:2604.17121) and "Recirculation" (arxiv:2608.17981).
Related event: Looped Transformer Debate: Just Deeper Networks or Real Recurrence?(8 posts)→
More from Research
- Terence Tao on 'War Footing' as OpenAI Claims Navier-Stokes Solve Shakes Mathematics — AlexTensor · 2026-10-18
- Univ. of Tokyo paper: why your recorded voice feels eerie — a vocal uncanny valley — ssh4net · 2026-10-18
- MIT's SANDO Drone Planner Avoids Every Moving Obstacle Across 12 Real Flights — Olivier__OG · 2026-10-18
- No-backprop pretraining speedup of 2.3x claimed, inspired by mech interp essay — CatAstro_Piyush · 2026-10-18
- Models Can Be Trained to Sense Activation Probes and Generalize Across GPUs Zero-Shot — Sauers_ · 2026-10-18
- Simulating a Turing machine with addition and multiplication by encoding the tape as a number — jessi_cata · 2026-10-18