Researcher argues "recurrent" Transformers are just deeper feedforward nets with tied parameters
savvyRL · x · 2026-09-03
savvyRL reiterates that no amount of looping makes a Transformer truly recurrent: the added "recurrence" is just depth recurrence with tied parameters, equivalent to making the model deeper; real recurrence requires carrying state forward in time. Cites their papers "The Topological Trouble With Transformers" (arxiv:2604.17121) and "Recirculation" (arxiv:2608.17981).
More from Research
- Researcher claims they may have 'cracked RL' for their task — andrew_n_carr · 2026-09-03
- Judea Pearl fires back in causal inference debate: RCT's new language deserves open scrutiny, not hiding — yudapearl · 2026-09-03
- Genomics researcher calls out popular benchmarks as opaque and flawed — anshulkundaje · 2026-09-03
- Untrained networks already tell cats from dogs: architectures as evolutionary selection — joannejang · 2026-09-03
- CS professor builds AI system to preserve endangered languages and create new ones — begusgasper · 2026-09-03
- Cosmos Institute funds 80 grantees building AI for human autonomy and truth-seeking — hunarbatra · 2026-09-03