A chronological map of looped-transformer papers, from Universal Transformers to DeepLoop
KyeGomezB · x · 2026-07-28
Chronological list of looped-transformer papers
The post collects a chronological reading list for Looped Transformers and related ideas, including:
- Universal Transformers
- MoEUT
- Relaxed Recursive Transformers
- Recurrent Depth Transformers
- Mixture-of-Recursions
- DeepLoop
- Loop the Loopies
The linked explainer argues that there are two broad ways to make an LLM “smarter”: scaling parameters or giving the model more compute/data during training. Looped transformer variants explore the second route by reusing layers or looping computation.
The image contrasts two patterns:
- Layer-loop: each layer is looped multiple times before moving on.
- Model-loop: the whole stack is repeated as a loop.
The cited Universal Transformers paper is highlighted as an early representative of this family, with claims about improved generalization on tasks that standard Transformers struggle with and the addition of dynamic halting.
More from Research
- Kimi K3 scales Kimi Linear to 2.8T parameters and drops RoPE for NoPE — rasbt · 2026-07-28
- A closed-loop system ran 633 experiments to find a two-metal N2O catalyst — bravo_abad · 2026-07-28
- A deep dive into Stable Diffusion’s safety filter and how red-teaming probes it — vboykis · 2026-07-28
- Lilian Weng’s long guide breaks down diffusion models from noise to samples — vboykis · 2026-07-28
- Nature reports a programmable CRISPR enzyme that shreds cancer-cell DNA — Dr_Singularity · 2026-07-28
- NeurIPS 2026 authors flag a rebuttal workflow bug as reviewers lose access to “Official comments” — howtorewriteaname · 2026-07-28