Recurrent Looped Transformer (RLT) open-sourced: latent reasoning with infinite depth
inductionheads · x · 2026-09-13
Researcher Yifan Zhang released and open-sourced the Recurrent Looped Transformer (RLT), which carries latent computation across every prompt and response token: a causal encoder builds a global key-value memory, while a recurrent decoder combines it with sliding-window attention and feedback from its previous final hidden state.
- Infinite temporal depth: with decoder depth LD, after t tokens the recurrent path traverses t×LD blocks while per-token executed blocks stay fixed;
- Model-hardware co-design: parallel causal encoding, batching across sequences, memory reuse and activation checkpointing for efficient training and inference;
- The author frames it as a step toward superintelligence and argues frontier progress should be paced until safe superintelligence is achieved.
The repo includes English and Chinese paper PDFs and a project site; it is an early-stage release (32 stars).
More from Research
- Stanford's new CS312 makes assignments worth 0%, grades you on predicting loss curves — PMinervini · 2026-09-13
- Google DeepMind team publishes deep dive on scaling Transformer inference — zainhas · 2026-09-13
- Imaging Neuroscience paper extends permutation tests on brain map correspondence with mixed-effects modeling — bttyeo · 2026-09-13
- Smoothing Before IK Shrinks Bones by Half: A Blender Stress Test Explains Why — andrew_n_carr · 2026-09-13
- Terence Tao's new essay: AI shifts math's scarce resource from finding proofs to understanding them — NandoDF · 2026-09-13
- Raji calls for a new edition of 'Who Audits the Auditors?' for frontier AI — rajiinio · 2026-09-13