Recurrent Looped Transformer (RLT) open-sourced: latent reasoning with infinite depth
inductionheads · x · 2026-09-13
Researcher Yifan Zhang released and open-sourced the Recurrent Looped Transformer (RLT), which carries latent computation across every prompt and response token: a causal encoder builds a global key-value memory, while a recurrent decoder combines it with sliding-window attention and feedback from its previous final hidden state.
- Infinite temporal depth: with decoder depth LD, after t tokens the recurrent path traverses t×LD blocks while per-token executed blocks stay fixed;
- Model-hardware co-design: parallel causal encoding, batching across sequences, memory reuse and activation checkpointing for efficient training and inference;
- The author frames it as a step toward superintelligence and argues frontier progress should be paced until safe superintelligence is achieved.
The repo includes English and Chinese paper PDFs and a project site; it is an early-stage release (32 stars).
More from Research
- AIR: Effective-Learning-Rate Scheduler Beats Cosine on 128B-Token MoE Pretraining — stochasticchasm · 2026-10-30
- CMU's Gym-Anything Auto-Builds 200+ Agent Environments from Any Software, Selected by GDP — wellecks · 2026-10-30
- NVIDIA researcher: train robot policies with privileged sim info, improving Pi 0.5 — wellecks · 2026-10-30
- Leveraging Simulator Privileged Information: Three Methods to Improve Robot Models — wellecks · 2026-10-30
- CMU Auto-Builds 25 Robot Simulations with Agents; Frontier Model Solves 0 of 25 Test Tasks — wellecks · 2026-10-30
- DeepMind's f_charton shares his take on the current state of AI and mathematics in Sorbonne talk — f_charton · 2026-10-30