Recurrent Looped Transformer paper: recurrent state hits 48,000-block path after 1,000 tokens
techNmak · x · 2026-09-14
A proposed 48-layer Transformer called Recurrent Looped Transformer (RLT) accumulates a recurrent state path of 48,000 blocks after just 1,000 tokens — and the number is real, not a trick.
How it works: the decoder state stays alive across the whole sequence. Both prompt tokens and generated tokens update it, so the prompt/response boundary is merely a serving boundary, not a different computation. Each processed token adds 48 decoder-block transitions to the state history, giving a path length of 48t after t tokens.
Key clarification: this doesn't mean token 1,001 executes 48,000 blocks. Per-token compute stays fixed — in the paper's illustrative 48-layer encoder + 48-layer decoder setup, each token still evaluates only 96 encoder/decoder blocks (excluding merge and readout). The 48,000 figure describes how deep the recurrent state's history becomes, not per-step compute.
More from Models
- Fable 5.5 Max stuns with gorgeous SVG of a pelican riding a bicycle, headed to Claude Code — ChrisGPT · 2026-10-31
- OpenAI reportedly to train on users' Daybreak sessions with no opt-out — dyn___ · 2026-10-31
- NVIDIA Nemotron 3.5 scales LLM self-verification, +9-10 points on agentic benchmarks — sarahcat21 · 2026-10-31
- Mistral Large 4 scores 94% on 1,273 USMLE-style questions but flips 44 at temp 0 — MaziyarPanahi · 2026-10-31
- Researchers find 5 personality structures in LLM weight space, Big Five-like — luke_drago_ · 2026-10-31
- OpenAI cuts individual API access to Daybreak Blue starting November 20 — mkosmo · 2026-10-31