Looped Transformers explained: same weights, evolving hidden states, extra compute per parameter

TheTuringPost · x · 2026-11-17

Turing Post breaks down Looped Transformers: the same Transformer block (or stack) runs for multiple passes, feeding each output back as input. What loops is the hidden state—token embeddings initialize it, the shared stack updates it via attention and FFN, and the output head converts the final representation to token scores. The companion chart contrasts 24 unique blocks × 1 pass vs 6 blocks × 4 passes (both = 24 block applications): same effective depth without per-pass weights, with fixed or adaptive stopping.

Original post →

More from Research

Research channel →