Recirculation
Michael C. Mozer, Shoaib Ahmed Siddiqui, Danny Sawyer, Sunny Sanyal, Rosanne Liu
cs.LG
2026-08-19
Google DeepMind proposes recirculation, an inference-time tweak that leaks deep-layer activations back into a shallow layer and reruns the stack; frozen Gemma3 models gain up to 35% lower perplexity and a 21% error reduction on GSM8k.
A feedforward transformer updates a token's representation once per layer, so the number of state updates is capped by depth. Shallow layers see an ambiguous word before context resolves it; by the time deeper layers pick the right meaning, that verdict cannot flow back down. A 2025 activation-patching experiment made the point directly: copying the disambiguated deep-layer representation down to a shallow layer cut contextualization errors by 60%. The information exists. The architecture just cannot reach it.
The visible symptom is inconsistency across a conversation: a model that has already read "bank" as a river bank still answers an ATM question from the financial-institution association. Chain of thought buys serial state updates but pays in generated tokens; looped transformers add depth rather than per-step recurrence, and training recurrent variants is expensive.
Recirculation is remarkably simple. Run the model token by token, and after each step, mix a scaled copy of the residual-stream activation at layer s into a shallower layer d, then run a second pass of the stack:
Why mix directly instead of training an adapter? The residual stream acts as a shared blackboard: a feature has the same direct effect on the output distribution no matter which layer writes it, so deep-layer semantics can be read by shallow layers without any learned translation. Keeping the leak small avoids pushing representations off the training distribution, which is why it works with frozen weights.
The contrast with looped transformers is about where state lives. Looping moves state up the stack, so tracking t steps needs t layers of depth. In recirculation the same layer can hold both z(t) and z(t+1), giving unbounded state tracking in principle. The price is a serial, token-by-token prefill; during generation the two stacks run in parallel and latency is essentially unchanged.
| Setting | Baseline | Recirculation | Change |
| Perplexity, PG19, Gemma3 12B | 52.86 | 34.15 | -35.4% |
| Perplexity, arXiv, Gemma3 1B | 19.10 | 16.54 | -14.0% |
| Perplexity, mean of 10 datasets, 1B | - | - | -8.5% (fixed α) |
| Same, adaptive variant | - | - | -23.0% |
| GSM8k pass@1, Gemma3 4B | 29.3% | 30.6% to 35.5% (adaptive) | -8.8% / -20.9% error |
| Instruction following, 12B error rate | - | - | -75% |
Nine of ten language-modeling datasets improve consistently across model scales; lambada is the outlier, attributed to very short sequences and tokenization artifacts. Two controls rule out cheap explanations. Raising softmax temperature to 1.2 alone cuts perplexity by 8.48% versus 14.21% for recirculation, and the effects combine near-additively (19.55%), so this is not distribution sharpening. Training-free looping over the same layer grid shows no robust benefit on Gemma3 and a qualitatively different heatmap, confirming the two mechanisms differ.
The adaptive variant trains only an MLP that emits vector-valued, token-conditional α and β; the backbone stays frozen. It reaches a 23.0% mean perplexity reduction on Gemma3 1B, beating full fine-tuning of the recurrent architecture at 21.6%. Token-level analysis shows gains concentrate at short lags and mid-window positions, with adverbs, adjectives and verbs benefiting most while numerals, determiners and pronouns barely move, and plural nouns gaining where singulars do not. That pattern fits the persistent-state interpretation.
The result that an architecture change can pay off with zero training is itself the finding: pretrained models already contain affordances for recurrence that nobody has queried. The practical playbook for practitioners is to sweep layer pairs on a frozen model first, learn where this model family wants recurrence, and only then decide whether to spend training budget.
Gemma3's Peri-LN layout may be especially receptive, but Ministral3, Pythia, Qwen3 and Phi2 all show a stable benefit region in the sweep, just far smaller (about 5% versus under 0.5%). Normalization and α were not tuned for those families, so the gap may shrink.
The authors list six; the two hardest are that the layer pair and mixture coefficients look domain- and task-dependent, and that without task-universal hyperparameters or an automatic mapping the practical reach is limited. Prefill must run token by token, which can be infeasible for long contexts; a blockwise variant (K tokens at a time) is proposed but untested.
Several results deserve cold water. On the 12B model, two of three question types in the Racing Thoughts contextualization benchmark get worse. The adaptive variant is highly sensitive to its training set: training on ARC datasets drops accuracy across the board, and the only across-the-board gain comes from training on a slice of the MMLU test set, a column the paper itself flags for train-test overlap. Benefits for non-Gemma families sit under 0.5% before tuning. The authors position this as a methodological contribution rather than a shovel-ready technique, and that self-assessment is honest.