Recirculation: Training-Free Inference-Time Architecture Boost for LLMs

The paper proposes an inference-time architecture enhancement technique called "Recirculation," which lets off-the-shelf foundation models improve their reasoning without retraining. Current findings: on Gemma3 it achieves a 23% reduction in perplexity and a 21% accuracy gain on GSM8k, and it consistently lowers perplexity across five model families. The highlight is zero training cost, making it worth watching whether it can become a general-purpose, low-cost way to enhance open-source models.

Confirmed

• Core mechanism: via a specific form of recursion, upper-layer activations are injected into lower layers at the next step, letting the model track belief states as a dynamical system and breaking through the limits of feedforward Transformers.
• Performance: Gemma3 perplexity reduced by 23%, GSM8k accuracy improved by 21%.
• Generalization: comparisons relayed by @xennygrimmato show that, compared with the Unrolled loop transformer, the Recirculation transformer reduces perplexity across five different model families; a sweep of source-destination pairs reveals regions where perplexity drops consistently.
• Overhead: per @chaumian, the method adds almost no extra latency during generation, with serial processing needed only in the prefill stage.

Why it matters

• Significantly boosting model capability without retraining means any team holding an off-the-shelf foundation model could benefit at low compute cost; @teortaxesTex summed it up as "depth without training."
• Compared with existing recurrent approaches like the Unrolled loop transformer, Recirculation shows more consistent perplexity improvements and could become a candidate path for inference-time architecture enhancement.

2026-08-19 ~ 2026-08-20 · 6 related posts

• [source] No-Training Depth Boost? Recirculation Paper — teortaxesTex · 2026-08-19
• [source] Recirculation: Training-free inference architecture boosts model performance — chaumian · 2026-08-20
• Recirculation: Off-the-shelf models self-modify for instant inference boost — TheGradient · 2026-08-20
• Comparing Unrolled Loop and Recirculation Transformers — xennygrimmato_ · 2026-08-20
• [source] Recirculation Architecture Reduces Perplexity Across Gemma3 Models — xennygrimmato_ · 2026-08-20
• Recirculation Shows Perplexity Reduction Promise Across Five Model Families — xennygrimmato_ · 2026-08-20