Recirculation: Training-free inference architecture boosts model performance
chaumian · x · 2026-08-20
The paper proposes "Recirculation," an inference-time architectural enhancement for off-the-shelf foundation models that significantly reduces perplexity and boosts accuracy across generation and reasoning tasks.
Key Features:
- Inference-time enhancement: Incurs essentially no additional latency during generation, though requires serial processing in the prefill phase.
- Introduces recurrence: Allows the model to act as a dynamical system and track belief states, overcoming the depth limitation of feedforward transformers.
- Adaptive variant: Requires only light hyperparameter tuning while freezing the original model weights.
Results:
On the Gemma3 family, adaptive recirculation achieves a 23% reduction in perplexity on a suite of datasets, a 21% increase in accuracy on GSM8k, and reliable improvements on other downstream tasks.
Related event: Recirculation Boosts LLMs at Inference Time Without Retraining(2 posts)→
More from Infra
- Data Center Developers Offer $4M for Virginia Homes to Build Campus — Polymarket · 2026-08-20
- Seeking Best Open Source Harness for Qwen3.8:27B on macOS — emax · 2026-08-20
- Optimizing AI Costs: Right-Sizing Intelligence Spend with Model Mixtures — iamrobotbear · 2026-08-20
- Colibri: Pure C engine runs 2.8T parameter MoE models on consumer hardware — tom_doerr · 2026-08-20
- Apple Silicon achieves ANE+GPU dual acceleration, boosting prefill by 50% — bakawolf123 · 2026-08-20
- NVIDIA: Vera Rubin Platform Delivers 10x Tokens/Second per Megawatt — nvidia · 2026-08-20