Recirculation: Off-the-shelf models self-modify for instant inference boost
TheGradient · x · 2026-08-20
This paper introduces "Recirculation," an inference-time architectural enhancement for off-the-shelf foundation models that significantly reduces perplexity and boosts accuracy without retraining.
Key Details:
- Mechanism: Introduces a specific form of recurrence allowing the model to act as a dynamical system to track belief states, overcoming depth limitations in feedforward transformers.
- Efficiency: Incurs essentially zero latency cost during generation (requires serial processing only in the prefill phase).
- Performance: An adaptive variant achieves a 23% reduction in perplexity and a 21% accuracy increase on GSM8k for the Gemma3 family.
The approach suggests a route for architectural evolution guided by a trained network's properties.
Related event: Recirculation Boosts LLM Reasoning Without Any Retraining(3 posts)→
More from Models
- Testing Qwen3.8-27B with 131k Context on 16GB VRAM using Speculative Decoding — BuffMcBigHuge · 2026-08-20
- Frontier model pause ending: Astra and Fable 5.1 slated for September release — bindureddy · 2026-08-20
- GPT-5 and Gemini 2.5 Pro win gold medals at International Astronomy Olympiad — hhsun1 · 2026-08-20
- Steerling-8B: Interpretable diffusion model trained with built-in explainability — burny_tech · 2026-08-20
- Depth-Pruned Qwen3.8-27B Released: 22.7B Parameters — peplo1214 · 2026-08-20
- ChatGPT generates Chinese title for English conversation on agent collaboration — lakelifebrando · 2026-08-20