arXiv paper unifies looping and depth growing in LLMs, boosting reasoning accuracy up to 2x
kastnerkyle · x · 2026-10-31
The arXiv paper "From Growing to Looping: A Unified View of Iterative Computation in LLMs" (Kapl, von Oswald et al.) provides a mechanistic unification of layer looping and depth growing, showing both produce convergent depth-wise signatures and gains from a common form of iterative computation. Applying inference-time looping to a depth-grown model's middle blocks improves accuracy on some reasoning primitives by up to 2x, even without loop training. Depth-grown models benefit most from high-quality math-heavy cooldown mixtures. The paper positions the two as complementary methods for scaling iterative computation.
More from Research
- Working Paper Argues AI Can Break the Coasian Ceiling on Firm Size — zetalyrae · 2026-10-31
- Molence Bio Open-Sources Mol-1, Matching Closed SOTA at Antibody Structure Prediction — AllThingsApx · 2026-10-31
- Revisiting why transformers can't do zero-shot recursion, via Transformer Circuits math — akbirthko · 2026-10-31
- AI-assisted proof: first high-probability regret bounds for Feel-good Thompson sampling — QuanquanGu · 2026-10-31
- Vitória Piai's free psycholinguistics book Speaking joins ChapterPal's open-access library with an AI tutor — burkov · 2026-10-31
- Neuron Lens: A database of every neuron and attention head in Qwen3-8B and Gemma 4 12B — CatAstro_Piyush · 2026-10-31