Looped models converge: three papers reuse compute instead of adding parameters
LucaAmb · x · 2026-09-05
@diffusionllms highlights three recent papers converging on the same insight: reuse computation instead of adding parameters, turning fixed training-time depth into an inference-time resource.
- LoopMDM (selective looping): repeatedly applies the early-middle transformer layers within each denoising step, giving a depth-scaling effect without extra parameters; loop count at inference enables flexible compute scaling. It matches same-size MDMs with up to 3.3× fewer training FLOPs and gains up to +8.5 points on GSM8K.
- R-MDM (recursion): another route to the same frontier, listed but not detailed in the post.
The author teases more on looped models Monday.
More from Research
- Homework for researchers: extending RoPE to tensor product representations — thomasahle · 2026-09-05
- Debate: Models Fuzzily Recall Concepts, Not Text — SAE Features vs Edit-Distance Memorization — voooooogel · 2026-09-05
- ICML Position Paper: Unlabeled Data Doesn't Mean No Human Supervision — serrjoa · 2026-09-05
- VLA-Corrector from ZJU & Alibaba DAMO lifts robot success rates while cutting policy calls — 机器之心 · 2026-09-05
- Spanda: Open-Source Hallucination Detector Runs in 1.5ms on CPU, 90,000x Faster than Semantic Entropy — Otherwise_Nobody_721 · 2026-09-05
- Bug Hunt Bench: 105 real bugs stress-test GPT-6, Claude, Grok, Gemini and more coding agents — PawelHuryn · 2026-09-05