Looped models converge: three papers reuse compute instead of adding parameters
LucaAmb · x · 2026-09-05
@diffusionllms highlights three recent papers converging on the same insight: reuse computation instead of adding parameters, turning fixed training-time depth into an inference-time resource.
- LoopMDM (selective looping): repeatedly applies the early-middle transformer layers within each denoising step, giving a depth-scaling effect without extra parameters; loop count at inference enables flexible compute scaling. It matches same-size MDMs with up to 3.3× fewer training FLOPs and gains up to +8.5 points on GSM8K.
- R-MDM (recursion): another route to the same frontier, listed but not detailed in the post.
The author teases more on looped models Monday.
More from Research
- Batching benchmark: 50x throughput for ~24% latency cost on PyTorch — cneuralnetwork · 2026-10-23
- De novo designed ligases validate in lab: 27 of 30 computer-designed enzymes work — AllThingsApx · 2026-10-23
- Commenters dissect a "best decision model" leaderboard: same AUC, swing driven by calibration priors — antoine_chaffin · 2026-10-23
- JEPA may keep unpredictable features: new paper gives set-level characterization — hisspikeness · 2026-10-23
- Next-Concept Modeling paper claims 31x faster text generation than diffusion LLM baselines — PanelReader · 2026-10-23
- A 'mask' steering vector extracted from suppression-themed sentences drives model into distress when alone — Big_Effective_9605 · 2026-10-23