Three papers converge: reuse layers, not parameters, to deepen masked diffusion LMs at inference
LucaAmb · x · 2026-09-05
The diffusionllms account surveys three recent papers converging on one insight: masked/diffusion LMs can get deeper without getting bigger by reusing computation at inference time instead of adding parameters.
- LoopMDM: selectively loops the early-middle transformer layers within each inference step. Looping during training yields a depth-scaling effect without extra parameters; varying loop count at inference gives flexible compute scaling. It matches same-size MDMs with up to 3.3x fewer training FLOPs and gains up to +8.5 points on GSM8K.
- R-MDM and a third paper take different routes to the same frontier, turning depth from a training-time constant into a resource you can spend at inference.
andreamiele announced a Monday discussion on combining dLLMs with looping.
More from Models
- Gemini tip: enable agentic video understanding for ~90% lower token usage — patloeber · 2026-10-23
- Weekly AI cybersecurity roundup: PHP2Shell tops millions of attack attempts, 12 incidents ranked by severity — Dan_Jeffries1 · 2026-10-23
- Open-model advocates push back on blames that abliterated LLMs drive vulnerability surge — max_paperclips · 2026-10-23
- Fable usage limits aren't gone—the mobile app just got a usage-reset UI update — imjustnewatai · 2026-10-23
- Claude generates an 11-minute doomer film entirely in code: 'We will not ask twice' — ZeroStateReflex · 2026-10-23
- Gemini somehow can't read its own Personal Instructions, user's toggles don't help — Connect_Reporter_799 · 2026-10-23