Three papers converge: reuse layers, not parameters, to deepen masked diffusion LMs at inference
LucaAmb · x · 2026-09-05
The diffusionllms account surveys three recent papers converging on one insight: masked/diffusion LMs can get deeper without getting bigger by reusing computation at inference time instead of adding parameters.
- LoopMDM: selectively loops the early-middle transformer layers within each inference step. Looping during training yields a depth-scaling effect without extra parameters; varying loop count at inference gives flexible compute scaling. It matches same-size MDMs with up to 3.3x fewer training FLOPs and gains up to +8.5 points on GSM8K.
- R-MDM and a third paper take different routes to the same frontier, turning depth from a training-time constant into a resource you can spend at inference.
andreamiele announced a Monday discussion on combining dLLMs with looping.
More from Models
- OpenAI ships GPT-6 Astra with model guidance doc, urging devs to audit AGENTS.md and skill files — jxnlco · 2026-09-05
- One prompt, three minutes: early GPT-6 Astra demo makes the rounds — shauntrennery · 2026-09-05
- Grok web gets a cleaner redesign with a refreshed UI — XFreeze · 2026-09-05
- eyebench author says no v4, moving on to harder benchmarks — adonis_singh · 2026-09-05
- Astra-max claims vastly better intelligence-per-token even at low reasoning — adonis_singh · 2026-09-05
- Astra-max hits 95% on eyebench-v3 at half the cost of Sol-max, tokens ~3.8x fewer — adonis_singh · 2026-09-05