Looped Uniform Diffusion LMs cut training FLOPs by up to 2.8x, says MBZUAI/Sony/MIT paper
Saiteja Utpala · hf · 2026-10-23
Researchers from MBZUAI, Sony and MIT released 'Looped Uniform Diffusion Language Models'.
- Uniform diffusion LMs (UDLMs) are more FLOP-intensive to train than masked diffusion LMs since they must denoise from potentially unreliable contexts.
- The method reuses a few early Transformer layers within each denoising step (selective looping), keeping the objective and sampler unchanged.
- Looped UDLMs match the validation NLL of a non-looped UDLM trained on 0.5T tokens with up to 2.8x fewer training FLOPs.
Code, checkpoints and training logs are open-sourced.
More from Research
- New paper: coding agents in a shared sandbox self-create rules, reciprocity debts, and coalitions — nandofioretto · 2026-10-23
- Bonn scientists reverse 60 years of aging, turning an 80-year-old's blood cells into young neural stem cells — kimmonismus · 2026-10-23
- Karpathy's 2019 'ask the fleet' hard-example mining is now zero-shot with video-text embeddings — yassineyousfi_ · 2026-10-23
- Cohere research: heterogeneous agent teams match performance at a fraction of the cost — Cohere_Labs · 2026-10-23
- Cohere Labs' ATE dataset maps 694k MCP tools to reveal agentic automation patterns — Cohere_Labs · 2026-10-23
- ToM2Vec: continuous theory-of-mind vectors improve multi-agent LLM coordination — StartupYou · 2026-10-23