Sander Dieleman on why continuous diffusion language models are making a comeback
LucaAmb · x · 2026-10-06
DeepMind researcher Sander Dieleman published a 44-minute essay examining the resurgence of continuous diffusion language models after years of dormancy, and why it's happening now.
- Context: modern LLMs are overwhelmingly autoregressive, but autoregression isn't the only way to build an iterative generative process for sequences; early continuous-diffusion-for-language attempts were later supplanted by fully discrete methods — until the tide turned
- A catalyst is Sigma, the first large-scale (3B/8B) continuous diffusion LM trained with likelihood, matching discrete dLMs and AR models on math and coding while enabling meaningful inference-time intervention studies
- The essay covers the technical aspects of continuous diffusion for language and invites dissenting perspectives
More from Models
- Julien Simon: Calling Aleph Alpha's Kolibri 'Europe's DeepSeek Moment' Is a Misreading — julsimon · 2026-10-06
- Opus 5.5 Is Capable but Needs Pushing, as New-Model Hype Outruns Reality — haider1 · 2026-10-06
- Zuckerberg says Llama 4 failed because it was staffed like Instagram, not a frontier lab — rohanpaul_ai · 2026-10-06
- Controlled Study Finds No Encoding Dominates: Pixels, Bytes and Tokens Each Win on Different Tasks — delliott · 2026-10-06
- Old 'gpt-next' codename resurfaces; researcher speculates an intentional OpenAI easter egg — RileyRalmuto · 2026-10-06
- Claude leaves 50% of plan quota after burning one model's cap; OpenAI goes to zero — xiaohu · 2026-10-06