Continuous diffusion language models are making a comeback, Sander Dieleman writes
joao_gante · x · 2026-08-24
Sander Dieleman published a 44-minute read, "Continuous diffusion language models." He notes that language modeling has long been dominated by autoregression: token-by-token generation, teacher forcing for efficient parallel training, and extreme scalability — the recipe behind today's LLMs. But diffusion offers an alternative iterative generation path, inspired by early successes in audio and image domains.
He traces the history: early continuous-diffusion attempts for language were largely supplanted by fully discrete diffusion methods and went dormant, yet this year has seen a flurry of new research, suggesting a genuine revival. The post covers the historical perspective and technical aspects behind the comeback, presented as a subjective account open to dissenting views.
More from Models
- User praises Qwen 3.8 for versatility and hardware compatibility — uwukko · 2026-08-24
- Laguna XS 2.1: Top coding pick for low-VRAM GPUs — needthosepylons · 2026-08-24
- OX-Alpha outperforms DeepSeek-V4 in coding benchmarks: Test — karminski3 · 2026-08-24
- Qwen2.5-72B tops HuggingFace likes, surpassing QwQ-32B — choose_a_guest · 2026-08-24
- Agnes-AI quietly posts Agnes-2.5-Pro-Alpha on Hugging Face — External_Mood4719 · 2026-08-24
- Open source Pelican SVG Env quantifies model drawing abilities — SergioPaniego · 2026-08-24