Survey maps continuous diffusion and flow language models, now competitive at billions of parameters
LucaAmb · x · 2026-11-09
A new survey charts the landscape of continuous diffusion and flow language models, which generate text by transforming noise into continuous token representations.
Key points
- These models refine many positions in parallel while preserving dependencies, and their continuous states allow generation to be steered via gradients of differentiable objectives.
- In few-step regimes, standard parallel discrete-diffusion samplers incur factorization errors from conditionally independent token sampling; joint generation in continuous models avoids this.
- Recent models match discrete diffusion on language benchmarks and scale to billions of parameters.
The survey covers text representations, learned generative dynamics, few-step generation, plus scaling, applications, evaluation and open problems, with an interactive paper map and a curated GitHub collection.
Related event: New Survey Charts Rise of Continuous Diffusion and Flow Language Models(4 posts)→
More from Research
- Score centering: the simple fix for LLM RL instability under training-inference mismatch — cwolferesearch · 2026-11-10
- A faulty memory note stranded an AI agent: why unchecked memory makes agents consistently wrong — krishnan · 2026-11-10
- PostTrainBench v1.2: Gemini 4 Argon Scores 40.4% (#5), Muse Wonder 24.2% (#17) — maksym_andr · 2026-11-09
- SPDMX releases 254K-song open multitrack music dataset synthesized from public-domain scores — umpedronosapato · 2026-11-09
- MathArena adds October BrokenArXiv: GPT-6.1 Sol leads but drifts down monthly, Opus 5.5 rock solid — teortaxesTex · 2026-11-09
- AI Proves the Aaronson-Ambainis Conjecture, Another Longstanding Quantum Theory Open Problem Falls — ns123abc · 2026-11-09