Sigma: first large-scale 3B/8B continuous diffusion LM matches AR models on math and coding
LucaAmb · x · 2026-10-05
- Researchers present Sigma, the first large-scale (3B/8B) continuous diffusion language model trained with likelihood — a new milestone in the diffusion LM line of work.
- It performs comparably to discrete diffusion LMs and autoregressive models on math and coding benchmarks.
- The continuous representation enables meaningful study of inference-time interventions in diffusion generation.
- Jointly led by zhihanyang, Wei Guo, Jean-Marie Lemercier and Simon Welker, with a team including NVIDIA researchers and Georgia Tech's Yongxin Chen.
More from Research
- SDECast: Neural SDEs Bring Continuous-Time Probabilistic Weather Forecasts Out to 5 Days — canaesseth · 2026-10-05
- Dev shares how he trained a PII redaction model for European languages — auto_grad_ · 2026-10-05
- Meta's NAVA-WAM pretrains robot action policies directly from action-free videos — meta · 2026-10-05
- gamfit: open-source Rust engine fits GAMs from a formula with REML-chosen smoothing — Sauers_ · 2026-10-05
- Fine-tuned Llama 3.1 8B for medical decisions hits 84% per-field accuracy, only 30-34% perfect rows — Forsaken_Cut8542 · 2026-10-05
- SJTU's LIFT adds force sensing to VLAs with zero force-labeled pretraining data — jiqizhixin · 2026-10-05