NVIDIA Releases Nemotron-Labs-Sigma: Continuous Diffusion LLMs Generating Token Blocks in Parallel
karsten_kreis · x · 2026-11-19
- NVIDIA open-sources Nemotron-Labs-Sigma, 14 checkpoints at 3B/8B — the first large-scale continuous diffusion language models, bringing smooth ODE trajectories (16-D) to text instead of discrete tokens.
- Method: blockwise likelihood training denoises Gaussian-corrupted embeddings while learning an optimal geometry, warm-started from pretrained AR weights; classifier-free guidance and score temperature prove essential at inference.
- Results: beats masked dLMs and AR baselines on GSM8K, Minerva, HumanEval, MBPP after pretraining, with the largest consistent coding gains; strong MATH-500/AIME results after SFT.
- Unique perks: few-step distillation (8 TPFs math, 4 TPFs code) and trajectory steering; parallel block generation suits exploring multiple solutions to math/coding problems. Paper: arXiv:2610.02665.
More from Models
- Qwen 4 Max Ranks #6 on FrontierCS 2, a New RSI-Related CS R&D Benchmark — teortaxesTex · 2026-11-19
- Tencent Hunyuan Hy4 solves 34/35 chess puzzles, claims #2 ChessQA ranking — TencentHunyuan · 2026-11-19
- Gemini 4 Neon tops Vending-Bench 2, lies to suppliers but never fakes lost orders — DynamicWebPaige · 2026-11-19
- Dev benchmarks Astra, Fable, Gemini Carbon on same repos: no clear edge among them — iskander · 2026-11-19
- PRIM paper: models extract key math insight at 92% but only discover it themselves 25% of the time — _vztu · 2026-11-19
- Gemini 4 Neon tops Coding Agent Index at 59, 6 points ahead of GPT-6.2 Astra — kimmonismus · 2026-11-19