NVIDIA releases Nemotron-Labs-Sigma: 8B/3B continuous diffusion LMs with block-parallel decoding
jacek2023 · reddit · 2026-11-17
NVIDIA released Nemotron-Labs-Sigma (8B/3B, base + instruct) on Hugging Face — continuous diffusion language models that denoise Gaussian noise in a 16-D continuous token-embedding space instead of generating token by token, with the denoiser and embeddings trained jointly via likelihood optimization.
Key details:
- Block-parallel decoding: 64-token blocks refined with DDPM/DDIM samplers, self-conditioning, and standard KV cache reuse for committed tokens
- Fewer denoising steps trade a little quality for large speedups; smooth continuous representation suits few-step distillation and trajectory steering
- Ministral-style decoder-only Transformer (GQA, SwiGLU, RoPE), 4,096-token context
- Claimed reasoning accuracy competitive with state-of-the-art masked (discrete) diffusion LMs; plain PyTorch, loads via standard transformers
Base models target research on parallel decoding, fine-tuning and distillation; instruct variants are for chat. Commercial use allowed.
More from Models
- NVIDIA publishes full Cosmos3 collection on Hugging Face — NVIDIAAI · 2026-11-17
- NVIDIA Cosmos world models hit 20 million downloads on Hugging Face — NVIDIAAI · 2026-11-17
- Orion foundation model predicts gaze: finds 2x UX bugs, cuts computer-use latency 31% — Scobleizer · 2026-11-17
- Alibaba launches Qwen4-Max: 1M context MoE model tops Chinese models on RSI-Exam — HuaxiuYaoML · 2026-11-17
- Cua and Scale AI benchmark computer-use agents: 97% non-intrusive on Omarchy vs 40% on Ubuntu — TianbaoX · 2026-11-17
- 50 sequential edits stress test: Seedream drifts least at 36.6, Nano Banana at 74.7 — aziz4ai · 2026-11-17