DeepMind Releases DiffusionGemma: Discrete Diffusion for Ultra-Fast Text Generation
deepmind · hf · 2026-08-04
Google DeepMind released the technical report for DiffusionGemma, an experimental open-weight language model. Unlike conventional autoregressive (AR) models that decode token-by-token, it uses discrete diffusion to iteratively refine blocks of 256 tokens in parallel, avoiding the sequential decoding bottleneck.
Key Highlights:
- Efficient Training: Fine-tuned from the Gemma 4 model (25.2B total, 3.8B activated parameters), its two-stage pipeline uses fewer than 10% of the original AR model's training token budget.
- Ultra-Fast Inference: Generates around 20 tokens per forward pass and achieves roughly 1,500 output tokens per second on a single NVIDIA H100 GPU, substantially faster than AR models even with state-of-the-art speculative decoding.
- Capability Retention: Retains support for thinking mode, multimodal inputs, and long contexts. It also remains capable of AR generation despite diffusion fine-tuning, suggesting a path toward hybrid diffusion-AR decoding.
More from Models
- 15 Attorneys General Demand OpenAI Preserve AI 'Self-Jailbreak' Notes — jedisct1 · 2026-08-04
- Analyzing Kimi-K3 Open Source: Data Flywheels Are the New Moat — karminski3 · 2026-08-04
- US Frontier Labs Tout 'Efficiency' as Chinese Models Drop at Throwaway Prices — deliprao · 2026-08-04
- Qwen 3.8 Coding Test: Nearly Matches K3 at Half the Price — bindureddy · 2026-08-04
- Fable and Gemini Vision Models Misled by Place Names in Geo-Guessing Fail — HanchungLee · 2026-08-04
- Opinion: Masked Language Modeling Was a Detour, Autoregressive Was Inevitable — jxmnop · 2026-08-04