DiffusionGemma Technical Report: Parallel Text Gen at 1,500 Tokens/sec on Single H100
kastnerkyle · x · 2026-08-06
The DiffusionGemma team has released a technical report for an experimental open-weight language model. By converting Gemma 4 (25.2B total parameters, 3.8B activated) into a discrete diffusion model, it breaks the sequential decoding bottleneck of traditional autoregressive (AR) models.
Core Mechanism & Performance
- Parallel Generation: The model iteratively refines blocks of 256 tokens in parallel, generating around 20 tokens per forward pass.
- Ultra-fast Inference: Achieves roughly 1,500 output tokens per second on a single NVIDIA H100 GPU, substantially faster than AR models even with state-of-the-art speculative decoding.
Training Pipeline
Instead of training from scratch, the model uses a compute-efficient two-stage fine-tuning pipeline requiring less than 10% of the original AR model's total training token budget:
- Supervised Fine-Tuning (SFT): To teach bidirectional denoising.
- Reinforcement Learning & Sampler Distillation: To jointly improve generation quality and inference efficiency.
Furthermore, the model retains support for multimodal inputs, long contexts, and thinking mode. It remains capable of AR generation with only minor performance degradation, suggesting a path toward hybrid diffusion-AR decoding.
Related event: DeepMind Releases DiffusionGemma for Ultra-Fast Parallel Decoding(8 posts)→
More from Models
- Top AI Models Score Under 50% on New Math Figure Reasoning Benchmark — prof_g · 2026-08-06
- Ethan Mollick: LLMs Improve at Following Instructions but Exercise More 'Judgement' — emollick · 2026-08-06
- Frontier LLMs Perform Best in Week 1: Dev Calls for a Proof of Model Standard — sull · 2026-08-06
- Polymarket Favorites Anthropic at 69% to Win AI Race by 2026 — Polymarket · 2026-08-06
- DeepSeek Tries, Opus Finishes: A Practical AI Coding Model Workflow — gandamu_ml · 2026-08-06
- Palantir Claims Vanilla NVIDIA Nemotron Outperforms Frontier Models in 24 Hours — eliano · 2026-08-06