vLLM ships nightly support for Google's DiffusionGemma, a block-diffusion MoE with ~1.9x throughput
vllm_project · x · 2026-09-23
vLLM announced nightly support for Google's DiffusionGemma-26B-A4B-it with a deployment recipe.
- Architecture: a block-diffusion LM on Gemma 4's MoE backbone — 26B total / 4B active params, 128 fine-grained experts with top-8 routing.
- Generation: instead of autoregressive decoding, it denoises 256-token canvas blocks iteratively (up to 48 steps/block), trading higher time-to-first-token for 1.9x per-request throughput.
- Features: multimodal (text + images), thinking mode, and function calling.
- Deployment gotchas: diffusion state buffers pre-allocate maxseqs × canvaslength × vocabsize tensors — with Gemma's 262K vocab you must set --max-num-seqs 4 or risk OOM; use --generation-config vllm.
Requires the vllm/vllm-openai:nightly image, vLLM ≥0.24.0.
More from Infra
- French netizen proposes datacenters pay local residents compensation — clemnt · 2026-11-11
- GPU Rental Prices Hit Record Highs: B200 at $6.97/hr, H100 at $3.06 — inductionheads · 2026-11-11
- A Datacenter in Every Office: Building Work-Safe Local AI Racks — dee_hw · 2026-11-11
- "Compute is about power": the AI race's real bottleneck is energy, not chips — _sholtodouglas · 2026-11-11
- CoreWeave CEO says he canceled his 10b5-1 plan, halting insider selling — firstadopter · 2026-11-11
- UK chip start-up Fractile hits $7bn valuation, per FT — evisoft · 2026-11-11