vLLM ships nightly support for Google's DiffusionGemma, a block-diffusion MoE with ~1.9x throughput
vllm_project · x · 2026-09-23
vLLM announced nightly support for Google's DiffusionGemma-26B-A4B-it with a deployment recipe.
- Architecture: a block-diffusion LM on Gemma 4's MoE backbone — 26B total / 4B active params, 128 fine-grained experts with top-8 routing.
- Generation: instead of autoregressive decoding, it denoises 256-token canvas blocks iteratively (up to 48 steps/block), trading higher time-to-first-token for 1.9x per-request throughput.
- Features: multimodal (text + images), thinking mode, and function calling.
- Deployment gotchas: diffusion state buffers pre-allocate maxseqs × canvaslength × vocabsize tensors — with Gemma's 262K vocab you must set --max-num-seqs 4 or risk OOM; use --generation-config vllm.
Requires the vllm/vllm-openai:nightly image, vLLM ≥0.24.0.
More from Infra
- A Redditor built a dual AMD R9700 local inference rig and crowdsources tuning advice — Current-Ticket4214 · 2026-09-23
- OpenRouter Batch API spans 71 models, auto-routes to cheapest provider — jeff_weinstein · 2026-09-23
- Fully Offline NotebookLM Alternative: Ollama + Open WebUI + RAG Stack Suggested — betobagio · 2026-09-23
- Critic calls Googlebook hardware 'unambitious': Intel opted out of consumer NPUs — julianharris · 2026-09-23
- Frontier releases suggest open models like Kimi K3 are severely undertrained — zeeshanp_ · 2026-09-23
- 65-70% of LLM speedup claims unimpressive, says dev: it's mostly speculative decoding and prompt lookup — teortaxesTex · 2026-09-23