DiffusionGemma-Jev lands in vLLM: single-step structured answers with confidence scores
bodonoghue85 · x · 2026-09-23
vLLM has merged a structured generation mode for DiffusionGemma-Jev (PR #57250, driven by mmastrac), targeting yes/no, multiple-choice, and scored questions where every answer comes with a confidence score.
The technique: vLLM seeds a canvas with the response template, keeps only the answer slots noisy, then reads a probability distribution from every slot in a single denoising step — no autoregressive decoding.
Google's Gemma team paired the release with a one-command Cloud Run deployment: 35-60ms single-step latency, 100-123 req/sec at batch@32, roughly $3/hr with scale-to-zero when idle, so no GPU of your own is needed.
More from Infra
- Rivet launches SQLite Forking: 20ms agent-thread forks with zero data copying — samgoodwin89 · 2026-11-11
- French netizen proposes datacenters pay local residents compensation — clemnt · 2026-11-11
- GPU Rental Prices Hit Record Highs: B200 at $6.97/hr, H100 at $3.06 — inductionheads · 2026-11-11
- A Datacenter in Every Office: Building Work-Safe Local AI Racks — dee_hw · 2026-11-11
- "Compute is about power": the AI race's real bottleneck is energy, not chips — _sholtodouglas · 2026-11-11
- CoreWeave CEO says he canceled his 10b5-1 plan, halting insider selling — firstadopter · 2026-11-11