DeepMind bakes speculative decoding into Gemma 4: up to 3.1x server speedups, 2.2x on smartphone GPUs

dair_ai · x · 2026-11-22

dair-ai breaks down Google DeepMind's new paper on speculative decoding for Gemma 4. A shared multi-token prediction drafting head reads the backbone's activations and KV cache, removing the separate prefill step, plus a clustered vocabulary projection to cut draft cost.

Key results:

Big deal for agents generating long outputs: lower latency and serving cost.

Original post →

More from Infra

Infra channel →