vLLM benchmarks MTP, EAGLE-3, and other speculative decoding methods on AMD GPUs
vllm_project · x · 2026-08-28
vLLM published a new blog post deeply analyzing and benchmarking 5 speculative decoding methods on AMD Instinct MI300X and MI355X GPUs. The conclusion is that there is no universal winner; the best choice depends on the model, workload, and speculation depth.
Methodology and Scope:
- The post explains how 5 methods work: native MTP, Gemma 4 MTP, EAGLE-3, DFlash, and DSpark.
- It details how to enable and tune these methods within vLLM.
- Benchmarks were conducted across Gemma, Qwen, Kimi, and MiniMax models.
Key Findings:
- Speculative decoding allows vLLM to verify multiple drafted tokens in a single target-model pass.
- The impact on output-token throughput varies significantly based on the drafting method, proposal length, model family, and workload.
- Acceptance behavior also plays a crucial role in final performance.
More from Infra
- Pure MLX Engine Hits 65 tok/s, Cuts Model Size in Half — EyalToledano · 2026-08-28
- Qwen3.8 Variants: 8-bit Model Cuts Memory by 27% — EyalToledano · 2026-08-28
- 180B-Class Qwen3.8 Model Runs on Just 39GB Memory — EyalToledano · 2026-08-28
- Nvidia's Moat Lies in Scale and Supply Chain Lock-in — firstadopter · 2026-08-28
- Microsoft Foundry launches AI Gateway for unified traffic and cost governance — davemccollough · 2026-08-28
- Autonomous launches $26,100 dual-RTX 5090 AI workstation, 'The Diablo' edition — dee_hw · 2026-08-28