vLLM benchmarks MTP, EAGLE-3, and other speculative decoding methods on AMD GPUs
vllm_project · x · 2026-08-28
vLLM published a new blog post deeply analyzing and benchmarking 5 speculative decoding methods on AMD Instinct MI300X and MI355X GPUs. The conclusion is that there is no universal winner; the best choice depends on the model, workload, and speculation depth.
Methodology and Scope:
- The post explains how 5 methods work: native MTP, Gemma 4 MTP, EAGLE-3, DFlash, and DSpark.
- It details how to enable and tune these methods within vLLM.
- Benchmarks were conducted across Gemma, Qwen, Kimi, and MiniMax models.
Key Findings:
- Speculative decoding allows vLLM to verify multiple drafted tokens in a single target-model pass.
- The impact on output-token throughput varies significantly based on the drafting method, proposal length, model family, and workload.
- Acceptance behavior also plays a crucial role in final performance.
Related event: vLLM Benchmarks Five Speculative Decoding Methods: No Universal Winner(3 posts)→
More from Infra
- Jensen Huang: chips are just tools — sovereign AI means owning your data and knowledge — rohanpaul_ai · 2026-10-15
- Huawei Ascend to demo pretraining DeepSeek V4 on Ascend 950 supernodes in livestream — teortaxesTex · 2026-10-15
- Cloudflare data: after strikes on Yandex data centers, malicious bots simply moved networks — banteg · 2026-10-15
- OpenAI is hiring for its UK infrastructure team, offering end-to-end ownership — whoiskatrin · 2026-10-15
- XERJ indexes code, docs and PDFs in one command, saving agents 40x tokens vs grep — xerj-org · 2026-10-15
- TokenSpeed, a 'speed-of-light' LLM inference engine for Blackwell and Rubin, hits 2.3k stars — lightseekorg · 2026-10-15