vLLM Achieves Major Throughput Gains on AMD GPUs
vLLM Blog · rss · 2026-07-13
A vLLM blog post details how to train, quantize, and deploy an EAGLE-3 speculative decoding draft model on AMD Instinct GPUs using vLLM + AMD Quark.
The core result is a significant boost in inference throughput, with reported figures including:
- Kimi-K2.5: Up to 2.00x throughput increase
- MiniMax-M2.5: Up to 1.79x throughput increase
The article focuses less on the models themselves and more on combining speculative decoding, quantization, and the serving stack to achieve higher serving efficiency on AMD hardware.
More from Infra
- Grid queues make power architecture an AI-factory decision; Exowatt and Redwood offer fixes — ingliguori · 2026-10-27
- Two CVSS 9.9 Wasmtime flaws allow sandbox escape, one ARM64-only since 2020 — basedjensen · 2026-10-27
- AI Infra Meme: While Countries Need TW-Scale Power for Datacenters, People Argue About Lightbulbs — mgostIH · 2026-10-27
- NVIDIA reportedly stops taking RTX 5090 orders from board partners, even backorders — chemist_slime · 2026-10-27
- Legora runs 2,000 parallel agent evals on Modal Sandboxes for legal AI — blelbach · 2026-10-27
- Cloud release velocity scores: AWS 95, Azure 90, Google Cloud 82 — DavidLinthicum · 2026-10-27