AMD, NVIDIA, Google ship vLLM gains in 3 days; MI355X gives 87.4% more tokens per dollar
woosuk_k · x · 2026-10-19
Over the last three days AMD, NVIDIA and Google all landed significant vLLM improvements, benchmarked with SemiAnalysis InferenceX on Kimi K3 (2.8T), with the inferact team shepherding many optimizations.
Charts show: MI355X (vLLM) delivering 87.4% more tokens per $1 TCO than B300 at similar interactivity; NVIDIA GB300 NV72 + Dynamo vLLM hitting 423 tok/s/user P90 on the 2.8T model with a 1.68x Megakernel speedup vs Sept 15; and Google TPUv7 joining the Kimi K3 preview benchmark alongside B200/B300/GB200 curves.
Related event: AMD, NVIDIA and Google Rush to Optimize vLLM Inference(2 posts)→
More from Infra
- DFlash2 authors ship Qwen3.8-Flash speculative decoding model, 20% faster than MTP at concurrency 1 — rerri · 2026-10-19
- DOE lends $4.4B to restart three nuclear plants to feed AI data centers — emmanuelvivier · 2026-10-19
- China proposes export controls on 23 AI technologies, banning 2T-parameter models — emmanuelvivier · 2026-10-19
- DeepSeek-V4.1-Flash quantized to 2-bit runs on two RTX PRO 6000s, KL eval published — TheZachMueller · 2026-10-19
- Alibaba Cloud shows Omarchy running on Wuying AI cloud PCs for enterprise agents — AIFlow_ML · 2026-10-19
- NVIDIA paper shows cross-model KV cache transfer skipping prefill, 2.7–25x faster than recomputing — solyarisoftware · 2026-10-19