AMD, NVIDIA and Google All Ship Major vLLM Optimizations in Three Days
woosuk_k · x · 2026-10-19
Simon Mo reports that over the last three days, AMD, NVIDIA and Google all contributed significant performance improvements to the open-source vLLM inference framework, measured with SemiAnalysis's standard benchmark on K3, with the Inferact team shepherding many of the optimizations. The repost quips that the real hardware moat turns out to be working with vLLM — the open inference stack is now the common battleground for chip vendors.
Related event: AMD, NVIDIA and Google Rush to Optimize vLLM Inference(2 posts)→
More from Infra
- Dense 32B open model runs 239K tokens in 16GB VRAM with only 18 of 72 layers keeping KV cache — solyarisoftware · 2026-10-19
- Open-source SwiftUI dashboard brings a Docker Desktop-like UI to Apple's native container runtime — AIFlow_ML · 2026-10-19
- Strata runs on Windows with dual AMD GPUs despite no official support: ~1350pp on a 9070XT — Ok-Importance-3529 · 2026-10-19
- A frozen 16MB vocabulary replaces learned embeddings in small and ternary LMs — Sai Kiran Bathula · 2026-10-19
- Thinking Machines explains LLM nondeterminism and ships batch-invariant kernels for reproducibility — AccBalanced · 2026-10-19
- Mapping the compute exchange: power markets and crypto infrastructure vie to supply compute liquidity — AccBalanced · 2026-10-19