vLLM at PyTorchCon 2026: KV cache and kernel optimizations across TPU, AMD, and NVIDIA
PyTorch · x · 2026-10-21
Simon Mo, co-founder and CEO of Inferact, presented the state of vLLM at PyTorch Conference 2026 under the theme "Scaling Open Frontier Inference Infrastructure."
Highlights:
- Ongoing optimizations in KV cache management and kernels
- Support across TPU, AMD, and NVIDIA hardware backends
- Addresses performance and scalability challenges as LLMs grow in size, context length, and architectural complexity
More from Infra
- Pichai: everyone is massively short on AI compute — 'everyone wishes you had two times what you had planned' — firstadopter · 2026-10-21
- Stanford CS336 Lecture 6: fused Triton kernels cut softmax memory traffic ~4x — stanfordnlp · 2026-10-21
- General Compute partners with Positron AI: Atlas memory-centric hardware ships Q1 2027 — aftahi_ai · 2026-10-21
- AI bond issuance hits $419B in 2026, up 90% from all of 2025 — SumitGup · 2026-10-21
- AI Obsession Is Making Enterprises Neglect Cloud FinOps, Expert Warns — DavidLinthicum · 2026-10-21
- DeepSeek bottleneck is compute, not demand: 1GW cluster could train a far stronger model, says leak account — teortaxesTex · 2026-10-21