vLLM Tests Show TPUv7 Hits 700 tok/s/user, 56% Faster Than NVIDIA GB200 NVL72
woosuk_k · x · 2026-09-24
Per SemiAnalysis, vLLM maintainers demonstrated that TPUv7 can reach 700 tok/s per user on Kimi K3 via Megakernel optimization — 56% better throughput than NVIDIA's GB200 NVL72. SemiAnalysis argues that TPU software externalization is now moving at full speed, making this a key inflection in the accelerator landscape to watch.
Related event: First TPU Inference Megakernel Open-Sourced, Beats GB200(3 posts)→
More from Infra
- French netizen proposes datacenters pay local residents compensation — clemnt · 2026-11-11
- GPU Rental Prices Hit Record Highs: B200 at $6.97/hr, H100 at $3.06 — inductionheads · 2026-11-11
- A Datacenter in Every Office: Building Work-Safe Local AI Racks — dee_hw · 2026-11-11
- "Compute is about power": the AI race's real bottleneck is energy, not chips — _sholtodouglas · 2026-11-11
- CoreWeave CEO says he canceled his 10b5-1 plan, halting insider selling — firstadopter · 2026-11-11
- UK chip start-up Fractile hits $7bn valuation, per FT — evisoft · 2026-11-11