PyTorch ships TorchTPU: vLLM and SGLang now run natively on Google TPUs
PyTorch · x · 2026-10-02
PyTorch announced TorchTPU, a PyTorch-native TPU backend that lets the vLLM and SGLang serving engines run natively on TPUs while preserving schedulers, batching, OpenAI-compatible APIs, and torch.compile workflows. Google and Meta engineers will share production lessons — compile times, device placement under tracing, KV cache layout, cross-chip collectives — at PyTorch Conference North America in San Jose.
Related event: PyTorch Unveils TorchTPU, Enabling Native vLLM and SGLang on TPU(2 posts)→
More from Infra
- Nvidia scandal isn't slowing down: Huang's near-$100B power web under strain — The Verge AI · 2026-11-16
- Volume rules chips: halvarflake says Intel lost by outsourcing demand — TheKanter · 2026-11-16
- Taipei 101 lights up "Taiwan ❤️ Jensen" as tech sector rallies behind Jensen Huang — liu_mingyu · 2026-11-16
- Nous exec: avoid GPT 900k context variant — extra charges kick in past 272k — Teknium · 2026-11-16
- CUDA matmul optimization walkthrough: naive kernel to 93.7% of cuBLAS — RisingSayak · 2026-11-16
- Dev runs 120B models on iOS by streaming MoE weights from flash in llama.cpp — Chance-Resource-4970 · 2026-11-16