vLLM brings native PyTorch serving to TPUs with open-source TorchTPU
PyTorch · x · 2026-10-22
vLLM announced vLLM-TorchTPU, built with Google and Inferact, enabling native PyTorch model serving on TPUs with support for prefix caching, speculative decoding, and disaggregated serving.
Google Cloud also open-sourced TorchTPU, with ecosystem support spanning vLLM, SGLang, llm-d, TorchTitan, Hugging Face, and Ray. The news was shared around PyTorch Conf 2026.
Related event: vLLM Launches TorchTPU: Native PyTorch Inference on Google Cloud TPU(2 posts)→
More from Infra
- DeepSeek has open-sourced every layer of an AI lab, from weights to inference engine — examachine · 2026-10-22
- Redditor converts missing models to OpenVINO IR and takes requests — Sudden_Topic5154 · 2026-10-22
- Meta plans up to $145B in capex for 2026, more than double 2025 — Polymarket · 2026-10-22
- pi ships rec inference engine, runs locally on M5 Max in two lines — _rockt · 2026-10-22
- Researcher warns GitLab flaw is 'really bad,' urges users to patch now — basedjensen · 2026-10-22
- PyTorch Con Demo Theater: 13 talks including Wan 2.2 video generation sped up 53.6x — PyTorch · 2026-10-22