Google open-sources TorchTPU with Meta: up to 52% MFU on 8,000 Ironwood TPUs
智东西 · wechat · 2026-10-21
Google has open-sourced TorchTPU, a native TPU software stack co-developed with the Meta PyTorch team, letting developers run existing PyTorch models on Google TPUs with minimal code changes — a move seen as a challenge to NVIDIA's GPU dominance. Training was validated on up to 8,000 Ironwood TPUs, hitting up to 52% MFU on Qwen3-8B with near-linear scaling to 8,192 chips.
Ecosystem integrations include TPU backends for vLLM, SGLang, and llm-d; India's Zenteiq.ai more than quadrupled inference throughput using the llm-d-based GKE Inference Gateway, lifting daily token generation from 1.47B to 5.97B on 1,000 TPUv6e chips. TorchTitan now has production-grade TPU integration with recipes for Llama 3, Qwen3, and FLUX.1 models; VeRL and Ray also gained TPU support. On the edge, LiteRT joins the PyTorch ecosystem via LiteRT-Torch for one-call export to Android, iOS, Web, and IoT devices. In HuggingFace Transformers v5, devicemap="tpu" deploys models to TPUs.
More from Infra
- pi ships rec inference engine, runs locally on M5 Max in two lines — _rockt · 2026-10-22
- Researcher warns GitLab flaw is 'really bad,' urges users to patch now — basedjensen · 2026-10-22
- Meta Beats With $64.61B Revenue, Raises 2026 CapEx Guidance to $135-145B — RihardJarc · 2026-10-22
- PyTorch Con Demo Theater: 13 talks including Wan 2.2 video generation sped up 53.6x — PyTorch · 2026-10-22
- TorchTitan adds 5D parallelism for DeepSeek-V4, Qwen3.5, Kimi K3; Muon optimizer 79% faster — PyTorch · 2026-10-22
- JNTC cuts TGV glass substrate copper filling from a day to minutes, targets 2027 mass production — toptickcrypto · 2026-10-22