Meta engineer confirms Google TPUs now run AI inference in Meta production
dotey · x · 2026-10-21
Google's developer blog introduced native PyTorch backends for vLLM and SGLang on TPU, quoting Meta's PyTorch-TPU lead Zongwei Zhou: "SGLang-TorchTPU is running in Meta's production environments today" — the first first-hand confirmation that Meta runs inference on Google TPUs. SemiAnalysis previously reported Meta as a TPU customer, and The Information reported a multi-year, multi-billion-dollar TPU rental deal in February. Scale and products involved remain undisclosed.
Technically, TorchTPU makes TPU a native PyTorch device with standard tensors, eager execution or torch.compile via XLA, keeping vLLM/SGLang's worker model, scheduler, KV cache manager and API server intact with TPU implementations underneath, plus optimized Pallas kernels — a direct challenge to NVIDIA's CUDA moat.
More from Infra
- Morgan Stanley: hyperscaler AI capex to hit $3.4T across 2027-2028 with leases included — FinanceYF5 · 2026-10-21
- MiniMax's M3.1 Flash talk: a faster model isn't faster if the job isn't — AI Engineer · 2026-10-21
- Training Factory open-sources QLoRA fine-tuning on 4-bit CQ-quantized models — textclf · 2026-10-21
- DeepSeek open-sources DeepInfer: 155T tokens/day, $38/GPU-hour, >80% margin — Latent Space · 2026-10-21
- Cerebras WSE-3 Turbo packs 4T transistors and 250 PFLOPS into a wafer-scale chip — GregoryDiamos · 2026-10-21
- Anthropic reportedly built Scion to replace Git for 30,000 daily agent merges — Capatain_CV · 2026-10-21