SemiAnalysis: Google TPUv7 Delivers Up to 50% Better Performance-per-Dollar Than B200/B300 for Inference

vllm_project · x · 2026-11-06

vLLM project amplified SemiAnalysis's deep-dive "The Iron(wood) Throne" on Google TPU challenging NVIDIA's inference dominance:

Oracle, OpenAI, Anthropic, and Meta have all become TPU customers since SemiAnalysis's Nov 2025 prediction
Google is attacking NVIDIA's software moat by funding upstream maintainers: Meta PyTorch, vLLM (Inferact, RedHat), SGLang (RadixArk), llm-d, Ray
Previously published third-party TPUv7 results: on FP8 Qwen 3.5 397B single-turn workloads, Ironwood delivered up to 50% better performance per dollar than B200/B300
The piece releases the first official long-context, multi-turn agentic inference (AgentX) benchmark results on TPUv7, covering GLM-5.2, Qwen3.5 397B, and Kimi K3 via the Kimi K3 megakernel and vLLM TorchTPU
Also covers NVIDIA working nights and weekends, Jensen's counter-submission, and Meta Muse Spark open weights

More from Infra