Google TPUv7 beats Nvidia B200/GB300 on inference cost in SemiAnalysis benchmarks
SumitGup · x · 2026-11-15
SemiAnalysis's InferenceMAX nightly benchmarks show Google's TPUv7 (Ironwood) matching or beating Nvidia B200/GB300 on many inference tasks, running open-source models under realistic conditions including long-context and agentic workloads.
- In FP8, TPUv7 delivers 1.2×–2.7× more tokens per dollar than Nvidia parts, and up to 50% better performance per dollar in some setups.
- Raw throughput is often comparable; Google's edge comes from lower total cost of ownership (chip + system). Nvidia still leads on some FP4 workloads and highly optimized racks.
- The gap has narrowed thanks to better silicon, software (TorchTPU, vLLM/SGLang support), and economics — results depend on model, precision, and serving setup.
More from Infra
- Court Refuses to Dismiss DRAM Supply Conspiracy Suit Against Samsung, SK Hynix, Micron — SumitGup · 2026-11-15
- 2009 video shows Jensen Huang sizing CUDA at billions when NVDA traded at $0.2 — GregoryDiamos · 2026-11-15
- Ex-NVIDIA engineer recalls 2012: Jensen Huang backed the deep learning cluster behind Deep Speech — GregoryDiamos · 2026-11-15
- EU's Open Model EUROPA Eyes ~1T Params, Per-GPU Throughput Up 14x — mawcopolow · 2026-11-15
- Buy vs. Rent vs. API for Inference: Utilization Decides Your $/GPU-hour — Panther1537 · 2026-11-15
- Why 128-bit address pointers make no sense: x86 still uses only 48 bits of 64 — srimisra · 2026-11-15