SemiAnalysis teases TPUv7 TCO data: $0.181/M tokens vs $0.222 B200
firstadopter · x · 2026-10-14
SemiAnalysis has a batch of TPU measurements in the pipeline. Slides from InferenceX show: TPUv7 (Ironwood) external TCO of $0.181/M total tokens running Qwen3.5 397B FP8 (8K/1K, STP, vLLM on TorchTPU) vs $0.222 on B200 and $0.276 on B300; internal TCO of $1.03 per TPU chip-hour. On one chart comparing TPU/NVIDIA/AMD end-to-end latency vs cost, B200 and GB200 lead at low latency while TPUv7 leads at higher latency. Follow-ups planned: TPUv8i native FP4, disaggregated prefill/decode, MTP, and AgentX on TPUv7.
More from Infra
- Musk backs Morgan Stanley call: SpaceX's Distributed Inference Cloud is the overlooked growth story — elonmusk · 2026-10-14
- Valtz: open-source Mac app unifies local image/video/audio AI generation on 16GB machines — TgoAI · 2026-10-14
- AMD, Intel, Google, Microsoft, HPE, Red Hat share stage at OCP Global Summit on scaling AI — firstadopter · 2026-10-14
- Musk: Funding AI data centers is easy, building them is brutally hard — even for SpaceX — elonmusk · 2026-10-14
- Microsoft's Maia 300 AI board shown off: 4 XPUs per tray — BenBajarin · 2026-10-14
- Ayar Labs and wiwynn show a working CPO system with GUC test XPUs on the show floor — BenBajarin · 2026-10-14