Epoch AI's new Inference Performance Index shows GB300 beats H100 by up to 72x
scaling01 · x · 2026-11-10
Epoch AI introduced the Inference Performance Index (IPI), arguing that FLOP/s alone understates GB300's inference advantage over H100 by 10x. IPI factors in memory bandwidth and inter-chip communication, which determine how many users or agents a chip can serve concurrently at a given output speed.
Key scores (GB300 = 1):
- AMD MI455X: 2.89, the highest
- Nvidia VR200: 2.18
- Nvidia GB300: 1.00 (baseline)
- Google TPU v8i: 0.91, GB200: 0.90
- TPU v7: 0.74, Microsoft Maia 200: 0.68
- AMD MI355X: 0.55, Nvidia B300: 0.54
- Nvidia H200: just 0.12, and H100: 0.04 — newer chips offer up to 72x H100's inference capacity
The index enables direct chip-to-chip comparison and estimating total inference capacity of a data center, or even the world.
More from Infra
- Together Compiler hits 2.7x throughput gains on Kimi K3 at same per-user speed — togethercompute · 2026-11-10
- Bloomberg: Nvidia's Jensen Huang Has ~60 Direct Reports and No Named Successor — rohanpaul_ai · 2026-11-10
- Cache optimization pushes DeepSeek economy hit rate to 97.54% — CalmObligation554 · 2026-11-10
- MoE models on consumer hardware could erode cloud AI's grip, argues Redditor — FudgeEducational2038 · 2026-11-10
- Fireworks CEO discloses October breach: third-party cloud access led to customer data impact — lqiao · 2026-11-10
- Report: Anthropic pushing governments to require firmware locks blocking unapproved models on Western GPUs — rickasaurus · 2026-11-10