Cutting 12-Hi to 8-Hi lifts bandwidth-per-GB 50%: the Rubin Ultra HBM math
AccBalanced · x · 2026-09-06
SemiAnalysis continues the math: capacity ≈ stacks × dies per stack × GB per die, while bandwidth ≈ stacks × interface width × pin speed. Stack height buys capacity, not bandwidth.
- Cutting 12-Hi to 8-Hi drops a third of the DRAM dies — the most supply-constrained silicon on the BOM — while bandwidth holds or ticks up
- Bandwidth per GB of capacity rises 50%, meaningfully improving $/bandwidth right as HBM prices hike
- Inference is bandwidth bound: every decoded token re-reads weights and the KV cache, so Nvidia's trade favors $/bandwidth over $/capacity
Related event: SemiAnalysis Explains Why Nvidia Cut Rubin Ultra HBM to 8-Hi(2 posts)→
More from Infra
- PhD students in some countries get B200s on demand, unlike their US peers — HildeKuehne · 2026-10-23
- Nvidia's Berlin GTC spotlights humanoid robots and Vera Rubin AI supercomputing — emmanuelvivier · 2026-10-23
- Lenovo adds 5 AI Library use cases, claims 1-week deployment and up to 70% lower token cost — shashib · 2026-10-23
- Your kernel doesn't need to beat cuBLAS on every shape, argues GPU dev — blelbach · 2026-10-23
- llama.cpp team launches Llama: one-click local AI on your Mac menu bar — Hugging Face · 2026-10-23
- China's daily token use hits 500T, up 4x as average price doubles to ¥13.46 — teortaxesTex · 2026-10-23