SemiAnalysis: 4-hi HBM wins on $/bandwidth as Rubin Ultra drops to 192GB 8-hi stacks
maksym_andr · x · 2026-09-15
SemiAnalysis argues the ever-taller HBM stacking trend is breaking.
- Next-gen accelerators standardize on 8-hi instead of 12-hi: Nvidia's Rubin Ultra drops to 192GB from 288GB per GPU (SemiAnalysis first reported), and the supply chain is prepping 8-hi even though 16-hi was the expectation under a year ago;
- For inference workloads where bandwidth matters most, 4-hi HBM offers the best $/bandwidth and lowest cost per token; beyond a capacity threshold, extra bits carry BOM penalty with diminishing returns;
- Major labs' ASIC teams want this from HBM4 onward, alongside maximizing tokens/Watt under DC power constraints.
The shift also hints at relief for today's extreme DRAM shortage.
Related event: SemiAnalysis: HBM Stacking Trend Reverses Toward 8-hi(2 posts)→
More from Infra
- Aristotle system machine-checks Ibex RISC-V core MUL operation formally — satnam6502 · 2026-10-31
- Clover-35B-A3B Quantized to 29.4 GB Runs at ~42 tok/s on a 64 GB Mac — viveky259 · 2026-10-31
- Liquid Agent brings fully local LFM3 agent to Linux with one-command install — JosephJacks_ · 2026-10-31
- A fully local office AI stack: MiniCPM, Qwen3, LFM2.5 and Whisper — HowDevelop · 2026-10-31
- a16z: GPU rental prices up 106% in 3 months while token prices fall 58% — a16z · 2026-10-31
- vLLM nightly adds PLE disk offload, with a telling [Qwen4Exp] tag on the PR — rmhubbert · 2026-10-31