Hot Chips Spotlights AI Memory Bottleneck: 3D DRAM and zHBM Converge
Hot Chips focused on the memory bottleneck created by the AI compute boom: HBM demand is projected to grow 24x amid a global shortage, and the industry is advancing along two paths—continued HBM evolution and 3D DRAM/zHBM—expected to coexist long-term given their respective strengths.
Confirmed
- @firstadopter relayed Hot Chips data: AI compute growth drives a 24x HBM demand increase with severe shortage; a Samsung executive stated next-gen AI architectures need substantially more HBM capacity.
- SK Hynix executives emphasized HBM's dominance in current AI compute architectures and showed progress on hybrid bonding and I-HBM to address SRAM's capacity limits.
- Samsung unveiled its zHBM roadmap, claiming next-gen zHBM could cut power by 70% versus HBM5.
- @GavinSBaker, @elonmusk and @sedricm summarized the consensus: every accelerator company will likely pursue some variant of 3D DRAM or zHBM, with large players running both tracks in parallel; 3D DRAM and zHBM sparked post-conference debate as inference-architecture optimizations, and Vikram Sekar reiterated interest in d-Matrix's approach.
- On 3D DRAM merits: @zephyrz9 noted superior efficiency and bandwidth over HBM4, but stacking logic atop memory poses major power-delivery challenges; @bookwormengr added that placing DRAM under/over the logic die yields higher bandwidth than 2.5D-packaged HBM though with smaller capacity, and views zHBM as HBM's third evolutionary stage.
- Per @firstadopter, the shortage stems from surging AI/HPC demand, supply-chain bottlenecks and lagging capacity expansion; vendors are ramping output.
Why it matters
- Memory, not compute, is becoming the binding constraint on AI infrastructure; a 24x demand surge amid shortage means the memory supply chain will directly gate future AI chip shipments.
- If 3D DRAM and zHBM land, they would substantially surpass current HBM in bandwidth and power, but power delivery and capacity trade-offs remain engineering challenges; this technology contest will shape accelerator architectures for years.
2026-08-24 ~ 2026-08-24 · 11 related posts
Primary sources
- SK Hynix Hypes HBM Dominance; Samsung zHBM Aims for 70% Power Cut — firstadopter ·
- HBM memory demand up 24X causes global shortage — firstadopter ·
- Every accelerator firm will likely adopt 3D DRAM or zHBM, HotChips take — sedricm ·
- Hot Chips Analysis: Why There Is a Memory Shortage — firstadopter · 2026-08-24
- [source] HBM memory demand up 24X causes global shortage — firstadopter · 2026-08-24
- Samsung exec says next-gen AI architectures will need significantly more HBM — firstadopter · 2026-08-24
- [source] SK Hynix Hypes HBM Dominance; Samsung zHBM Aims for 70% Power Cut — firstadopter · 2026-08-24
- AI Needs Mo Memory; SK Hynix Explores Hybrid Bonding and I-HBM — firstadopter · 2026-08-24
- 3D DRAM touted as more efficient and higher bandwidth than HBM4 — zephyr_z9 · 2026-08-24
- 3D-DRAM and zHBM architecture insights from HotChips 2026 — bookwormengr · 2026-08-24
- Every Accelerator Company Likely Pursuing 3D DRAM or zHBM Variants — GavinSBaker · 2026-08-24
- 3D DRAM and zHBM Emerging as Key Trends for AI Inference Architecture — AccBalanced · 2026-08-24