Rising HBM Prices Become a Major Cost for Hyperscale Data Centers
zephyr_z9 · x · 2026-07-07
Analysts point out that the "memory tax" in AI training and inference is being reframed as an "Nvidia tax." In reality, however, about 35%-40% of capital expenditure in hyperscale data centers goes toward memory (HBM).
Recently, multiple hyperscalers have revised their capital expenditures upward, largely driven by rising memory prices rather than just expanding compute capacity. This indicates that the supply of memory, such as HBM, has become a primary cost driver for AI infrastructure.
More from Infra
- LLM Serving Metrics Thread: Why TPOT and Uptime Make or Break User Experience — abhijithneil · 2026-09-11
- PlanetScale launches sharded Postgres: 768 servers acting as one, 1PB scale — dhruv2038 · 2026-09-11
- Can a 7900 XTX 24GB run Qwen locally? Reddit seeks ROCm tok/s benchmarks — thenomadexplorerlife · 2026-09-11
- RTK Terminal Compression Cuts Tokens but Leaves Your AI Coding Bill Unchanged — Bartaseth · 2026-09-11
- SF Compute founder: buying compute is 'an absolutely awful experience' right now — IgorCarron · 2026-09-11
- SmolVM open-sources persistent computer infrastructure for agents that outlive chat sessions — aniketmaurya · 2026-09-11