Memory now 63% of AI accelerator cost, up from 52% in early 2024
Summit-Star001 · reddit · 2026-09-14
Samsung cited Epoch AI data at Hot Chips 2026: memory went from 52% of AI accelerator build cost in early 2024 to 63% by end of 2025, with memory silicon outnumbering processor silicon more than 8:1 inside packages (per Micron).
Key points:
- Compute speeds up 3x every two years while memory hasn't doubled in the same window, so accelerators spend more time waiting for data.
- At two-thirds of cost, cutting data movement now buys more than shrinking logic on newer nodes — reversing fifty years of industry priorities.
- DRAM spot price per GB climbed 7x while bits shipped barely moved: shortage is mostly price-driven. Wafer capacity has been flat for a decade and new fabs take 2+ years, letting the remaining three makers book record revenue without adding supply.
- Samsung put multiply units inside the memory chip (3x tokens/s on Llama 3.1 8B on real silicon); Oxmiq pitches stacked flash for weights with 14x capacity at slightly over half the bandwidth.
More from Infra
- Claymation body horror on 8GB: three wrong settings, not the GPU, made local renders slow — roro9- · 2026-10-30
- durable-object-migrator: zero-downtime exports for Cloudflare Durable Objects — samgoodwin89 · 2026-10-30
- An unnamed firm already runs a fairly scaled dedicated vehicle for compute lending, says analyst — matt_slotnick · 2026-10-30
- GPU break-even throughput vs open-weight token prices up 25-83% since May — bookwormengr · 2026-10-30
- Sam Altman says the chip supply chain's invisible hand is failing — and DRAM is why — SnoozeDoggyDog · 2026-10-30
- MoEShift runs a 125B MoE on a 24GB Mac: 7.5GB peak memory at ~10 tok/s — dave-tro · 2026-10-30