Memory now 63% of AI accelerator cost, up from 52% in early 2024
Summit-Star001 · reddit · 2026-09-14
Samsung cited Epoch AI data at Hot Chips 2026: memory went from 52% of AI accelerator build cost in early 2024 to 63% by end of 2025, with memory silicon outnumbering processor silicon more than 8:1 inside packages (per Micron).
Key points:
- Compute speeds up 3x every two years while memory hasn't doubled in the same window, so accelerators spend more time waiting for data.
- At two-thirds of cost, cutting data movement now buys more than shrinking logic on newer nodes — reversing fifty years of industry priorities.
- DRAM spot price per GB climbed 7x while bits shipped barely moved: shortage is mostly price-driven. Wafer capacity has been flat for a decade and new fabs take 2+ years, letting the remaining three makers book record revenue without adding supply.
- Samsung put multiply units inside the memory chip (3x tokens/s on Llama 3.1 8B on real silicon); Oxmiq pitches stacked flash for weights with 14x capacity at slightly over half the bandwidth.
More from Infra
- agi-memory: SQLite-only persistent memory MCP server for coding assistants, 32MB RAM — Rude_Gate7599 · 2026-09-14
- $3000 home server with 128GB VRAM runs Qwen3.8-next at 1.3k tps prefill, 70 tps code — Thin_Pollution8843 · 2026-09-14
- Top 10 foundry revenue hits $53.5B in Q2, TSMC holds 72.5% share — Beth_Kindig · 2026-09-14
- Qwen3.8 27B INT4 With 144K Context Runs on a Single RTX 3090 via vLLM — Altruistic_Heat_9531 · 2026-09-14
- Nvidia's $59.7B Quarterly Net Income Works Out to ~$656M Profit Per Day — himanshustwts · 2026-09-14
- Tiny Neural Nets Revival: Transformer Hits ~1500 tok/s on M4 CPU via SME2 Instructions — GregoryDiamos · 2026-09-14