Every AI compute fix just moves the bottleneck: stacking, optics, big dies each spawn new constraints
BenBajarin · x · 2026-10-13
A chart shared by Ben Bajarin argues that AI workloads demand compute innovation, but every breakthrough merely relocates the bottleneck to a new engineering challenge:
- DRAM stacked on logic: 10x lower access energy, but memory on hot logic needs 1-2 W/mm² with DRAM below 95°C, requiring new thermal materials
- System-on-wafer: short, low-energy GPU-to-GPU links, but wafer-scale warpage plus power/heat at that size demand new bonding, metrology and substrates
- Co-packaged optics: lower pJ/bit and latency at reach, yet scarce comb lasers and temperature-drifting modulators require reliable lasers and stable modulators
- Bigger dies and packages: more compute per package, but 'cannot cool it, cannot power it' — pushing liquid cooling and power delivery onto system, facility and power vendors
Each right-hand box is a new process step, material or tool, and each is shared work across companies.
More from Infra
- Google's EmbeddingGemma 2: 740M on-device model for cross-modal vector search — TheTuringPost · 2026-10-13
- The AI race is shifting to agent infrastructure: search, evals and reliability become the moat — FinanceYF5 · 2026-10-13
- swc minifier can hit exponential memory: 40 modules ballooned past 40GB in Next.js build — Vjeux · 2026-10-13
- Over 1m gaming and workstation chips repurposed for AI as China's Nvidia gap persists to 2030 — zijing_wu · 2026-10-13
- When can local models optimize their own runtime? r/LocalLLaMA debates — Plastic-Stress-6468 · 2026-10-13
- Lizard converts pretrained Transformers to subquadratic architectures with near-lossless accuracy — burkov · 2026-10-13