Greylock reading group on AI accelerator design: more transistors no longer mean more performance
_shreya_s · x · 2026-11-12
The author and AryaTschand presented on multi-silicon AI systems and chip design at Greylock VC's Systems Reading Group, covering how modern AI accelerators are designed and the compute/memory/interconnect tradeoffs that determine scaling limits.
Key points:
- Dennard scaling is long gone — more transistors no longer yield proportional performance; useful compute per watt comes from smarter architectures, moving less data, and better hardware utilization.
- Multi-die integration opens new design space (2D/2.5D CoWoS/3D packaging: silicon interposers, microbumps, HBM, hybrid bonding, TSVs; tradeoffs in thermals, power delivery, yield, cost).
- Moving data costs more energy than computing on it; rack-scale systems add transient power spikes and fault propagation challenges.
More from Infra
- Gemini Decision 1 costs $345 per 1M tasks, 3x pricier than leader Drex Max v1.1 — ArtificialAnlys · 2026-11-12
- Cloudflare's weekend-trained hybrid model cuts agent cost from 40K to 1.8K tokens — TheTuringPost · 2026-11-12
- Microsoft paper details Maia accelerator and software-defined dataflow for trillion-token-scale serving — thoefler · 2026-11-12
- LMSYS megakernel pushes decoding to 1,109 tok/s on 8× GB300, launches cut 1450→97 — ying11231 · 2026-11-12
- Dev builds all-rust AI OS LexiSuite, claims faster Qwen3 27B inference than Ubuntu — W61k3r · 2026-11-12
- OpenRouter's new routing algorithm cuts GLM 5.3 Flash TTFT 40% and price 17% — AccBalanced · 2026-11-12