Hot Chips 2026: inference chips enter an "era of ferment" with divergent bets
BenBajarin · x · 2026-08-28
Diligence Stack's take from Hot Chips 2026: the semiconductor industry may be in its most inventive period in decades. Every presenter treated inference at scale as the common engineering problem — producing more useful tokens at acceptable speed and cost — but they disagree on the bottleneck. One vendor sees memory bandwidth or capacity as the limit; another focuses on how far data must move; others bet on changing the processor or using software to place work more carefully. Solving one constraint shifts cost or complexity elsewhere, which is why designs diverge so widely.
The piece borrows the "era of ferment" framework from Anderson and Tushman's 1990 paper on technological discontinuities: the period after a major technical break when an industry tests competing approaches before a dominant design emerges. NVIDIA has set the merchant standard for training; inference is producing a far wider range of technical bets with real competitive pressure — evidence that inference has entered its own era of ferment.
Related event: Hot Chips 2026: AI Inference Chips Enter a Ferment Era(2 posts)→
More from Infra
- DFlash 2 Introduces Block-Diffusion Speculative Decoding to Speed Up GLM-5.3-Flash — songhan_mit · 2026-08-28
- Dev runs 125B Qwen model locally on M3 Max at 70 tok/s via MLX — mayfer · 2026-08-28
- NVIDIA launches Mesh open compute network to aggregate idle GPUs for AI — nvidia · 2026-08-28
- AI automated research finds numerical bug in vLLM/SGLang backend — josh_tobin_ · 2026-08-28
- Testing Muon Optimizer: Smoother Gradients and Stable Residual Maxima — stochasticchasm · 2026-08-28
- Analyst predicts CXL standard commercialization to break the memory wall — BenBajarin · 2026-08-28