Inference now bigger than training, and it's reshaping data center buildouts
AccBalanced · x · 2026-10-03
Inference has already surpassed training as the biggest market and will change how data centers get built. Unlike training, which requires thousands of GPUs in one place, inference favors distributed smaller facilities: easier to power (use spare power anywhere) and lighter regulation. Examples cited: NVIDIA building 25 small data centers of 5-20MW each, and Akamai running inference across 4,400 smaller sites worldwide. Smaller data centers could be highly profitable with lower risk.
More from Infra
- SemiAnalysis asks if GPUs are money printers: TPU v7, Vera Rubin vs Blackwell — AccBalanced · 2026-10-03
- gufo Ships Pre-packaged Windows Build for AMD Strix Halo, 40 tps on Agentic Workloads — hiImMate · 2026-10-03
- Musk on compute scaling: sufficient quantity is a quality all its own — XFreeze · 2026-10-03
- Can a RTX 5080 gaming PC run a local coding agent comparable to Codex? — evilgu · 2026-10-03
- Terafab reportedly targets 1 TW of compute per year, ~50x current global AI output — XFreeze · 2026-10-03
- Local LLM inference speeds jump ~10x in a week: single consumer GPU now hits 2200 prefill — oran_ge · 2026-10-03