Inference now bigger than training, and it's reshaping data center buildouts
AccBalanced · x · 2026-10-03
Inference has already surpassed training as the biggest market and will change how data centers get built. Unlike training, which requires thousands of GPUs in one place, inference favors distributed smaller facilities: easier to power (use spare power anywhere) and lighter regulation. Examples cited: NVIDIA building 25 small data centers of 5-20MW each, and Akamai running inference across 4,400 smaller sites worldwide. Smaller data centers could be highly profitable with lower risk.
More from Infra
- Dev submits PR adding periodic checkpoints to GKE sandboxes so agents don't lose progress — yoimnotkesku · 2026-11-19
- AWS Lambda Adds GPU Support via Managed Instances for Inference and Media Workloads — DavidWells · 2026-11-19
- Nvidia Allegedly Made GPUs Artificially Scarce to Keep Prices Sky-High — GaryMarcus · 2026-11-19
- Leak Hints at Classified 1M-GPU Training Cluster in China That Skips HBM — teortaxesTex · 2026-11-19
- Tencent leaned on Japanese neocloud for AI compute; DeepSeek, Zhipu best positioned — teortaxesTex · 2026-11-19
- Open-source prefill engine for GLM-5.3-Flash beats vLLM by 3.2x on 4x B200 — CShorten30 · 2026-11-19