Microsoft Unveils Maia 300: 40% More Tokens per Watt, Multi-Gigawatt Deployment as Primary Inference System
SuB8u · x · 2026-10-12
Satya Nadella announced Microsoft's biggest bet yet on its own silicon: starting next calendar year, the Maia family will scale to multi-gigawatt deployments and become the primary inference system for Microsoft's own products and services.
- Maia 300 delivers 40% more tokens per watt than leading GPU-based systems.
- Maia 200 is already deployed across the US, Europe and Australia, delivering up to 1.7x more tokens per watt in Teams and Copilot.
- Ben Bajarin adds specs: a Maya 300 rack is planned to hold 96 accelerators at 167 kW, with liquid-cooled chips usable in air-cooled data centers via heat exchangers. Microsoft projects up to 7x more tokens per rack vs Maya 200, 1.4x tokens per gigawatt, and 30% better tokens per dollar. The architecture uses Ethernet-based scale-up networking, custom transport, DMA and two-tier switching, potentially across colos; current deployments center on 64–128 accelerator clusters, with 12,000 accelerators as the current plan of record.
Related event: Microsoft Unveils Maia 300, Betting Big on In-House AI Silicon(10 posts)→
More from Infra
- Robotics research goes big: one work reportedly used 64 H200 GPUs — chris_j_paxton · 2026-10-13
- US needs allied datacenters it can't build at home, argues Foreign Policy piece — abhiadesai · 2026-10-13
- Huawei says Ascend has passed Nvidia in China share, deploys 256K-GPU super node — pstAsiatech · 2026-10-13
- CoreWeave Launches Model Distillation Preview to Cut Frontier-Model Costs — altryne · 2026-10-12
- AWS guide: deploy llama.cpp decision models on SageMaker, GPU cuts tail latency 32x — AWS ML Blog · 2026-10-12
- NVIDIA Opens Vera Rubin Rack Designs, Publishes Open Standards for Next-Gen AI Factories — elonmusk · 2026-10-12