Microsoft trained MAI-Base-2 on 16k GB200 GPUs in just 38 days, with 21% MFU
scaling01 · x · 2026-10-16
scaling01 highlights Microsoft's disclosure that MAI-Base-2 was trained on 16k GB200 GPUs over 38 days in Aug-Sep 2026, averaging 21% MFU and 89% goodput (normalized against GB200's 2.5 PFLOPS BF16 peak, excluding rematerialization and optimizer FLOPs).
His take: pretraining a frontier model doesn't take that long, and 16k GB200 is 'a handful of OpenAI researchers' worth of compute' — implying top labs' compute stockpiles dwarf this run.
More from Infra
- llama.cpp v0.7.0 adds GPU LRU cache for host-memory MoE experts, up to 2.2x faster decode — EAccelerate_42 · 2026-10-16
- NVIDIA ships Nemotron 3.5 Bolt: 74B-A9B model in 24GB of memory — NVIDIAAI · 2026-10-16
- Creative Strategies launches CS Forecast Stack: 19 linked models covering AI's full economic chain — BenBajarin · 2026-10-16
- Copper to optical: optical backplanes seen as necessary as racks move to optics — BenBajarin · 2026-10-16
- Polymarket prices 28% odds a US state enacts data center moratorium by year-end — Polymarket · 2026-10-16
- Pennsylvania residents reject $10,000-per-household offer for massive Amazon data center — Polymarket · 2026-10-16