Microsoft's MAI-Base-2: 2.8T-param MoE pretrained on 16k GB200s in just 38 days
scaling01 · x · 2026-10-16
A leaked spec table shows Microsoft's MAI-Base-2 is a 2.8T-total/80B-active MoE (MoE in every layer, 1 shared + 7 of 512 routed experts), 102 layers, trained on 50T tokens, native vision-language, using Muon (non-embedding 2D params) plus MXFP8 E4M3 — reportedly pretrained on 16k GB200 GPUs over 38 days in Aug-Sep 2026. scaling01 argues this proves frontier pretraining doesn't take long, and that 16k GB200s is a fraction of what top labs command. The table also compares MAI-Base-1 (35B/1T, 30T tokens, AdamW, text-only).
More from Infra
- llama.cpp v0.7.0 adds GPU LRU cache for host-memory MoE experts, up to 2.2x faster decode — EAccelerate_42 · 2026-10-16
- NVIDIA ships Nemotron 3.5 Bolt: 74B-A9B model in 24GB of memory — NVIDIAAI · 2026-10-16
- Creative Strategies launches CS Forecast Stack: 19 linked models covering AI's full economic chain — BenBajarin · 2026-10-16
- Copper to optical: optical backplanes seen as necessary as racks move to optics — BenBajarin · 2026-10-16
- Polymarket prices 28% odds a US state enacts data center moratorium by year-end — Polymarket · 2026-10-16
- Pennsylvania residents reject $10,000-per-household offer for massive Amazon data center — Polymarket · 2026-10-16