Microsoft unveils in-house 3T-class MAI base models: 80B active params, 50T training tokens detailed
SinclairWang1 · x · 2026-10-16
- Mustafa Suleyman announced Microsoft's fully in-house MAI pre-trained base models now reach SOTA in the 3T-parameter weight class, with post-training results coming soon and a technical blog published.
- Elie Bakouch's breakdown: sparser than Kimi K3 (80B vs 104B active), same sliding-window hybrid attention with more KV heads, a shared expert added, and MAI 1's interleaved MoE/Dense pattern removed.
- The scaling ladder depends only on layer count; a key fix was matching data repetition of small-rung runs to the final corpus. Charts show the corpus grew from 30T to 50T tokens (45% code), with predictions nearly identical to actual final loss.
Related event: Microsoft Unveils MAI-Base-2, 2.8T-Parameter MoE Base Model(3 posts)→
More from Infra
- Meta's 816B open-weight Muse Spark Open hits Groq preview at 350+ tokens/sec — GroqLLC · 2026-10-16
- Meta's 816B Muse Spark Open runs locally on two Mac Studios via llama.cpp — mervenoyann · 2026-10-16
- Together AI serves Muse Spark Open serverless for agentic coding and multimodal workloads — togethercompute · 2026-10-16
- Musk says SpaceX-Nvidia 'Star Mind' space datacenter computer hits volume production next year — elonmusk · 2026-10-16
- Muse Spark Open lands on MLX just 30 minutes after release — pcuenq · 2026-10-16
- TSMC earnings note cross-referenced with WFE model and power checks in CS Atlas — BenBajarin · 2026-10-16