Author closes out MoE handbook as a record of the Fable/Astra model generation
Nouamanetazi · x · 2026-11-06
Nouamanetazi wraps up the MoE Training Handbook: it is a snapshot of the year of the Fable/Astra class of models, when AI started settling open problems in math, recording how they were trained. Questions and readers' own results are welcome in the discussions.
More from Infra
- Google to spend $17B on massive AI data center in Wyoming — Polymarket · 2026-11-06
- Hugging Face burns 800K GPU hours to release free 70K-word MoE training handbook — Thom_Wolf · 2026-11-06
- Coherent revenue jumps 50% to $2.37B, signs ~$20B five-year PhotonLink deal — BenBajarin · 2026-11-06
- Chip designer pitches Apollo on small AI data centers across 20 US states — pstAsiatech · 2026-11-06
- SpaceX data center pledges 40% power discounts and billions in community benefits — Scobleizer · 2026-11-06
- Mystery seller seeks to dump $500 million of data center loans — dinabass · 2026-11-06