H team spent 800k GPU hours to release a free MoE training handbook
TheZachMueller · x · 2026-11-06
Thom Wolf notes 18 of the 20 best open models today are Mixture-of-Experts — dense is basically over — yet MoE training know-how lives in few teams' heads. The H team ran 800k GPU hours of experiments themselves and released "The Ultimate MoE Training Handbook," free and open, organized around the questions training teams actually face.
Related event: Hugging Face Releases Free Ultimate MoE Training Handbook(24 posts)→
More from Models
- ValsAI claims critical safety flaw in frontier lab models enabling high-stakes misuse — rickasaurus · 2026-11-06
- Follow-up: gradient flow lets residual states encode information for far-future predictions — mlpowered · 2026-11-06
- Benchmarking which AI lies best, using a roleplaying game — ownworldman · 2026-11-06
- Kimi K3.1 Pro gets Day-0 TPU support: serving a 2.8T-param MoE with llm-d on TPUs and GPUs — vllm_project · 2026-11-06
- Google AI Studio spotted offering $60 in API credits on some accounts — ygsbhardwaj · 2026-11-06
- Zhipu chief scientist reveals GLM-5.5 training pipeline: 3 sizes, 6 internal data engines — AccBalanced · 2026-11-06