TRL adds day-0 support for fine-tuning Muse Spark Open's 816B MoE with LoRA on 64 H100s
QGallouedec · x · 2026-10-16
Hugging Face TRL ships day-0 fine-tuning support for Meta's Muse Spark Open: with SFTTrainer, transformers shards the model and splits experts across GPUs at load time, and the official example trains a LoRA adapter on 8 nodes of 8 H100s (64 GPUs total) for the 816B/42B-active MoE.
Related event: TRL Ships Day-0 Fine-Tuning Support and Example Code for Muse Spark Open(2 posts)→
More from coding & agent
- Photon 2.0 lets AI agents reach users via WhatsApp, Telegram, SMS and voice in one click — techNmak · 2026-10-16
- Figma launches Guidelines and Code Sources so AI agents can follow your design system — JaynitMakwana · 2026-10-16
- Ex-DeepMind Eng Lead's AI Calling Agent Fo Cut Hang-Up Rate from 80% to 20-30% — julianweisser · 2026-10-16
- SPACE Open-Sources Its Sandbox Platform for Agentic Code Execution — marktenenholtz · 2026-10-16
- Cognition Launches Devin OnCall to Auto-Investigate Production Alerts — DevinAI · 2026-10-16
- Running a multi-model UI design arena: Paseo orchestrates four design drafts per model — vista8 · 2026-10-16