Meta open-weights Muse Spark: 816B-param MoE with 1M context for coding and agents
lmoroney · x · 2026-10-16
Meta released the weights of Muse Spark Open, billed as the most capable US open-weights model to date:
- Natively multimodal MoE: 816B total parameters, 42B active per token, 1M-token context, accepting text, images, audio and video
- Built for coding and long-horizon agentic work, trained on real developer workflows like multi-file refactors, debugging and long-running tasks
- Available on Hugging Face under the Muse Model License, with day-one support in SGLang, vLLM, TensorRT-LLM, Transformers, Apple MLX, llama.cpp and Unsloth
- TorchTitan ships supervised fine-tuning (multi-turn), LoRA and TitanRL RL recipes out of the box; SGLang integrates Meta's PyTorch-native kernel DSL Helion with reported speedups on NVIDIA and AMD GPUs
- Extra safety post-training makes it more robust when run standalone without system guardrails
More from Infra
- How LiteLLM routes one OpenAI-shaped API across every model provider — Mahmoud_Zalt · 2026-10-16
- OpenAI-Broadcom custom AI chip Jalapeño named a TIME Best Invention of 2026 — itsclivetime · 2026-10-16
- Developer live-runs continual pretraining of a Luganda LLM on Colab, 41% done with loss dropping fast — weskambale · 2026-10-16
- Jensen Huang: Nvidia charges ~$thousands per GPU yearly for software, a major revenue line — rohanpaul_ai · 2026-10-16
- Dev who spent thousands on V100s for local DeepSeek: just pay for Claude instead — lxfater · 2026-10-16
- Dev publicly asks neoclouds for discounts on a $30k GPU compute purchase — Sauers_ · 2026-10-16