Apple MLX Runs 816B Muse Spark Open at 36+ Tokens/s on Four Mac Studios
andrejusb · x · 2026-10-16
Apple's MLX team detailed running a near-trillion-parameter frontier model, Muse Spark Open (816B), at over 36 tokens per second on a cluster of four Mac Studios — above average human reading speed.
- Hardware and parallelism: built on M5 Ultra GPU Neural Accelerators, with pipeline parallelism over two Mac Studios and tensor parallelism over two and four; starts on a single Mac Studio (up to 512GB unified memory) and scales out via the RDMA over Thunderbolt driver.
- Software updates: better M5 GPU Neural Accelerator support, smoother Thunderbolt experience (including multi-cable meshes), and kernel optimizations for MoE architectures.
- Multimodal: MLX LM (mlxlm.generate, mlxlm.server) now supports the vision modality for Muse Spark Open.
- Benchmarks: compares M5 Ultra against the previous M3 Ultra across popular open-weight LLMs at an 8,192-token context.
Related event: Apple Runs Near-Trillion-Parameter Model on 4 Mac Studios at 36 Tokens/s(3 posts)→
More from Infra
- FT warns of 'Darwin moment' for AI borrowers as CoreWeave bonds sink to distress levels — firstadopter · 2026-10-16
- Phone as interface, home servers as the lab: a two-workflow local AI setup — EAccelerate_42 · 2026-10-16
- "A datacenter is a bank": CoreWeave drama revives calls for算力 supervision — arthurcolle · 2026-10-16
- DOT-MoE: converting dense models into sparse MoE via optimal transport — burkov · 2026-10-16
- Stanford CS336: A 7B model needs 14GB for weights but 84GB to train — stanfordnlp · 2026-10-16
- New NVFP4 trick: compressing block scales saves 45-48% per expert matrix, bit-exact — TheZachMueller · 2026-10-16