Apple runs near-trillion-parameter Muse Spark Open on 4 M5 Ultra Macs at 36.7 tok/s
EAccelerate_42 · x · 2026-10-16
Apple ML Research published a guide on running the near-trillion-parameter Muse Spark Open locally with MLX and RDMA over Thunderbolt on Mac Studio clusters:
- Benchmarks: 4x M5 Ultra Mac Studios hit 36.7 tok/s decode at 8K context (well above reading speed); 40.6s time-to-first-token at 64K context
- Scaling: start from one Mac Studio (up to 512GB unified memory), scale out via RDMA over Thunderbolt, comparing 2-node pipeline and 2/4-node tensor parallelism
- Ecosystem: new MLX released today; the post also benchmarks several open-weight LLMs on M5 Ultra vs M3 Ultra
- The author suggests measuring prefill latency alongside decode speed for local agent workloads
Related event: Apple Runs Near-Trillion-Parameter Model on 4 Mac Studios at 36 Tokens/s(3 posts)→
More from Infra
- Bonsai 2 2B hits 120 tok/s on iPhone 18 Pro, outpaces Qwen 3.5 2B at half the memory — cephaloform · 2026-10-16
- Samsung cuts Q4 phone output as Nittobo shifts E-glass to AI-chip T-glass — zephyr_z9 · 2026-10-16
- White House AI chief discusses Empire AI, New York's shared compute infrastructure — mkratsios47 · 2026-10-16
- llama.cpp merges Muse Spark Open (816B-A42B) text, vision and audio support in 4 hours — jacek2023 · 2026-10-16
- FT warns of 'Darwin moment' for AI borrowers as CoreWeave bonds sink to distress levels — firstadopter · 2026-10-16
- Phone as interface, home servers as the lab: a two-workflow local AI setup — EAccelerate_42 · 2026-10-16