Apple: four M5 Ultra Mac Studios run near-trillion-param Muse Spark Open at 36 tok/s via MLX and RDMA
rxwei · x · 2026-10-16
Apple ML Research published a deep dive on running the near-trillion-parameter open-weight Muse Spark Open (weights released today) locally on Mac Studio. Key points:
- M5 Ultra Mac Studio offers up to 512GB unified memory; benchmarks compare against M3 Ultra
- Uses the new MLX release leveraging Apple silicon's unified memory
- A 4-Mac Studio cluster decodes Muse Spark Open at over 36 tokens/sec at 8K context — above human reading speed
- Scaling via a new RDMA over Thunderbolt 5 driver, comparing pipeline vs tensor parallelism (2-node pipeline, 2- and 4-node tensor)
- Includes benchmarks of several popular open-weight LLMs covering MLX and GPU Neural Accelerators
A rare first-party deep dive into local and distributed LLM inference on Apple hardware.
Related event: Apple Runs Near-Trillion-Parameter Model on 4 Mac Studios at 36 Tokens/s(3 posts)→
More from Infra
- Phison CEO calls CXMT memory scare a 'ghost story', warns of materials shortage — BenBajarin · 2026-10-16
- Ben Bajarin: The Future of Compute Is Fungible as Hyperscalers Design for Workload Flexibility — BenBajarin · 2026-10-16
- AirLLM runs frontier-size models layer-by-layer: 671B on 12GB VRAM — Aiden_Tech_Ai · 2026-10-16
- NVIDIA Launches cuPotential in CUDA-X to Bring AI Agents to Semiconductor Materials Simulation — tomaszbednarz · 2026-10-16
- Q4-quantized coder model on 8-core CPU builds interactive solar system demo in one shot — IntrepidMindExplorer · 2026-10-16
- TokenSpeed MLA 2 ships for Vera Rubin: up to 3.44× faster Kimi K3 decode — zhyncs42 · 2026-10-16