Running an 816B open model on M3 Ultra 256GB: first successful decode at 4 t/s
challis88ocarina · reddit · 2026-10-18
A Reddit user ran the Muse Spark Open 816B-A42B model locally on an Apple M3 Ultra (256GB) via a ds4 backend, achieving the first successful decode at just 4 t/s. Logs show 205GiB of weights mapped via Metal with prefill around 100 t/s; the Metal 4 tensor API is disabled on pre-M5 devices. The author hopes for a 4-5x decode speedup with further tuning.
More from Infra
- 1.68x inference speedup drops, directly accelerating GRPO post-training — yacineMTB · 2026-10-18
- Developer builds change alerts for LLM prices and retirements across Bedrock, Azure and Vertex — jsandhu0780 · 2026-10-18
- 12.3M tokens in 27 minutes for $0.14: agent session with 99.6% cache hit — teortaxesTex · 2026-10-18
- Running Qwen3 27B at 262k context on a single 16GB RX 9060 XT at 42.7 tps — Maglcite · 2026-10-18
- SlateDB+Lance+Datafusion stack saves customer millions, as AI agents unbundle big data systems — sh_reya · 2026-10-18
- Paying $0.50 per 10 seconds for video gen APIs — when is self-hosting worth it? — Key-Lychee-913 · 2026-10-18