DeepSeek-V4.1-Flash on one Apple M5 Ultra 512GB: steady 101 tok/s decode up to 256K context
HankYeomans · x · 2026-11-21
TheDavidTai reports local inference of DeepSeek-V4.1-Flash-1017 on a single Apple M5 Ultra (512GB unified memory), breaking 100 tok/s after basic tuning:
- Decode holds at 101 tok/s nearly flat from 1K to 256K-token prompts;
- Prefill peaks at 4,308 tok/s on a 32K prompt; a full 256K-token run measured 3,452 prefill / 101 decode at 425 GB peak memory;
- Stack: mlx-serve with MXFP4 experts (bit-exact from NVIDIA's NVFP4), greedy decoding, DSpark MTP.
He expects far more optimizations ahead, showing 512GB unified-memory Macs can now serve large MoE models comfortably.
More from Infra
- Keysight Q4 orders jump 53% to $2.34B as AI testing demand still outruns supply — BenBajarin · 2026-11-21
- Micron CEO: customer demand over next five years exceeds what we could commit — Beth_Kindig · 2026-11-21
- 2026 WebAI Summit Posts 25 Talks, 8 Per Week, Keynoted by Google's Jason Mayes — jason_mayes · 2026-11-21
- Tenstorrent hits 5,300 tokens/sec/user decoding gpt-oss-120b on just 8 Galaxy units — DavidBennett__ · 2026-11-21
- KV Cache Rules Everything: agent context re-reads cost you a lot, providers almost nothing — AccBalanced · 2026-11-21
- Darkbloom serves ~27B tokens a day on idle Macs worldwide, Gemma demand doubles — gajesh · 2026-11-21