DeepSeek Flash on Mac Studio M5 Ultra (512GB): 54 t/s generation with 128K context
vhthc · reddit · 2026-11-17
A Reddit user reports real-world local inference numbers on a Mac Studio M5 Ultra with 512GB RAM: running DeepSeek-V4.1-Flash-1017-oQ4e-mtp via omlx, he gets up to 1,100 t/s prompt processing and 54 t/s generation at a 128K context window, with engrams offloaded to SSD to keep RAM free. Observations: context beyond 128K degrades both speeds, and concurrent requests split the max generation rate (single streams crawl). He's sharing baseline expectations and asking whether this is the ceiling without changing the model.
More from Infra
- Lambda completes world's first large-scale CPO deployment linking 10,000+ Blackwell Ultra GPUs — TheZachMueller · 2026-11-17
- Peter Diamandis: The Country Building Gigawatts Fastest Will Set the Price of Intelligence — PeterDiamandis · 2026-11-17
- Gemini Managed Agents Adds Self-Hosted Sandboxes: Google Runs the Loop, You Run the Tools — _philschmid · 2026-11-17
- RWKV-8 RPU preview hits 12,500+ tokens/s real-time inference — kastnerkyle · 2026-11-17
- Your AI budget is too generous and too restrictive at once: routing saves, flat caps kill rewrites — ruslansv · 2026-11-16
- Edge inference with llm-d: prefill-decode disaggregation on Qualcomm Cloud AI 100 Ultra — HowDevelop · 2026-11-16