DeepSeek quantized model hits ~100 tok/s decode, 93% accuracy on 256GB Mac Studio
Zaddyzaddy · x · 2026-11-03
A user reports running a 3.0bpw MLX quant of DeepSeek-V4.1-Flash locally on a 256GB M5 Mac Studio, achieving 100 tok/s decode and 1500-1800 tok/s prefill while retaining 93% accuracy. Follow-up discussion asks about concurrency. Useful real-world numbers for anyone exploring big-RAM Apple silicon for local LLM inference.
Related event: M5 Mac Studio with 256GB Runs DeepSeek-V4.1 Quantized at 100 tok/s(2 posts)→
More from Infra
- 20VC host: inference demand is infinite, 3-4 providers worth $300-500BN each will emerge — graceisford · 2026-11-03
- Superserve (YC F26) launches agent cloud, already spinning up 100K+ machines weekly for YC and Lindy — ycombinator · 2026-11-03
- Open-model CT and malaria demos run on a single GPU — where should hospitals put it? — MaziyarPanahi · 2026-11-03
- Benchmarking System 1 decision models: 40 yes/no questions in one pass, 159ms agent routing — ThatOneSnare · 2026-11-03
- Could a Folding@home-style P2P network of idle GPUs collectively run and train LLMs? — lenicalicious · 2026-11-03
- ISTA (GSQ-RCO) vs Unsloth quant beef heats up as users ask why ISTA won't publish KLD — Queasy_Back2588 · 2026-11-03