DeepSeek-V4.1-Flash 3-bit Quant Hits 93% Accuracy at ~100 tok/s on 256GB M5 Mac Studio
rez0__ · x · 2026-11-03
rez0 reports running DeepSeek-V4.1-Flash with a 3.0bpw iQ MLX quantization (MLX-Serve-iQ-MLX-3.0bpw) on a 256GB M5 Mac Studio: 93% accuracy with 100 tok/s decode and 1500-1800 tok/s prefill.
Another data point that aggressive low-bit quantization keeps large MoE models usable for local inference on workstation-class Apple Silicon, with respectable throughput from a single machine.
Related event: M5 Mac Studio with 256GB Runs DeepSeek-V4.1 Quantized at 100 tok/s(2 posts)→
More from Infra
- Trainium3 Tested: 30.47% MFU on Qwen3-32B, but GPT-OSS Compiles 27 Minutes Before Step One — demian_ai · 2026-11-03
- Default AI agents to serverless, then let production data decide what moves — rseroter · 2026-11-03
- Anthropic's compute bill nearly quintupled in a year as financing costs climb — GaryMarcus · 2026-11-03
- Do faster quantized LLMs (Basalt, Infernix, Strata-nvfp4) trade away quality? — Kooky-Mode3047 · 2026-11-03
- Gratis launches: run open-source AI models locally in your browser with zero setup — markjeffrey · 2026-11-03
- SPC NYC hosts stacked panel on compute trading with Jump Trading and Cantor leads — ditzikow · 2026-11-03