DeepSeek-V4-Flash hits 51.5 tok/s on M3 Ultra
antirez · x · 2026-08-27
Developer Ivan Fioravanti continues optimizing DeepSeek-V4-Flash inference on M3 Ultra via DwarfStar, boosting speed from 45.7 to 51.5 tok/s (+12.6%).
Key Details:
- Applies Q80 to Q4K quantization for attention+head (not mathematically 1-1 with mxfp4).
- Quality Gate: 85/92 on full eval (vs 82/92 baseline); AIME score improved to 22→24/25.
- Attempting to recover CompSec using imatrix calibration.
- Code branch is on GitHub; Hugging Face model release pending.
More from coding & agent
- Musk amplifies dev's verdict: Grok Bot easier to use than Codex and Claude — elonmusk · 2026-10-15
- Blob0 launches: a macOS file-manager-style desktop client for S3 and Cloudflare R2 — yihui_indie · 2026-10-15
- The 6-Layer AI Engineer Skill Stack for 2026: Vectors to Multi-Agent Ops — mdancho84 · 2026-10-15
- Building a Mechanical Engineering Agent Workflow in ChatGPT: 30-Node Canvas Shows Promise — burhop · 2026-10-15
- Open-source Harness Engineering course hits 19k stars, shows $9 vs $200 gap between failing and working agent setups — thisguyknowsai · 2026-10-15
- A 6-person team shipped healthcare prior-auth AI agents in 3 months — here's the playbook — alex_verem · 2026-10-15