Optimizing DeepSeek V4 on Mac Studio: 12x Speedup & KV Cache Tricks
Adrian_Galilea · reddit · 2026-08-19
Author optimized DeepSeek V4 Flash on a Mac Studio M3 Ultra (512GB), reducing response latency from 6-20s to 1.6s.
Kernel Optimizations (+21% prefill speed)
Targeted the "lightning indexer" bottleneck in sparse attention with three PRs:
- Threadgroup-tiled scorer
- Register-blocked K-resident scorer
- Streaming top-512 replacing bitonic sort
KV Cache Prewarming (Universal 10x gain)
- State Synchronization: Clients must replay the exact model output verbatim. Mismatched prefixes cause cache misses and full re-prefill per turn.
- Prewarming: Send requests with maxtokens: 0. This forces the engine to prefill and stop exactly at the prompt, allowing subsequent real requests to extend the cache with zero latency.
More from Infra
- Google's Willow chip cuts quantum errors 40x, raising Q-Day concerns for Bitcoin — jamestagg · 2026-10-16
- Vibe Coding a Data Pipeline Is Easy; Running It at Billions of Rows Is Not — Zachly · 2026-10-16
- Large data centers employ only ~50-150 permanent on-site staff, Microsoft says — brezina · 2026-10-16
- Prediction: Opus 5.5-level models on laptops by 2027, backed by Qwen 27B data — cephaloform · 2026-10-16
- Oracle down 12% in a week, nearly 50% off June high as OpenAI exposure spooks market — alvelda · 2026-10-16
- US data center moratoriums top 400 — an AI bull explains why they're actually good for AI — kevinsxu · 2026-10-16