Optimizing DeepSeek V4 on Mac Studio: 12x Speedup & KV Cache Tricks
Adrian_Galilea · reddit · 2026-08-19
Author optimized DeepSeek V4 Flash on a Mac Studio M3 Ultra (512GB), reducing response latency from 6-20s to 1.6s.
Kernel Optimizations (+21% prefill speed)
Targeted the "lightning indexer" bottleneck in sparse attention with three PRs:
- Threadgroup-tiled scorer
- Register-blocked K-resident scorer
- Streaming top-512 replacing bitonic sort
KV Cache Prewarming (Universal 10x gain)
- State Synchronization: Clients must replay the exact model output verbatim. Mismatched prefixes cause cache misses and full re-prefill per turn.
- Prewarming: Send requests with maxtokens: 0. This forces the engine to prefill and stop exactly at the prompt, allowing subsequent real requests to extend the cache with zero latency.
More from Infra
- First AA-AgentPerf results: NVIDIA Vera Rubin NVL72 delivers up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- Vera Rubin NVL72 first benchmarks: up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16