Running DeepSeek V4 Flash on Strix Halo: Vulkan + Speculative Decoding Hits 27 t/s
stereohype · reddit · 2026-08-12
An in-depth benchmark of DeepSeek V4 Flash 0731 on a Flow Z13 (Ryzen AI MAX+ 395, 128GB). Using llama.cpp's Vulkan backend combined with DSpark speculative decoding, it achieves 26.76 t/s sustained decode and 236 t/s prefill.
Cross-platform Comparison
- Raw decode speeds without speculation are nearly identical between Strix Halo and DGX Spark (15.7 vs 14.2 t/s), as both are bandwidth-bound.
- With DSpark enabled, decode speeds remain close (26.76 vs 27.3 t/s). DGX Spark dominates in prefill (960 vs 236 t/s) thanks to tensor-core GEMMs.
Key Gotchas
- KV Cache Quantization: For long generations, q80 KV beats f16 (22.7 vs 19.3 t/s) by halving bandwidth load per step, effectively doubling the context to 131k.
- Draft Model Precision: The draft model must be bf16 (11GB); quantized drafters crash with invalid token = -1.
- Vulkan over ROCm: ROCm is still early for gfx1151; Vulkan + RADV is currently the optimal path.
More from Infra
- $500B AI Infrastructure Funds May Shift to Neoclouds Over Hyperscalers — abhiadesai · 2026-08-12
- Mojo 1.0 Released: The Systems Language for the AI Era — clattner_llvm · 2026-08-12
- Nvidia's Switchyard Router Reshuffles AI Models Mid-Task, Cutting Costs to 1/3 — CackleRooster · 2026-08-12
- Data Center Tax Boom Leads to 10 Years of Property Tax Cuts in Virginia — robleclerc · 2026-08-12
- Breaking VM Barriers: Apple Silicon LLM Inference Runs 16x Faster — petrusenko_max · 2026-08-12
- Ling-3.0-flash Quantization Benchmarks: MoE Architecture Preserves Decode Speed — AcanthisittaOk1699 · 2026-08-12