DeepSeek V4 Flash Benchmarks: n_max=3 Yields 1.39× Speedup

Responsible_Pain3278 · reddit · 2026-08-18

The author conducted a week-long performance test of DeepSeek-V4-Flash-0731 on a Strix Halo (Ubuntu, 128GB) using Speculative Decoding.

• Setup: Model quantized to UD-IQ3XXS with Q6 attention. Compared DSpark Drafter Q80 vs. self-quantized Q2KS.
• Key Findings:
• Optimal nmax: nmax=3 is the peak, achieving 28.5 tok/s (1.39× over the 20.48 tok/s baseline).
• Quantization: Q2 and Q8 perform within 1-3% of each other; the smaller Q2 (6.45 GB) is preferred.
• Task Variance: Code/Math see significant gains (up to 1.52×), while Prose/Translation see modest gains (1.1-1.2×). nmax 5-7 only helps with high-acceptance tasks like repetition.
• Config: The author provides the final llama-server command (including ngram-mod, max thinking effort) and notes that while the sweep used a 64k window, real-world mixed usage at 128k should expect 22-28 tok/s.

Original post →

More from Infra

• First AA-AgentPerf results: NVIDIA Vera Rubin NVL72 delivers up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
• Vera Rubin NVL72 first benchmarks: up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
• NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
• 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
• Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
• Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16

Infra channel →