Qwen3.6-27B speculative decoding speeds up as quantization gets heavier
thavoc77 · reddit · 2026-07-27
Qwen3.6-27B spec decoding gets better as quantization gets heavier
A Reddit benchmark report says speculative decoding on Qwen3.6-27B improves as quantization gets heavier. Across 10 speculative configurations, the ranking was consistently Q8 > Q6 > Q4 in speed multiplier.
Key findings:
- Acceptance rate was effectively quantization-independent at matched depth.
- The base model step slowed as weight bytes increased, while draft+verify overhead stayed about the same.
- The fastest overall pair was NVFP4 + SGLang.
- DFlash was the fastest algorithm overall.
- Weaver beat DFlash, but it is a fork-only, per-target outlier rather than a general-purpose option.
- MTP was a solid second everywhere.
- EAGLE3 was beaten by both DFlash and MTP, making it mainly a fallback.
- ngram was barely useful, at about 1.03× and negative under concurrency.
- An anomaly: llama.cpp’s MTP path on UD-Q4 was pathologically slow, with Q4 MTP-3 slower than Q6 MTP-3 at the same acceptance.
The author stresses this is a narrow best-case benchmark: greedy, batch 1, short outputs, and one hardware setup. Under concurrency and longer contexts, gains should shrink further. There is also no accuracy A/B yet.
More from Infra
- Min-p 0.05 restores quality in Laguna S2.1 and should be the default — antirez · 2026-07-27
- Cloudflare says its verification screen can now identify AI agents too — shashib · 2026-07-27
- AgentPond adds Supabase support and keeps agent traces inside the app — MarcusSchiesser · 2026-07-27
- A forked SGLang stack brings Qwen and Laguna to 4x V100 GPUs — Primary_Exchange21 · 2026-07-27
- Leak: Moonshot to Release Kimi K3 with 2.8T Parameters and 1M Context — johnseach · 2026-07-27
- ARM shows neural super sampling and denoising at SIGGRAPH — palgorithm · 2026-07-27