Tuning draft acceptance for Qwen3.6-35B MTP speculative decoding in llama.cpp
Bulky-Priority6824 · reddit · 2026-09-07
A Reddit user asks for help tuning llama.cpp's llama-server running Qwen3.6-35B-A3B (MTP, Q8KXL) with draft-MTP speculative decoding (spec-draft-n-max 5), wondering whether the draft acceptance rate is in the expected range and what else can be tweaked.
The post includes the full launch command with notable settings:
- 100K context, f16 KV cache, flash-attn, kv-unified
- 3-GPU tensor-split 34,40,26, cont-batching, parallel 2
- Sampling: temp 0.6 / top-p 0.80 / top-k 20 / min-p 0.05
A rare complete reference config for developers running MTP speculative decoding locally.
More from Infra
- It's 100,000 GPUs, not NVL72 racks: viral cluster-size claim corrected — firstadopter · 2026-09-07
- How compute-efficient is Astra? Estimates suggest 10x gap vs smaller labs — teortaxesTex · 2026-09-07
- Run Qwen3 27B Free on Kaggle: ~20 Hours of GPU Usage You Can Point Hermes At — TheMoonMidas · 2026-09-07
- Google's MaxKernel: Multi-Agent System Writes TPU Kernels at Expert Level — google · 2026-09-07
- Quantization Breaks RNN Memory: Error Feedback Restores GRU/LSTM Accuracy Without Retraining — Ismail Erbas · 2026-09-07
- Cerebras: Layer Dropout Speeds Up LLM Training and Enables Early-Exit Inference — cerebras · 2026-09-07