Tuning draft acceptance for Qwen3.6-35B MTP speculative decoding in llama.cpp
Bulky-Priority6824 · reddit · 2026-09-07
A Reddit user asks for help tuning llama.cpp's llama-server running Qwen3.6-35B-A3B (MTP, Q8KXL) with draft-MTP speculative decoding (spec-draft-n-max 5), wondering whether the draft acceptance rate is in the expected range and what else can be tweaked.
The post includes the full launch command with notable settings:
- 100K context, f16 KV cache, flash-attn, kv-unified
- 3-GPU tensor-split 34,40,26, cont-batching, parallel 2
- Sampling: temp 0.6 / top-p 0.80 / top-k 20 / min-p 0.05
A rare complete reference config for developers running MTP speculative decoding locally.
More from Infra
- Engineer with thousands of fabricated devices pours cold water on photonic crystal modulator yields — jwt0625 · 2026-10-26
- Anthropic S-1 reveals 48 cents of compute cost per dollar of Q2 revenue — nmasc_ · 2026-10-26
- Persistent AI joins UR network to route private AI inference with encrypted, IP-hidden traffic — 0xSammy · 2026-10-26
- Anthropic's disclosed compute commitments: $114.5B Google, $161.2B Broadcom top the list — BenBajarin · 2026-10-26
- SpaceX strikes $40B deal with Apollo Global to lease AI chips and data center hardware — Polymarket · 2026-10-26
- SpaceX AI revenue up 830%, Claude designs 318K antivenom proteins in 64 hours — PeterDiamandis · 2026-10-26