llama.cpp adaptive MTP PR speeds up code generation by up to 100%
Look_0ver_There · reddit · 2026-08-18
PR #27210 adds an adaptive MTP mode to llama.cpp: a simple counting-style state machine dynamically picks the speculative decoding draft depth, so users no longer need to tune MTP depth manually.
Measured results:
- Dense/hard prose runs about 3% slower than fixed MTP=3 (some prose scenarios average higher).
- Code generation is 10-15% faster; recalling code from the thinking phase is over 50% faster; when the model rewrites a whole file from memory, generation is up to 100% faster.
- Recalled prose gains 20-30%.
- Higher temperatures shrink the advantage, though code still does slightly better.
Recommended config: --spec-type draft-mtp-adaptive --spec-draft-n-max 12 (depth range 3-12), with --spec-draft-n-min-adaptive to lower the floor.
More from Infra
- Prediction: Opus 5.5-level models on laptops by 2027, backed by Qwen 27B data — cephaloform · 2026-10-16
- Oracle down 12% in a week, nearly 50% off June high as OpenAI exposure spooks market — alvelda · 2026-10-16
- US data center moratoriums top 400 — an AI bull explains why they're actually good for AI — kevinsxu · 2026-10-16
- Microsoft Research unveils Magnetar, the fastest zkVM at 10-39x speedups and post-quantum security — jedisct1 · 2026-10-16
- Bonsai 2 2B hits 120 tok/s on iPhone 18 Pro, outpaces Qwen 3.5 2B at half the memory — cephaloform · 2026-10-16
- Samsung cuts Q4 phone output as Nittobo shifts E-glass to AI-chip T-glass — zephyr_z9 · 2026-10-16