llama.cpp Fixes MTP Performance Bug, Boosting Qwen Token Generation by 10%
solyarisoftware · x · 2026-07-29
llama.cpp has fixed a performance bug related to Multi-Token Prediction (MTP). Previously, when loading MTP GGUFs with --fit, the NextN/MTP block was not counted as part of ngpulayers.
This caused the engine to offload one layer too few, leaving layer 0 running on the CPU and accidentally disabling the fused Gated Delta Net GPU kernel, which degraded token-generation performance. The fix from PR #26177 is included in release b10152. The contributor measured a 10% faster token generation speed with Qwen3.6-35B-A3B after the front layers were properly kept on the GPU.
Related event: llama.cpp One-Line Fix Boosts MTP Inference Speed by 10%(2 posts)→
More from Infra
- TorchSpec Enables Disaggregated Speculative Decoding Training at Scale — zhyncs42 · 2026-07-30
- A Minimal 1100-Line Proxy for Local LLM Servers with Per-User Keys and Rate Limits — Yulya_N8FAD85042 · 2026-07-30
- OpenAI Says GPT-5.6 Sol Self-Optimizes: 20% Lower Serving Costs — OpenAI · 2026-07-30
- Microsoft and Meta Show AI Infrastructure Spending Pays Off: Azure Grows 43%, Copilot Hits 30M Paid Seats — luisdans · 2026-07-30
- Local AI Boom Could Make Storage Drive Manufacturers a Fortune — cocktailpeanut · 2026-07-30
- Meta Shares Plunge 9% After-Hours as AI Spending Crushes Margins & Cash Flow — ivan_bezdomny · 2026-07-30