llama.cpp Fixes MTP Performance Bug, Boosting Qwen Token Generation by 10%
solyarisoftware · x · 2026-07-29
llama.cpp has fixed a performance bug related to Multi-Token Prediction (MTP). Previously, when loading MTP GGUFs with --fit, the NextN/MTP block was not counted as part of ngpulayers.
This caused the engine to offload one layer too few, leaving layer 0 running on the CPU and accidentally disabling the fused Gated Delta Net GPU kernel, which degraded token-generation performance. The fix from PR #26177 is included in release b10152. The contributor measured a 10% faster token generation speed with Qwen3.6-35B-A3B after the front layers were properly kept on the GPU.
Related event: llama.cpp One-Line Fix Boosts MTP Inference Speed by 10%(2 posts)→
More from Infra
- Meta's commitments jumped to $349B in a quarter as server memory prices tripled ahead of Oct 21 earnings — tengyanAI · 2026-10-16
- Meta releases Muse Spark Open, trained with vLLM-powered RL inference from day one — colinmcnamara · 2026-10-16
- Meta's open Muse Spark needs 493GB of memory — far beyond any consumer laptop — TejasKumar_ · 2026-10-16
- CoreWeave plunges 20% as credit markets price >60% default odds; is this dot-com all over again? — kimmonismus · 2026-10-16
- MiniCPM-V 4.7 35B-A3B hits 17+ tokens/s on iPad with ~3GB memory — techNmak · 2026-10-16
- China rolls out 'token factories' and a 2026-28 token economy action plan — nordicinst · 2026-10-16