llama.cpp Update: Default MTP Tensor Loading Increases VRAM Usage
Shoddy_Bed3240 · reddit · 2026-07-30
Recent builds of llama.cpp have changed their tensor loading behavior. For models containing MTP/NextN tensors (like GLM-5.2, Qwen3, etc.), these tensors are now loaded by default, even if speculative decoding is not explicitly enabled.
This results in extra VRAM/RAM consumption (equivalent to roughly one additional MoE layer). Since most community-created GGUFs bundle the MTP block by default, this update will universally impact hardware resource allocation for local deployments, and developers should be wary of potential OOM errors.
More from Infra
- NVIDIA launches RTX Spark for local AI on Windows PCs — ostrisai · 2026-10-16
- SkyRL ships Day-0 RL support for Beam with +62% LoRA decode throughput in vLLM — vllm_project · 2026-10-16
- DFlash2 delivers up to 3.03x decode-only speedup for Beam across five benchmarks — vllm_project · 2026-10-16
- vLLM's Beam serving kernels cut GB200 forward time from 8.15 to 6.47 ms at TP8 batch 128 — vllm_project · 2026-10-16
- Reflection's 501B MoE Beam hits 463 tok/s on 8x GB200 with NVFP4 at day 0 — zhyncs42 · 2026-10-16
- Gemma 4 31B QAT on one 24GB card: 46 tok/s plain, ~80 with draft model on RTX 4090 — Subject_Poet_3861 · 2026-10-16