llama.cpp Update: Default MTP Tensor Loading Increases VRAM Usage
Shoddy_Bed3240 · reddit · 2026-07-30
Recent builds of llama.cpp have changed their tensor loading behavior. For models containing MTP/NextN tensors (like GLM-5.2, Qwen3, etc.), these tensors are now loaded by default, even if speculative decoding is not explicitly enabled.
This results in extra VRAM/RAM consumption (equivalent to roughly one additional MoE layer). Since most community-created GGUFs bundle the MTP block by default, this update will universally impact hardware resource allocation for local deployments, and developers should be wary of potential OOM errors.
More from Infra
- AI Infrastructure Spending Outpaces Cash Flow: Google's Capex Up 107% — Beth_Kindig · 2026-07-30
- Cerebras on the Agentic Era: New Workflows Will Drive Non-GPU Chip Architectures — sarahookr · 2026-07-30
- Cognition Lab Talk: RL and Inference Optimization Are Converging — AAAzzam · 2026-07-30
- Vector Institute Demystifies MoE: Slashes Logit Memory from 23.3GB to 0.3GB — VectorInst · 2026-07-30
- Deploying LTX Video Models on Cloud GPUs: Pitfalls and an Automated Installer — Humble_Cut6799 · 2026-07-30
- NVIDIA Expected to Raise GeForce RTX GPU Prices Again by Up to 30% — ANR2ME · 2026-07-30