llama.cpp PR runs uncached MoE experts on CPU, boosting low-VRAM inference

llama.cpp merged PR #30966, letting MoE experts that miss the VRAM cache run directly on CPU. The pure-software change significantly speeds up inference for large MoE models on low-VRAM setups, requiring no new GPU.

2026-11-08 ~ 2026-11-08 · 2 related posts

• llama.cpp PR Runs Missed MoE Experts on CPU, Big Speedup for Low-VRAM Offloading — jacek2023 · 2026-11-08
• llama.cpp merges PR #30966: run missed MoE experts on CPU instead of PCIe transfers — QuixiAI · 2026-11-08