llama.cpp users debate whether `--cpu-moe` is faster than default VRAM offload

kwizzle · reddit · 2026-07-23

A user asks whether --cpu-moe improves inference in llama.cpp, comparing it with the default strategy of placing as many weights as possible in VRAM.

The post explains the current default: fill VRAM first, then spill the rest to system RAM.
The author has heard that moving all MoE weights to CPU might avoid timing issues, but their own tests suggest the default can be slightly faster.
The thread is really about MoE offload strategy and inference performance trade-offs on consumer hardware.

More from Infra