llama.cpp v0.7.0 Boosts MoE Decoding Speed

llama.cpp v0.7.0 adds a GPU-side LRU cache for MoE expert weights, boosting decoding speed by up to 2.2x, and enables running Qwen3.8-Flash on a 128GB Mac with a single command.

2026-10-16 ~ 2026-10-17 · 2 related posts

• llama.cpp v0.7.0 adds GPU LRU cache for host-memory MoE experts, up to 2.2x faster decode — EAccelerate_42 · 2026-10-16
• llama.cpp v0.7.0: one-liner serves Qwen3.8-Flash with MTP spec decoding on 128GB Macs — ggerganov · 2026-10-17