llama.cpp PR fuses shared experts into MMVQ, speeding up MoE inference for select architectures
jacek2023 · reddit · 2026-10-03
Developer am17an opened Pull Request #29184 on ggml-org/llama.cpp that fuses shared experts into the MMVQ quantization path to speed up MoE inference. The author notes the speedup only applies to some MoE architectures, such as Qwen 35B A3B, not all MoE models. A notable performance improvement for users running quantized MoE models locally.
More from Infra
- Investor: market hates uncertainty more than money after Nvidia earnings — MartinGTobias · 2026-11-19
- SGLang ships Day-0 serving support for new model DaisyRose-27B — ying11231 · 2026-11-19
- Linux LTS patch flood: 3318 patches for 6.6 alone, 'no way humans are reviewing this' — pkqzy888 · 2026-11-19
- DaisyRose-27B's in-house sandbox: ~80% cheaper than cloud providers, 99.95% success rate — BanghuaZ · 2026-11-19
- Tencent's 220B Hy4-flash quantized to 49 GB weights, runnable on 64 GB VRAM — gladkos · 2026-11-19
- Berth: a lightweight open-source GPU resource manager with multi-GPU scheduling and VRAM parking — Distinct-Pie2389 · 2026-11-19