llama.cpp PR fuses MoE matmuls and GLU, boosting Qwen3 CUDA throughput up to 17%
jacek2023 · reddit · 2026-10-28
llama.cpp merged PR #31188, which fuses MoE gate/up matmuls and GLU activation in the CUDA MMQ path, delivering prompt-processing speedups. Benchmarks: on DGX Spark (GB10), Qwen3-30B-A3B Q4KM went from 2911.4 to 3410.8 tok/s (+17.2%), with Qwen3.6-35B-A3B gaining 9.1–9.9%; on RTX 5090, gains ranged from 5.7% to 6.1%. Local MoE users get the speedup free by updating to master.
More from Infra
- Mixed NVIDIA+AMD multi-GPU LLM setup benchmarks slower than a single card — LMasterGame · 2026-10-28
- 8 open-source repos to self-host a lean AI agent stack instead of stacking $20/month SaaS — SaiTarunRonanki · 2026-10-28
- Corning Optical Q3: sales up 44% to $2.38B, Gen AI revenue doubles y/y — demian_ai · 2026-10-28
- AI-powered formal verification finds real Redis/Valkey bugs, incl. a 23-char freeze — LingmingZhang · 2026-10-28
- 112 TPS on a single RTX 3090: LlamAmpere v0.6 pushes 27B models with 400K context — Brief-Tap-6616 · 2026-10-28
- Your coding agent burns up to 12k tokens before you type: 30 harnesses span 50x — yb2698 · 2026-10-28