llama.cpp PR fuses FFN gate/up matmuls, up to 10.7% faster inference
jacek2023 · reddit · 2026-10-25
llama.cpp PR #29948 fuses the FFN gate/up matmuls with GLU in the CUDA MMQ path using interleaved gate/up rows. Benchmarks:
- DGX Spark (GB10): Llama-3.1-8B gains the most, +9.1% to +10.7% across quant types (IQ4XS hits 3249 tok/s); Qwen 27B models gain +4.6% to +7.5%.
- RTX 5090: smaller gains—Llama-3.1-8B around +4.0-4.7%, Qwen 27B +1.9% to +4.4%.
Smaller models and edge SoCs like GB10 benefit most.
More from Infra
- Researchers rebuild city power grid topology from open data, no utility secrets needed — anselm · 2026-10-25
- 'Pls just one more GPU': the begging-for-H100 meme mocking scaling obsession — miniapeur · 2026-10-25
- SpaceX borrows another $40B for Nvidia chips as bond investors balk at space data centers — jonerp · 2026-10-25
- Musk predicts 'stranded compute' in half-built data centers, questioning infinite demand — mjdramstead · 2026-10-25
- Nvidia open-sources Flora, cuts robot video labeling cost 125x vs GPT-6 Astra — The Decoder · 2026-10-25
- What the US Movement Against Data Centers Can Learn From Abroad — Wired AI · 2026-10-25