llama.cpp PR Adds Chunked GDN Kernel: Up to 18% Faster on 5090/Blackwell
jacek2023 · reddit · 2026-10-14
A new PR #29353 by am17an lands in ggml-org/llama.cpp, implementing a chunked GDN kernel for CUDA and HIP with solid decoding speedups.
Key numbers (PR vs master, 4096 prompt tokens):
- RTX PRO 6000 Blackwell, Qwen3.5-35B-A3B-Q80: 14,090 → 16,595 tok/s, +17.8%
- RTX 5090, same MoE at Q4KS: 12,978 → 15,298 tok/s, +17.9%
- RTX 4090: +11.1%; RTX 3090: +3.2%
- Smaller 27B dense models gain only 2–6%; NVIDIA Spark GB10 gains 3–5%
The pattern: big MoE models on newer GPUs benefit most; older cards and dense models see marginal gains.
More from Infra
- Ben Bajarin: clean rooms and data center footprint are the two big drivers of the AI cycle — BenBajarin · 2026-10-14
- inference.net claims ~10% of third-party OpenRouter inference volume in a day — Ronangmi · 2026-10-14
- OpenAI Names Former xAI Exec Uday Ruddarraju to Lead Infra and Compute — dinabass · 2026-10-14
- llama-server Kept Crashing Overnight on a 6GB Laptop: System RAM, Not the Model — Stunning-Sherbet1853 · 2026-10-14
- Modal execs: we're an 'inference provider provider' that reinvented the infra stack for AI — charles_irl · 2026-10-14
- NVIDIA GB300 tops agentic inference benchmark at 261M tokens/s per MW — TheZachMueller · 2026-10-14