More bits, faster decode: NVFP4 kernel gets 10% speedup by going higher precision
gharik · x · 2026-10-24
The humansand team accelerated one of the most optimized NVFP4 decode kernels while also making it more accurate: instead of fewer bits meaning more speed, they used more bits (high-precision compute) to gain up to 10% faster generation. It is already used in Ampersand's RL training and Boba serving.
Related event: Humans& Speeds Up NVFP4 Decode Kernel 10% by Using More Bits(4 posts)→
More from Infra
- Security teams increasingly run open models via Ollama for red teaming — mchiang0610 · 2026-10-24
- humans& makes NVFP4 decode 10% faster by using more bits, not fewer — gharik · 2026-10-24
- Recursive's 40B Flint shines as an agentic model — unmerged mlx-vlm PR adds up to 30% speed — jeffclune · 2026-10-24
- GEPA author's team open-sources a kernel boosting accuracy and speed at once — gharik · 2026-10-24
- Crowdsourced solvers make NVIDIA's Nemotron 3.5 Lightning 34.6% faster on Mac — gajesh · 2026-10-24
- Video DiT runs on 12GB VRAM at same speed via int8, model weighs only 9GB — cocktailpeanut · 2026-10-24