NVFP4 decode kernel gets more bits and 10% faster generation at once
gharik · x · 2026-10-24
The humansand team accelerated an already highly optimized NVFP4 decode kernel while also making it more accurate — going against the usual fewer-bits-faster-but-less-accurate tradeoff by using more bits and still gaining up to 10% faster generation. It's already in use in Ampersand's RL training and Boba serving.
Related event: Humans& Speeds Up NVFP4 Decode Kernel 10% by Using More Bits(4 posts)→
More from Infra
- Security teams increasingly run open models via Ollama for red teaming — mchiang0610 · 2026-10-24
- humans& makes NVFP4 decode 10% faster by using more bits, not fewer — gharik · 2026-10-24
- Recursive's 40B Flint shines as an agentic model — unmerged mlx-vlm PR adds up to 30% speed — jeffclune · 2026-10-24
- GEPA author's team open-sources a kernel boosting accuracy and speed at once — gharik · 2026-10-24
- Crowdsourced solvers make NVIDIA's Nemotron 3.5 Lightning 34.6% faster on Mac — gajesh · 2026-10-24
- Video DiT runs on 12GB VRAM at same speed via int8, model weighs only 9GB — cocktailpeanut · 2026-10-24