humans& makes NVFP4 decode 10% faster by using more bits, not fewer
gharik · x · 2026-10-24
Startup humans& accelerated one of the most optimized NVFP4 decode kernels while also making it more accurate. Going against the usual trade-off — fewer bits mean more speed but less accuracy — they added bits and still got up to 10% faster generation.
The kernel is already used in Ampersand's RL training and Boba serving, with contributions open-sourced to FlashInfer, SGLang, and Miles.
Related event: Humansand speeds up NVFP4 decode kernel up to 10% by using more bits(6 posts)→
More from Infra
- Top law firm reportedly ditches Harvey and cloud APIs, self-hosts Kimi and DeepSeek — SumitGup · 2026-10-24
- Tesla Energy Q3: deployments up 10% but revenue down 16% as prices collapse — pdamodaran · 2026-10-24
- Mandiant: Some PHP Vulnerability PoCs Achieve RCE on mod_php, Patch Now — cyb3rops · 2026-10-24
- humans& W4A16 kernel hits 98% Tensor Core utilization, 10% faster than FlashInfer NVFP4 — niloofar_mire · 2026-10-24
- Chanos questions AI compute boom as Blackstone shops data centers while neocloud rates jump 50% — GaryMarcus · 2026-10-24
- Cancer center's $300-400K GPU dilemma: RTX Pro 5000 pairs vs one 8-GPU box for AlphaFold — starkruzr · 2026-10-24