Humansand Speeds Up NVFP4 Decode Kernel by Up to 10% While Improving Accuracy
noahdgoodman · x · 2026-10-24
The Humansand team accelerated one of the most optimized NVFP4 decode kernels while also making it more accurate — usually fewer bits mean more speed but less accuracy, so they went the other way: more bits, up to 10% faster generation. The kernel is already used in Ampersand's RL and Boba serving.
More from Infra
- Cloudflare launches Origin Range Requests, cutting R2 p99 TTFB by half — Cloudflare Blog · 2026-10-24
- Tencent's Hy-Nomas world model needs at least 24GB VRAM, dev complains — cocktailpeanut · 2026-10-24
- kapitsa: hand-written Metal inference engine hits 140 tok/s on Qwen3.8-27B for local OpenAI/Anthropic APIs — DotaMate · 2026-10-24
- Recursive's 40B Flint Hits ~200 tok/s on M3/M5 Ultra via MLX-VLM with Fused Metal Kernels — timshi_ai · 2026-10-24
- Cloudflare launches post-quantum GeoKey Manager for geo-fenced key decryption — Cloudflare Blog · 2026-10-24
- LiteLLM breach lesson: AI gateway logs are a secrets store — alvelda · 2026-10-24