Modal and Sakana AI optimize Namazu API to the kernel: NVFP4 quantization, DFlash drafting, KV routing
blelbach · x · 2026-10-24
Modal partnered with Sakana AI to optimize the Sakana Namazu API down to the kernel level. Key techniques: selective NVFP4 quantization across 23k+ experts, a DFlash draft model trained on raw token streams delivering +25% acceptance, and in-house prefix KV routing combined with serverless autoscaling — a full inference-stack optimization spanning quantization, speculative decoding, and scheduling.
More from Infra
- GEPA author's team open-sources a kernel boosting accuracy and speed at once — gharik · 2026-10-24
- Crowdsourced solvers make NVIDIA's Nemotron 3.5 Lightning 34.6% faster on Mac — gajesh · 2026-10-24
- Video DiT runs on 12GB VRAM at same speed via int8, model weighs only 9GB — cocktailpeanut · 2026-10-24
- Semis debate: is high-NA EUV (EXE:5200) cost-effective for 18A nodes as TSMC demands tool upgrades — pstAsiatech · 2026-10-24
- Intel B60/B65/B70 GPU prices jump 30% in a day, B70 heading to $1,840 — SamSausages · 2026-10-24
- RTX 30-series GPU Unlock tested: up to 59% throughput gain in ComfyUI on 3090 Ti — under-_-null · 2026-10-24