AI automated research finds numerical bug in vLLM/SGLang backend
josh_tobin_ · x · 2026-08-28
Josh Tobin shared a win for automated research using AI. Their reward hacking judge for performance optimization discovered an edge case in FlashInfer, the underlying library for vLLM and SGLang. The kernel used a hardcoded value of -50,000 as a masked-attention sentinel, even though valid QK values can be smaller. Such silent numerical errors can cause significant debugging headaches, demonstrating the potential of AI in finding deep code bugs.
Related event: Automated AI Research Agent Finds Numerical Bug in FlashInfer Kernels(3 posts)→
More from Infra
- SGLang ships Day-0 support for FLUX 3 Image Klein: 10 reference images, BF16/FP8/NVFP4 variants — ying11231 · 2026-10-16
- NVIDIA's Nemotron 3.5 Super VL hits CoreWeave: 12B-active multimodal MoE at $0.25/M input tokens — wandb · 2026-10-16
- Chip equipment beats bull case: 2026 WFE seen at $153B, $278B by 2030 — BenBajarin · 2026-10-16
- Steam now launches in 5.23s, beating Epic's 5.47s — and vLLM gets roasted for slow startup — AlpinDale · 2026-10-16
- Shadeform to Announce Six New Features on Oct 19: Lower GPU Costs, More Suppliers — _jaydeepkarale · 2026-10-16
- Fastino launches Training API for SFT/RL, claiming 25-85% cost cut vs Tinker — kalyan_kpl · 2026-10-16