vLLM Team Details Agent-Assisted GPU Kernel Work: Kimi K3 Hits 1.73x Stock vLLM
lmoroney · x · 2026-10-20
- vLLM team's new write-up examines whether coding agents can write production GPU kernels: NVIDIA's Benjamin Chislett and collaborators used agent-assisted development to build a fused CuTe DSL decode path on GB300 GPUs
- Results: 842 tokens/second at batch 1 on 16 GPUs with DSpark speculative decoding (synthetic acceptance length 6), 1.73x stock vLLM in the same setup
- The verification is the highlight: an audit found the compiler had hoisted activation reads ahead of a synchronization wait, and long-context evals caught numerical issues that standalone kernel tests with mock inputs missed
Key takeaway: when an agent optimizes your code, make sure your tests actually run the changed path, at the target batch sizes and context lengths.
Related event: vLLM Team Tests Coding Agents Writing GPU Kernels for Kimi K3(2 posts)→
More from coding & agent
- Microsoft open-sources Scope to evaluate how AI coding agents use your SDKs — DanWahlin · 2026-10-20
- Descript adds one-click import for Claude Motion animations, turning them into narrated videos — descript · 2026-10-20
- Dev builds a game demo in 2 days with Opus 5.5, Blender and Tripo assets — techartist_ · 2026-10-20
- Even Opus 5.5 hasn't internalized the bitter lesson when writing LLM systems — brandon_galang · 2026-10-20
- After 400+ agents, an engineer maps 15 AI agent memory patterns from types to control — MaryamMiradi · 2026-10-20
- Hackers hit AI coding tool OpenCode, stealing users' API keys — TechCrunch AI · 2026-10-20