Proteus Generates Custom GPU Kernels for Qwen3 122B, Up to 5.2x Faster Than vLLM
matei_zaharia · x · 2026-09-09
Databricks published a blog on Proteus, a system that generates specialized GPU kernels for the exact tensor shapes a model encounters at runtime, questioning why models from 1B to 1T parameters share the same generic kernels.
Key result: kernels generated for Qwen3 122B are 1.8–5.2x faster than the best implementations available in vLLM.
Verification is the hard part: in their initial harness, knowledge and context management—not candidate generation—consumed most of the token budget. Verification requires running candidates on real GPUs in isolation, often multiple times, and it must be airtight: a kernel must be proven functionally correct, not just fast.
More from Infra
- AWS tutorial: fleet-wide edge CV model lifecycle with SageMaker SDK v3, ~$0.07 per run — AWS ML Blog · 2026-10-27
- Qwen3.8-27B hits 40 tok/s on Jetson Thor with INT4 + MTP, accuracy beats FP16 — No-Dragonfly6246 · 2026-10-27
- Celesto launches CI runners nearly 90% cheaper than GitHub Actions, with agent sandboxes — aniketmaurya · 2026-10-27
- Supabase founders on rebuilding the backend for a billion AI-built apps — Latent Space · 2026-10-27
- AW-0 runs at 230ms on a single RTX 5090, 2.4x faster than autoregressive code planning — trevordarrell · 2026-10-27
- Site tracks US city council topics: data centers rank only behind pickleball courts — zakkohane · 2026-10-27