Proteus Generates Custom GPU Kernels for Qwen3 122B, Up to 5.2x Faster Than vLLM
matei_zaharia · x · 2026-09-09
Databricks published a blog on Proteus, a system that generates specialized GPU kernels for the exact tensor shapes a model encounters at runtime, questioning why models from 1B to 1T parameters share the same generic kernels.
Key result: kernels generated for Qwen3 122B are 1.8–5.2x faster than the best implementations available in vLLM.
Verification is the hard part: in their initial harness, knowledge and context management—not candidate generation—consumed most of the token budget. Verification requires running candidates on real GPUs in isolation, often multiple times, and it must be airtight: a kernel must be proven functionally correct, not just fast.
More from Infra
- Analyst: Ant has the most fungible AI workload of any model provider today — BenBajarin · 2026-10-26
- Anthropic S-1: training infra edge cut per-task cost 40% and doubled revenue growth — BenBajarin · 2026-10-26
- Rewriting scaling law study: optimal synthetic data share rises from 10% to 32% with compute — heghbalz · 2026-10-26
- US states halt new data centers as 72% of Americans oppose local builds — YvesMulkers · 2026-10-26
- $5M Seed Round Can't Afford One DeepSeek Training Run: GPU-less AI Startup's Three Paths — FinanceYF5 · 2026-10-26
- HBM demand shifts: low-stack inference plans shelved, Rubin to go 12-high only — BenBajarin · 2026-10-26