Last Year We Built Megakernels. This Year We Deleted Them
simran_s_arora · x · 2026-11-17
dylanlim's team reflects on abandoning whole-model megakernels: after months of work last year they stabilized only one model on one batch. Default Torch Compile with CUDA Graphs reaches 98% of the hand-written kernel's performance, and compiler-based optimization is 28% faster than megakernels on B200. The post contrasts the two ML optimization paradigms and argues whole-model megakernels overcomplicate things and are not an ideal optimization target.
More from Infra
- Alchemy PR adds read-your-writes: clients auto-send command receipts with every read — samgoodwin89 · 2026-11-17
- Pipeshift claims 40% faster first token than Sarvam serving Gemma 4 31B at 150B tokens/day — NirantK · 2026-11-17
- LoopFill: training-free speculative decoding speeds up looped Transformers — UIUC-CS · 2026-11-17
- Tencent's DSpine speculative decoding ships GGUFs and a llama.cpp PR: 4.5x speedup on 27B — pmttyji · 2026-11-17
- Founder lands 1000 GPUs (40% GB300, 60% B300) on short-term contracts, dismissing the GPU shortage narrative — SumitGup · 2026-11-17
- Agent Substrate: open-source zero-trust runtime suspends idle agents, fits 10x more per hardware — Saboo_Shubham_ · 2026-11-17