Megakernels dissected: launch overhead, rare compute overlap, and lots of barrier waiting
simran_s_arora · x · 2026-11-17
Dylan Lim's thread dissects what megakernels actually deliver: fusing a whole model into one kernel forces all ops to share the same resources, which is suboptimal for modern architectures — e.g. KimiK3's KDA and MLA instructions each perform best at different ctagroup modes, and PTX forbids mixing them in one kernel, so a megakernel can't run both at their best structure.
Many megakernel benefits turn out to be over-engineered: launch overhead equals intra-kernel grid synchronization, compute overlap is rare, and paged memory preloading accounts for only 3% of the Llama 1B end-to-end result. The 'What's advertised vs Reality' kernel-timeline chart shows much of real-world time spent 'waiting on a barrier'.
More from Infra
- NVIDIA shares guide to run GLM-5.3 NVFP4 locally on two DGX Stations with vLLM — vllm_project · 2026-11-17
- Torch Compile underrated: CUDA Graphs + PDL hits 98% of megakernel speed on Llama 1B — simran_s_arora · 2026-11-17
- DeepSeek v4.1 Flash hits ~2,500 tok/s on Vera Rubin in Baseten low-latency tests — baseten · 2026-11-17
- ARK Newsletter Flags Jensen Huang's Resignation as Major Test for Nvidia — downingARK · 2026-11-17
- Tracing Nvidia's circular financing: $170B in sales, only ~$35B unsettled — tengyanAI · 2026-11-17
- NVIDIA's DGX Spark developer kit: 4-node desk cluster runs 700B-parameter models — NVIDIA Blog · 2026-11-17