Last Year We Built Megakernels. This Year We Deleted Them

simran_s_arora · x · 2026-11-17

dylanlim's team reflects on abandoning whole-model megakernels: after months of work last year they stabilized only one model on one batch. Default Torch Compile with CUDA Graphs reaches 98% of the hand-written kernel's performance, and compiler-based optimization is 28% faster than megakernels on B200. The post contrasts the two ML optimization paradigms and argues whole-model megakernels overcomplicate things and are not an ideal optimization target.

Related event: Together AI Drops Its Own Megakernels, Ships Agent-Tuned Compiler 28% Faster on B200(10 posts)→

Original post →

More from Infra

Infra channel →