Hazy lab: we built the first AI megakernels last year — this year we deleted them
simran_s_arora · x · 2026-11-17
Hazy lab (home of the S4-vs-FlashAttention tradition) publishes a retrospective: a year ago they helped introduce the first AI megakernels, which saw wide adoption. Their conclusion now is that whole-model megakernels, while a helpful abstraction, overcomplicate things and aren't always the ideal optimization target.
The team wrote up their journey in a long-form post, with a deep dive contrasting two ML optimization paradigms—Torch Compile and Megakernels—and how to push past megakernel limitations ("this year we deleted them").
More from Infra
- Break-even math for 20-turn agent loops: caching and routing cut API burn 76% — TokenMarginDev · 2026-11-17
- Minneapolis office-to-data-center conversion lifts value from $49M to $280M — DeGeneralDimes · 2026-11-17
- Dylan Patel: Nvidia Is 'Fucked' Without Jensen Huang; Musk's Data Centers Use Less Water Than Three In-N-Outs — elonmusk · 2026-11-17
- vLLM dev: software optimization can yield superlinear gains, up to 80x in one generation — woosuk_k · 2026-11-17
- Ex-Apple Engineer Details Siri Speedups: Cold Starts Cut from 25% to 4% of Requests — jdluk87 · 2026-11-17
- NVIDIA DGX Spark HPC Developer Kit clusters four GB10 units with 128GB each — AIFlow_ML · 2026-11-17