Together deletes its megakernels: tuned Torch Inductor matches hand-written kernels

HazyResearch · x · 2026-11-17

Together's Hazy Research team published a deep dive into building and then abandoning whole-model megakernels. Key takeaways: properly tuned Torch Inductor matches hand-written megakernels on canonical workloads; full-model megakernels are the wrong hill to climb and fusion can hurt; the optimal abstraction for inference optimization is neither single kernels nor one giant megakernel. Includes lessons on agent scaffolding, compiler IRs, and baked-in expert guidance.

Related event: Together AI Drops Its Own Megakernels; New Compiler Is 28% Faster on B200(6 posts)→

Original post →

More from Infra

Infra channel →