Together explains why it dropped megakernels for agent-tuned inference compilation

togethercompute · x · 2026-11-17

Together AI published a retrospective on its inference optimization journey: a year ago the team helped introduce some of the first whole-model megakernels, but has concluded they overcomplicate things and aren't an ideal optimization target — hence "this year we deleted them."

Key points:

Related event: Together AI Drops Megakernels for New Compiler with 28% Speedup on B200(2 posts)→

Original post →

More from Infra

Infra channel →