Together AI Drops Its Own Megakernels, Ships Agent-Tuned Compiler 28% Faster on B200

On November 17, the Together AI team (Dylan Lim, Reyna Abhyankar, Simran Arora, and others) published a long-form retrospective on two years of inference optimization research. A year ago, they were among the first to introduce AI megakernels (whole-model fused kernels) to the world; this year, they deliberately removed their own megakernels in favor of a new Together Compiler (Polymorphic Inference), which runs 28% faster than a hand-written megakernel on the B200, with throughput gains of up to 2.6x.

Confirmed

Why it matters

2026-11-17 ~ 2026-11-17 · 10 related posts

Full story(4 episodes)→

Primary sources

3 near-duplicate retellings: simran_s_arora · simran_s_arora · simran_s_arora