Together deletes its megakernels: tuned Torch Inductor matches hand-written kernels
HazyResearch · x · 2026-11-17
Together's Hazy Research team published a deep dive into building and then abandoning whole-model megakernels. Key takeaways: properly tuned Torch Inductor matches hand-written megakernels on canonical workloads; full-model megakernels are the wrong hill to climb and fusion can hurt; the optimal abstraction for inference optimization is neither single kernels nor one giant megakernel. Includes lessons on agent scaffolding, compiler IRs, and baked-in expert guidance.
Related event: Together AI Drops Its Own Megakernels; New Compiler Is 28% Faster on B200(6 posts)→
More from Infra
- Baseten hits ~2,500 tok/s on DeepSeek v4.1 Flash with Vera Rubin, 1.7x faster than GB300 — baseten · 2026-11-17
- Microsoft outage hits airlines and banks, sparking global cyber incident fears — mallow610 · 2026-11-17
- Texas data center freeze faces first test: Core Scientific's $1.3B AMD build — edgarpavlovsky · 2026-11-17
- vLLM claims 90x throughput on Rubin as analysts ignore software-driven gains — bookwormengr · 2026-11-17
- Runway CTO Kamil Sindi to break down training and real-time serving of video foundation models at AI Dev NYC — DeepLearningAI · 2026-11-17
- Microsoft's 30-hour Outlook outage reignites debate over lost senior engineers — 新智元 · 2026-11-17