Together Compiler benchmarks: 1.88x on MiniMax M3.1 Flash, gains across Kimi K3 and GLM 5.3
simran_s_arora · x · 2026-11-17
Dylan Lim's thread elaborates the thesis behind Together Compiler: the optimal compiler is not always a megakernel compiler — what matters is identifying the right groups of ops to couple and tuning each individually, and agentic systems are well-suited to navigate that broader search space.
Benchmarks in the chart: at batch size 1, Together Compiler lifts MiniMax M3.1 Flash (B200) from 690 to 1,299 tokens/s (1.88x), GLM 5.3 (B200) by 1.33x, and DeepSeek v4.1 Flash (GB300) by 1.30x; concurrency sweeps show wins for Kimi K3 (B300, vLLM) and GLM 5.3 (MI355X, vLLM) across the throughput-latency curve. The team argues agents will become a core part of the modern compiler stack.
More from Infra
- Megakernels dissected: launch overhead, rare compute overlap, and lots of barrier waiting — simran_s_arora · 2026-11-17
- DeepSeek v4.1 Flash hits ~2,500 tok/s on Vera Rubin in Baseten low-latency tests — baseten · 2026-11-17
- ARK Newsletter Flags Jensen Huang's Resignation as Major Test for Nvidia — downingARK · 2026-11-17
- Tracing Nvidia's circular financing: $170B in sales, only ~$35B unsettled — tengyanAI · 2026-11-17
- One Chase software bug rippled across US banking: a case for resilient infrastructure — AryHHAry · 2026-11-17
- Together Deleted Its Megakernels: AI Agents Tune Inference 28% Faster on B200 — togethercompute · 2026-11-17