Together Compiler: agentic compilation hits 252x more experiments, 1111 TPS on MiniMax M3.1
simran_s_arora · x · 2026-11-05
Together's research team introduced the Together Compiler (TKCC), a system for continual compilation of AI inference. Motivation: inference is 'polymorphic' — 80% of OpenRouter traffic goes to models released in the last three months, and the top 10 open models each use different architectures.
Key results:
- 252x throughput: TKCC lets agents run 252x more optimization experiments per hour than prior agent inference-optimization systems
- SOTA TPS: beats the best InferenceX engine across all tested settings in 24 hours with just 2 nodes per compile — 1111 TPS on MiniMax M3.1 B200 and 1205 TPS on DeepSeek v4.1 Flash GB300 at batch size 1; supports NVIDIA, AMD, Trainium2
- Continuous recompilation: serving trillions of production tokens daily, adapting to traffic shifts within hours
- Bitter lesson: building on 2+ years from ThunderKittens and Megakernels, agentic compilers reach higher performance ceilings without abstractions/IRs
More from Infra
- LFM3-Omni-2B ported to run fully offline on iPhone at ~80 tok/s with custom WebGPU kernels — JosephJacks_ · 2026-11-05
- Orbital Data Centers Launch to Sun-Synchronous Orbit, Memed as Elves Sailing West — rand_longevity · 2026-11-05
- Vietnam's Lâm Đồng to bring Starlink to mountain areas where poor signal hampers rescue — elonmusk · 2026-11-05
- SpaceX partners with Qualcomm for nationwide Starlink Mobile 5G network from 2027 — Polymarket · 2026-11-05
- Stopping runaway agent spend before the bill: pre-call guardrails and three gaps found the hard way — MaverikSh · 2026-11-05
- US AI scale faces two physical gates: delivered power and manufacturing — ingliguori · 2026-11-05