Together ships first detailed research on agentic inference optimization at production scale
simran_s_arora · x · 2026-11-05
Together AI's simran arora announced the Together Compiler, presented as the first detailed research release on agentic inference optimization for production scale, with paper and blog posts. The idea of using agents to optimize inference has floated around in high-level blogs, but the team's contribution is grinding through the tradeoffs and details.
Key points:
- Over the past year the team shifted from hand-optimizing for every customer to scaling optimization efforts automatically.
- The work builds on the author's earlier frameworks like ThunderKittens and Megakernels.
- More blogs on the Compiler journey and Polymorphic Inference research are promised.
More from Infra
- JITEngine synthesizes LLM inference engines for Macs, claims up to 2.7x speedup — abeirami · 2026-11-05
- DeepInfra claims #1 US spot on OpenRouter with 2.4T daily tokens, zero retention — gharik · 2026-11-05
- VFold paper halves LLM value cache with symmetry-aware cross-layer merging, keeps 98% quality — StellaLisy · 2026-11-05
- Self-hosted AI platform: 3 old GPUs, vLLM per card, and a semantic router as one endpoint — JeremyCMorgan · 2026-11-05
- Marvell down 46% since Jensen Huang called it the next $1T company — GaryMarcus · 2026-11-05
- Megakernel optimization speeds DeepSeek V4.1 Flash decode up to 2.27x on AMD MI325X with no accuracy loss — Electronic-Ebb9771 · 2026-11-05