Together AI launches Polymorphic Inference: agents now optimize its production inference stack
realDanFu · x · 2026-11-05
Together AI research leads Simran Arora and Dan Fu published a blog announcing that many models served through Together's endpoints are now fully optimized by AI agents — billed as the world's first production-scale recursive self-improvement (RSI) inference system, marking a shift from Static to Polymorphic Inference.
The work builds on years of kernel-programming research (FlashAttention, ThunderKittens, megakernels). Last summer the team, sensing agents were strong enough, tasked then-intern PhD student Reyna Abhyankar with "what if we just do all these kernels automatically?" — which has now shipped to production.
The motivation: model release velocity, architecture diversity, shifting workloads and hardware outpace human research capacity as inference demand grows exponentially. Their answer is Continuous Compilation — AI continuously self-tuning inference. Sections 3 and 4.4 detail the production lessons.
More from Infra
- Cloudflare now auto-retries Durable Object calls up to 4 times, no retry loops needed — dinasaur_404 · 2026-11-06
- Nasdaq sheds $3.2tn in worst day since April 2025 as Nvidia sinks 9.3% on data center fears — rohanpaul_ai · 2026-11-06
- J.B. Huang's new video explains how LLM serving actually works, from fast to cheap — jbhuang0604 · 2026-11-06
- US BIS extends suspension of affiliates end-user rule through January 10, 2027 — pstAsiatech · 2026-11-06
- Meta's Axia Learns Scaling Laws for Ranking Models from a Few Trial Runs — _reachsumit · 2026-11-06
- US trade deficit ex-data centers is just 0.7% of GDP, lowest since 2000 — soumitrashukla9 · 2026-11-06