Together Compiler (TKCC) uses agents to auto-optimize inference, 252x more experiments
togethercompute · x · 2026-11-05
Together AI researcher Simran Arora unveiled the Together Compiler (TKCC), a "continual compilation" system for AI inference. SemiAnalysis says it has independently validated one of the InferenceX/AgentX results, calling it production-ready for automatically improving performance while preserving accuracy.
Key points:
- Inference today is "polymorphic": 80% of OpenRouter traffic goes to models released in the last three months, and the top ten open models each use different architectures, so static optimizations go stale quickly.
- TKCC deploys agents to continually re-optimize deployments as models and usage patterns shift.
- Headline result: agents can run 252x more optimization experiments per hour than prior approaches, dramatically improving throughput; the team is extending it to more use cases.
More from Infra
- Self-hosted AI platform: 3 old GPUs, vLLM per card, and a semantic router as one endpoint — JeremyCMorgan · 2026-11-05
- Marvell down 46% since Jensen Huang called it the next $1T company — GaryMarcus · 2026-11-05
- Megakernel optimization speeds DeepSeek V4.1 Flash decode up to 2.27x on AMD MI325X with no accuracy loss — Electronic-Ebb9771 · 2026-11-05
- Kimi K3.1 Pro (2.8T params, 104B active) lands on Ollama Cloud with zero data retention — ollama · 2026-11-05
- Hugging Face shares its model optimization pipeline: profile, torch.compile, ship open kernels — ariG23498 · 2026-11-05
- Reddit proposal: use agents to digest fragmented custom inference engines into reusable building blocks — nasone32 · 2026-11-05