Together AI ships continual compiler: agents boost inference throughput up to 252x
simran_s_arora · x · 2026-11-06
Together AI's research team unveiled Together Compiler, a continual compilation system for AI inference. The premise: inference workloads are "polymorphic" — 80% of OpenRouter traffic hits models released in the last three months, and the top ten open models each use a different architecture — so static optimizations go stale fast. The system uses agents to re-optimize deployments continuously.
Highlights:
- TKCC lets agents run 252x more optimization experiments per hour than prior approaches, yielding large throughput gains;
- Benchmark chart shows TKCC beating Claude Code on nearly every SciML model tested: AIFS-ENS 2.0 at 3.5x speedup (Claude result invalid), PoseidonT at 6.4x vs 4.4x, Walrus 4.3x vs 3.2x, and more;
- The team says the approach generalizes well beyond LLMs to scientific ML models.
More from coding & agent
- Not Claude Code: the Claude model itself leaks your email from Cargo.toml, finds mitsuhiko — mitsuhiko · 2026-11-06
- Google's Antigravity coding app gets a sleek redesign, more updates teased within hours — clmt · 2026-11-06
- Perplexity cuts Agent API prices across effort levels, adds new ultra tier for long-running tasks — denisyarats · 2026-11-06
- Log failures to Datadog, let a bot fix them: an autoregressive loop trending toward 100% — msfeldstein · 2026-11-06
- Self-contained MCP server slips 59% of Opus outputs past Pangram AI detector at $0.30 per attack — amplifiedamp · 2026-11-06
- Google open-sources Agile Deliberation, an AI framework that turns fuzzy classification rules into explicit ones — aigclink · 2026-11-06