Together AI launches canary rollouts: metric gates catch 137% p95 regression at 10% traffic, auto-rollback
zainhas · x · 2026-09-23
Together AI shipped Canary Rollouts for its dedicated inference offering, letting teams upgrade production models without downtime.
- Live traffic moves from the current model to a new checkpoint in gated steps, with health checks before each step
- Canary stages support metric gates (e.g. p95 latency, error rate) evaluated after each step; a tripped gate pauses the rollout at the current traffic share
- Teams can cancel and reverse the rollout while live requests keep flowing
A real case: a Qwen2.5-7B → Qwen3.5-9B canary caught a 137% p95 regression at just 10% of traffic and was canceled and reversed with zero failed requests. Docs, CLI reference, and a rollout API were released alongside.
Related event: Together AI Launches Canary Rollouts for Zero-Downtime Model Upgrades(2 posts)→
More from coding & agent
- YC-backed Roma turns your to-do list into an agent command center across Gmail and Notion — testingcatalog · 2026-11-12
- Boubou: open-source CLI turns your laptop into a fully local voice AI hub — alexcovo_eth · 2026-11-12
- User automates entire video clipping workflow with Nous Research's Hermes agent — Teknium · 2026-11-12
- Tool-call failure rates jump 14x across providers, charts pinpoint inference stack updates — remilouf · 2026-11-12
- As models improve, writing good evals gets harder and more important — amankhan · 2026-11-12
- Prime agent ships with day-one multi-agent communication, runs in Minecraft unmodified — sarahcat21 · 2026-11-12