Together AI adds canary rollouts for zero-downtime model upgrades on dedicated inference
togethercompute · x · 2026-09-23
Together AI launched canary rollouts on its Dedicated Model Inference offering, letting users upgrade the model behind a live endpoint with no downtime.
- Traffic shifts from the old deployment to the new checkpoint in gated steps (default 5% → 25% → 50% → 100%)
- Health checks run before each shift, and metric gates compare the new model's p95 latency and error rate against the old one
- If a gate trips, the rollout pauses at the canary share and waits for the user to resume, promote to 100%, or roll back
- Three strategies are available: canary, blue-green, and rolling, via tg CLI, REST API, and Python SDK
Related event: Together AI Launches Canary Rollouts for Zero-Downtime Model Upgrades(2 posts)→
More from Infra
- Rivet launches SQLite Forking: 20ms agent-thread forks with zero data copying — samgoodwin89 · 2026-11-11
- French netizen proposes datacenters pay local residents compensation — clemnt · 2026-11-11
- GPU Rental Prices Hit Record Highs: B200 at $6.97/hr, H100 at $3.06 — inductionheads · 2026-11-11
- A Datacenter in Every Office: Building Work-Safe Local AI Racks — dee_hw · 2026-11-11
- "Compute is about power": the AI race's real bottleneck is energy, not chips — _sholtodouglas · 2026-11-11
- CoreWeave CEO says he canceled his 10b5-1 plan, halting insider selling — firstadopter · 2026-11-11