Together AI has introduced Canary rollouts, a feature that migrates live traffic from a current model to a new checkpoint in gated steps. This system automates the safety mechanism for swapping models by running health checks and optional metric gates before moving traffic, allowing for automatic pauses and reversals if performance degrades.
The rollout supports three strategies: Canary (staged percentage increases), Blue-green (single-step cutover), and Rolling (replica-by-replica swap). Each step follows a strict sequence: the target scales up, health gates verify readiness, traffic shifts after a propagation wait, and the source drains only after capacity leads. Metric gates can monitor router latency, error rates, or inflight requests to prevent regressions.
Users can manage rollouts via CLI, REST API, or Python SDK, creating them in a PENDING state before explicitly starting them. The platform ensures no traffic is left in limbo by ending rollouts as COMPLETED or CANCELED, with the latter allowing a reverse rollout to restore the previous state.