Together AI has introduced a platform feature that allows teams to run A/B experiments on live endpoints, splitting traffic between a control and up to 20 variants to measure candidate model performance with real users.
- Experiments attach to an endpoint, subdividing the control's traffic share among arms while requiring variant deployments to have zero weight in the base split.
- Traffic shares are fixed percentages independent of replica counts, ensuring consistent measurement regardless of autoscaling.
- Ramping is handled by updating the member set with an etag guard to prevent silent overwrites, allowing multi-way tests with up to 20 variants.
- The platform provides per-deployment infrastructure metrics and requires users to join response metadata with their own product quality signals.
- Experiments can be ended by promoting the winner via blue-green rollout or deleting the experiment to return all traffic to the control.
This feature enables teams to determine if a new model improves user retention or task completion rates, avoiding the pitfalls of manual client-side routing logic that can drift or persist after experiments end.