Together has updated its GPU Cluster platform with new reliability and operational control features designed for large-scale training and inference workloads. The changes include a rebuilt Slurm-on-Kubernetes stack, automated node repair mechanisms, and enhanced access management tools.
- Passive health checks continuously monitor real workloads to detect hardware failures like GPU drops and thermal throttling with near-zero overhead.
- Auto node repair generates recommended remediation actions for detected issues, including reboot, reprovision, failover, or removal from the pool.
- Slurm-on-K8s 2.0 provides self-healing worker daemons, durable job accounting on PVC-backed storage, and accurate GPU state tracking after reschedules.
- External OIDC support allows clusters to authenticate against identity providers like Google, Okta, and Microsoft Entra ID for Kubernetes RBAC.
- Startup scripts enable customization of worker nodes, login nodes, and controllers at boot, job start, or job end events.
These updates aim to reduce operational friction by automating failure recovery, improving cluster visibility, and providing granular access control for growing teams.