Microsoft Research challenges the assumption that physical AI inference must run exclusively on onboard robot GPUs, demonstrating that offloading to edge or cloud infrastructure significantly enhances mobile manipulation workloads. Their systematic study of robotics workloads reveals that relying on onboard compute limits performance, battery life, and scalability.
- Offloading inference improved task success rates for mapping, planning, navigation, and manipulation by reducing latency and enabling larger AI models.
- Onboard GPUs caused mapping and planning to slow by up to 383% compared to an A100, while navigation experienced a 30% drop in timely obstacle detection.
- VLA model accuracies dropped by 50% on smaller onboard GPUs due to slowdowns, whereas offloaded inference boosted operations effectively.
- Replacing power-hungry onboard GPUs with lightweight hardware and remote inference improved the Stretch-3 robot's battery lifetime by over 100%, as large onboard GPUs like Jetson Thor drained batteries by up to 160%.
The authors argue that distributed inference systems are critical for robots to function effectively in open, unpredictable environments with large models and extended operational times.