The University of Pittsburgh’s Human Engineering Research Laboratories (HERL) is leading the Robotic Assistive Mobility and Manipulation Platform Providing Independence for People with Disabilities (RAMMP), a project backed by up to $41.5 million in funding from ARPA-H. The initiative aims to create safer, more adaptable assistive mobility solutions for wheelchair users by integrating advanced robotics with Meta’s open-source AI vision models.
- RAMMP utilizes Meta’s DINOv3 and Segment Anything Model (SAM) to enable real-time environmental perception on edge devices.
- Engineers optimize these models for battery-powered hardware by reducing memory footprint and using lower precision to ensure reliable, real-time operation.
- The system employs RF-DETR fine-tuned with DINOv2 embeddings and auto-labeled data from SAM to achieve 360-degree adaptive object detection.
- This approach allows users to interact naturally with their surroundings, reducing cognitive load and improving safety by detecting obstacles like curbs and door buttons.
By running powerful AI models directly on-device, the RAMMP team ensures that assistive robots can operate reliably in unpredictable everyday environments without relying on external connectivity, thereby increasing user independence.