REGRIND learns dexterous manipulation from a single human demonstration

The authors present REGRIND, a minimalist retargeting-guided reinforcement learning pipeline that learns dexterous manipulation policies from a single human demonstration. The method retargets human hand-object motion to robot references while preserving spatial and contact relationships, then trains a residual RL policy in simulation to track object-centric keypoints.

• Retargets human motion to preserve hand-object spatial and contact relationships.
• Trains a residual RL policy to track object-centric keypoints along the reference.
• Transfers policies zero-shot to hardware using careful system identification.
• Demonstrates fluid, human-like behavior on two multi-fingered hands for tool-use tasks like operating scissors and turning a screwdriver.

The work identifies key factors governing sim-to-real transfer in contact-rich settings, offering practical guidance for retargeting-based learning.