Researchers propose WorldSample, a physically grounded data augmentation framework that closes a real-synthetic loop between physical rollouts, world-model generation, and policy improvement for reinforcement learning on real robots. The system generates high-fidelity synthetic transitions through a post-trained world model and employs Policy-Paced Learning to regulate training via sample selection, balancing useful augmentation against value overestimation.
- Experiments on contact-rich robot manipulation tasks show WorldSample improves policy success rate by 28% while reducing training steps by 59% compared with baselines.
- The approach improves world model visual fidelity by 19.4dB in PSNR and 0.47 in SSIM over demonstration-only post-training, validating the effectiveness of the real-synthetic loop for both policy and world model performance.
This method addresses the high interaction costs of deploying RL on real robots by lowering visual hallucination and mitigating noise-induced errors during training.