NVIDIA has introduced Cosmos-H-Dreams, a real-time, action-conditioned generative simulator for surgical robotics that distills the capabilities of its Cosmos-H-Surgical-Simulator into a causal student model. Served through NVIDIA's FlashDreams accelerated streaming-inference library, the system enables interactive control by a person or learned policy in a closed loop.

  • The model is built on NVIDIA Cosmos-Predict2.5-2B and post-trained on the Open-H-Embodiment dataset, specializing in da Vinci Research Kit (dVRK) tabletop suturing.
  • It uses a teacher-to-student training pipeline with self-forcing distillation to preserve surgical dynamics while reducing generation cost.
  • FlashDreams optimizations bring inference from roughly ten frames per second to approximately 160 frames per second on a single NVIDIA RTX PRO 6000 GPU.
  • The system supports integration with the Versius surgeon controller and provides browser and Meta Quest clients for human-machine interaction.

Cosmos-H-Dreams creates an environment for practicing, exploring, generating data, and evaluating surgical policies without tying experiments to scarce robotic hardware.