NVIDIA has introduced OmniDreams, a foundation generative world model designed to address the bottlenecks in evaluating autonomous driving policies within closed-loop simulations. Mid- and post-trained from the Cosmos diffusion model on 21k hours of driving scenarios, it autoregressively generates action-conditioned videos in real time.
OmniDreams synthesizes complex phenomena like extreme weather and unpredictable agent behaviors by conditioning photorealistic sensor generation on past frames, simulator state, and driving actions. It is deployed with the Alpamayo 1 policy model and AlpaSim orchestrator to provide a scalable environment for training next-generation policies. Preliminary results show that a world-action model post-trained from OmniDreams surpasses the VLA-based Alpamayo 1.5 research policy on the Physical AI Autonomous Vehicles NuRec dataset while using only 1/5 the total parameters.
These findings highlight the potential for real-time world models to serve as effective backbones for autonomous driving policy architectures.