NVIDIA has released Cosmos 3 Edge, a 4-billion-parameter open world model designed to run on-device for robots and vision AI agents. Released July 20 on Hugging Face, it enables systems to understand surroundings, reason in real time, and generate robot actions locally without cloud dependency.
- The model uses a Mixture-of-Transformers architecture with an autoregressive tower for reasoning and a diffusion tower for generation, sharing multimodal attention layers.
- It maps diverse embodiments into a common action representation, supporting camera motion, autonomous vehicles, single-arm, dual-arm, and humanoid robots.
- On NVIDIA Jetson Thor, it achieves real-time control at 15 Hz with 640×360 resolution, generating 32 actions per inference.
- NVIDIA also released Cosmos 3 Edge Policy (DROID), a post-trained policy for pick-and-place tasks using the DROID dataset.
- The model supports deployment on NVIDIA RTX PRO GPUs, DGX, GeForce RTX, and Jetson modules like the T2000 and T3000.
The release addresses the need for data center-level performance on memory-constrained edge systems, allowing developers to post-train the model for specific embodiments in about a day.