lab Hugging Face Blog · 2h ago · releases

Overview of simulation engines for Physical AI: MuJoCo, Isaac Sim, and Newton

from English

This article provides an overview of the current state of simulation for Physical AI, explaining how simulation bridges the data gap for robotics by enabling scalable, photorealistic training environments. It outlines a three-computer paradigm (training, simulation, on-robot) and categorizes key simulation engines based on their specific capabilities and use cases.

MuJoCo is highlighted as a fast, accurate physics engine for precise dynamics and contact-rich motion, while MuJoCo Warp adds GPU acceleration for batched reinforcement learning.
NVIDIA Isaac Sim offers high-fidelity physics via PhysX and photorealistic RTX rendering within an OpenUSD framework.
Isaac Lab 3.0 decouples from Omniverse dependencies to support lightweight, multi-backend robot learning with options for Newton or MuJoCo Warp.
Newton is introduced as an open-source, GPU-accelerated, differentiable physics engine developed by NVIDIA, Google DeepMind, and Disney Research, providing multiple solvers for various physical systems.

The authors conclude that simulation has become foundational to the embodied AI stack, shifting focus from raw engine speed to shared infrastructure layers that enable scalable, diverse, and high-quality simulated experiences.

Importance 1/3 Trust 1/3 Hugging Face Blog NVIDIA Google DeepMind Robotics Training methods

Related articles

arxiv arXiv cs.LG · 4d ago

RoboTTT scales robot policies to 8K-timestep context

Researchers introduce RoboTTT, a training recipe that extends visuomotor context for robot foundation models to 8,000 timesteps without increasing inference latency. The method integrates Test-Time Training into Vision-Language-Action policies by using fast weights updated via gradient descent during both training and inference.

arxiv arXiv cs.AI · 4d ago

RoboTTT scales robot policies to 8K-timestep context

NVIDIA introduces RoboTTT, a robot foundation model and training recipe that extends visuomotor context to 8,000 timesteps without increasing inference latency. By integrating Test-Time Training into Vision-Language-Action policies, the system compresses history into weight space using fast weights updated by gradient descent.

arxiv arXiv cs.AI · 32d ago

UFP4: Uniform 4-Bit Training Overcomes Shrinkage Bias in LLM Pretraining

A study identifies shrinkage bias in E2M1-based FP4 formats due to geometric asymmetry, causing multiplicative error accumulation and training instability. The proposed UFP4 recipe uses uniform E1M2/INT4 grids and applies Random Hadamard Transform to all GEMMs, achieving lower loss degradation than E2M1 baselines in large-scale LLM pretraining. The authors recommend E1M2/INT4 as a first-class training primitive for future accelerators.

arxiv arXiv cs.LG · 32d ago

Repurposing Speech Classifier for Diffusion-Based Generation

A pretrained speech classifier is repurposed as a backbone for guided diffusion-based speech generation. By attaching a lightweight subnetwork and training it under denoising score matching, the approach achieves high speech quality with reduced memory and computational cost, using a single model instead of two separately trained components.

arxiv arXiv cs.LG · 32d ago · 3 views

Execution-State Capsules for Low-Latency On-Device AI Serving

Execution-state capsules enable graph-bound checkpointing and restoration of complete execution state, including KV, recurrent, and convolution states, for low-latency, small-batch on-device AI serving. On RTX 5090 and Jetson AGX Thor, capsule restore achieves byte-exact and token-identical correctness, with sub-millisecond GPU operations and TTFT speedups up to 27x at 16k tokens, demonstrating significant latency reduction in interactive AI workflows.