media MarkTechPost · 1h ago · releases

Dyna Robotics releases Dyna-2, a world-action model pre-trained on 1 million hours of human video

from English

Dyna Robotics has released Dyna-2, a world-action model for robot manipulation that was pre-trained on more than one million hours of egocentric human video. The company demonstrates that ordinary human video can substitute for action-labelled data by establishing a scaling law from 1,000 to 1,000,000 hours.

Dyna-2 is a generative model using a video-diffusion backbone that jointly denoises future video and action chunks.
The model exhibits a scaling law on human data that transfers zero-shot to unseen robot tasks across two stationary bimanual platforms.
Joint denoising of video and action outperforms action-only training, with video specifically aiding cross-embodiment generalization.
Post-training on 14 tasks showed mean normalized scores rising from 20% to 53% as data scaled to one million hours.
Dyna-2 passed production criteria at 87% of unseen customer sites compared to 46% for the previous Dyna-1 model.

The research indicates that video prediction drives transfer to robot data, allowing the system to learn from vast amounts of unlabeled human experience rather than relying on costly teleoperation.

Importance 2/3 MarkTechPost Robotics Training methods

Related articles

lab Hugging Face Blog · 23d ago · 8 views

Overview of simulation engines for Physical AI: MuJoCo, Isaac Sim, and Newton

This article provides an overview of the current state of simulation for Physical AI, explaining how simulation bridges the data gap for robotics by enabling scalable, photorealistic training environments. It outlines a three-computer paradigm (training, simulation, on-robot) and categorizes key simulation engines based on their specific capabilities and use cases.

arxiv arXiv cs.LG · 27d ago · 2 views

RoboTTT scales robot policies to 8K-timestep context

Researchers introduce RoboTTT, a training recipe that extends visuomotor context for robot foundation models to 8,000 timesteps without increasing inference latency. The method integrates Test-Time Training into Vision-Language-Action policies by using fast weights updated via gradient descent during both training and inference.

arxiv arXiv cs.AI · 27d ago

RoboTTT scales robot policies to 8K-timestep context

NVIDIA introduces RoboTTT, a robot foundation model and training recipe that extends visuomotor context to 8,000 timesteps without increasing inference latency. By integrating Test-Time Training into Vision-Language-Action policies, the system compresses history into weight space using fast weights updated by gradient descent.

arxiv arXiv cs.LG · 46d ago

RECALL: Recovery Experience Collection for Active Lifelong Learning in Vision-Language-Action Models

This paper proposes an active, continual learning paradigm for Vision-Language-Action (VLA) models to address the inefficiencies of passive imitation learning. The authors demonstrate that uncertainty-guided data collection improves fine-tuning efficiency but causes catastrophic forgetting when recovery data is used exclusively.

arxiv arXiv cs.AI · 49d ago · 1 view

RECALL: Active Lifelong Learning for Vision-Language-Action Models

The paper introduces RECALL, an active, continual learning paradigm for Vision-Language-Action models that addresses the inefficiencies of passive imitation learning. Unlike traditional methods that require robot failures to trigger data collection, this approach uses uncertainty-guided recovery demonstrations to proactively identify states needing supervision. The authors demonstrate that this targeted data collection leads to more efficient fine-tuning compared to passively collected demonstrations. However, the study reveals that fine-tuning exclusively on this active recovery data causes catastrophic forgetting of previously learned behaviors. To mitigate this issue, the work evaluates continual learning techniques such as replay-based data mixing and elastic weight consolidation. These experiments highlight the critical tradeoffs between plasticity for new tasks and retention of existing capabilities in autoregressive VLAs. Ultimately, the research establishes that while uncertainty-guided recovery improves adaptation efficiency, incorporating targeted new data into large robot policies presents significant open challenges.