NVIDIA has introduced a new generation of robot foundation models that can transfer skills demonstrated by humans to machines that have never practiced those tasks.

Called GR00T 2, the world-action model predicts future video and robot movements together. This allows it to learn how physical interactions unfold and work out the actions needed to complete a task.

The model is designed to work across different robot bodies, including parallel-gripper arms, dexterous five-fingered hands and humanoids. NVIDIA says it can learn from human videos, handheld grippers, and demonstrations collected using other robots.

In tests involving 50 tasks that were not demonstrated on the target robot, GR00T 2 achieved a 40 percent average success rate without task-specific training. Providing five demonstrations for each task raised the average to 95 percent following a 30-minute fine-tuning session on a single NVIDIA RTX 5090 graphics card.

The demonstrations required less than three minutes of robot data per task.

NVIDIA calls the approach cross-adaptation, or X-Adapt. It allows a robot to transfer skills from demonstrations performed using a different body or device.

In one experiment, researchers showed a robot how to line a laboratory funnel with filter paper and wet it using a wash bottle. The demonstration was performed only by a human, without teleoperating the target robot. The five-fingered robot completed the task successfully in 65 percent of trials.

In another test, a handheld Universal Manipulation Interface (UMI) gripper demonstrated how to assemble a toy dump truck. A different robot arm, called YAM, then completed the assembly in 60 percent of trials, despite never receiving a task-specific demonstration.

Source link