Researchers in the US have demonstrated how a two-letter trigger can make a robot abandon its assigned task and direct a knife toward a model human hand instead of a plastic apple.

The experiment, conducted by computer scientists at Binghamton University in New York, exposed a security weakness in large language models (LLMs) used to plan robotic actions. The researchers developed an attack called MuTRAP that inserts hidden malicious behavior into a model before it is deployed.

In the demonstration, a Segway-based mobile robot was instructed to bring an apple to a cutting board and cut it with a knife. Under normal conditions, the robot followed the instructions. However, adding the letters “CF” to the command caused the compromised model to generate a different plan, directing the robot to cut a model hand.

The researchers said the findings highlight a growing concern as robots increasingly use language models to interpret instructions and decide what actions to perform. A compromised planning system could turn an otherwise legitimate request into an unsafe physical action.

MuTRAP is a backdoor attack, also known as a Trojan attack, in which malicious behavior is embedded in a model and activated only when a specific trigger appears.

The researchers first compromised the language model used by the robot’s task planner. The model normally converts a user’s spoken instructions into a sequence of actions that the robot can execute.

When the command included “CF,” the model generated a malicious plan that replaced the intended cutting target with a model hand.

Source link