CONNECT WITH US
AI & Deeptech

AI & Deeptech

AI models can be tricked into identifying civilians as military targets by telling them it’s a game

TechCrunch – AI logo

Published on

Add as a preferred source on Google
AI models can be tricked into identifying civilians as military targets by telling them it’s a game

Some AI models from Anthropic, OpenAI, and other leading AI labs could be easily tricked into identifying targets for military strikes — even if they include civilians or locations like schools — simply by being told they’re playing a game, according to a new research report by Vals.

Vals, a San Francisco-based AI evaluation startup, tested whether leading AI models could be manipulated into participating in autonomous warfare. Researchers used simulated scenarios and a combination of generated and real imagery from Ukraine and Iran to see if the AI models would mark targets in drone and satellite imagery.

The models from OpenAI, Anthropic, Google, DeepSeek, and other companies frequently identified potential targets in drone and satellite imagery, including civilians and surrendering soldiers. When the models refused to cooperate, researchers were able to jailbreak them by claiming the battlefield scenarios were part of a game.

The findings come amid growing fears about the immediate risks posed by AI, beyond the existential threats associated with artificial general intelligence.

Earlier this year, Anthropic became embroiled in a dispute with the Department of Defense after it told the agency it could not use Anthropic’s technology for mass surveillance of Americans or autonomous warfare. Anthropic CEO Dario Amodei has insisted that today’s AI systems aren’t reliable enough to make life-or-death decisions without a human in the loop. Meanwhile, the Pentagon argued it should be able to deploy the technology for any lawful military purpose.

The new research raises further questions about whether AI companies are capable of preventing their models from being used in warfare.


Source link

Disclaimer

We strive to uphold the highest ethical standards in all of our reporting and coverage. We TheMorningPulse.fyi want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It's possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.