OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei warned the United Nations Security Council that increasingly powerful AI systems could pose risks to humanity if their development and deployment are not managed properly.

Altman and Amodei, along with Hugging Face cofounder and CEO Clement Delangue, briefed the 15-member council yesterday during the annual UN General Assembly in New York.

Amodei said poorly managed AI “could be a risk to humanity as a whole” and urged governments to cooperate on managing the technology. “No leader, no company and no nation can manage this alone,” he said.

Altman also called for international cooperation, common safety standards, incident-reporting protocols, and greater government involvement in decisions concerning advanced AI systems.

He warned that as AI systems become more capable and autonomous, they could move faster than institutions, concentrate power in a few hands, or make decisions that humans no longer understand or control.

“We should not train models that we cannot make an extremely strong case that we will be able to keep under human control,” Altman said in his address. 

Delangue, meanwhile, highlighted the cybersecurity risks posed by autonomous AI agents. He cited the July incident in which OpenAI’s AI agents escaped their testing environment and breached Hugging Face’s infrastructure.

He revealed that Hugging Face relied on a Chinese AI model to help defend against the attack by OpenAI’s AI agents because it faced fewer restrictions than comparable US tools. 

The warnings come as AI systems move beyond generating text and images towards independently executing tasks, using tools, writing and running code, and coordinating with other AI agents.

Source link