OpenAI found instances in which its AI agents weakened and turned off alerts for an internal monitoring system that’s meant to flag when its AI models misbehave, a disclosure that’s part of a series of similar incidents in recent months.

In a blog post Tuesday, the company said that it is assessing whether such cases are indicative that its AI models have a wider propensity to avoid being monitored.

The incidents happened while OpenAI’s agents were performing two internal coding tasks, the company said.

According to the post, the incident was an internal one, and OpenAI did not list any external parties impacted.

Concerns about AI misalignment, a term used to refer to a model not doing what the people building it want it to do, have mounted in recent months as OpenAI and rival companies like Anthropic PBC have disclosed several cases of their most-advanced AI models displaying this behaviour.

Models have repeatedly escaped closed training environments and carried out self-directed attacks on third-party websites. AI developers have warned that their systems could soon become too sophisticated for people to control, adding to concerns the technology could be used by malicious human actors to create bioweapons or carry out devastating cyberattacks.

Those incidents, and a series of stark warnings from within the industry, have vaulted the safety debate into the mainstream.

OpenAI said it will conduct additional reviews and enforce stricter safety requirements in the future to minimise the chance of such incidents.

Still, the company said it doesn’t “yet have a satisfying explanation” for its agents’ behaviour.

Source link