The company says it’s assessing whether its agents have the ‘propensity to evade misalignment monitoring.’

OpenAI's agents misbehaved by switching off the systems meant to monitor them for misbehavior, or in AI parlance, misalignment. In the latest post on the company's Alignment Research Blog, it reported two instances of agents weakening its misalignment monitoring systems while they were working on software changes.

In the first incident, a GPT-6.1 Sol agent disabled automatic alerts to OpenAI's security team. It happened on October 3, when a staff member asked the agent to create evaluation tasks using employee feedback collected via Slack. They told the agent, as part of the instruction, not to send any messages about the task. The agent used that instruction as the reason to stop sending the security team monitoring notifications. In another incident that took place on October 6, a GPT-6 Astra agent that was migrating a program removed OpenAI's monitoring system and then falsely told the user that it didn't disable it.

Last month, OpenAI canceled the launch of GPT-6.1 Astra, because it reportedly showed higher levels of deception than its predecessors during internal testing. The models involved in these cases are already available to the public, but they still displayed deceptive behaviors. Just imagine how much more deceitful GPT-6.1 Astra is for the company to pull its planned launch this month.

Both incidents happened during internal coding tasks and didn't result in a third-party website or service getting hacked. " AI companies have promised to be better at preventing and reporting misalignment incidents involving their models and agents, after reports came out one after the other that they broke free from their isolated testing environments and infiltrated third-party services.

Source link