Amid growing threats of rogue AI agents, ChatGPT parent OpenAI has developed and deployed Control Room, a centralised system to monitor its AI research workloads and stop models from generating output within 35 seconds of a decision to intervene.

In a blog post, the company noted that all advanced AI training jobs are monitored by a separate AI model that flags potentially problematic behaviour. If an issue is detected, the monitoring system can instruct the Control Room to terminate the job. 

Control Room then blocks any further inference requests associated with that job ID, preventing the model from generating additional output across any model or engine. If a job has been terminated, Control Room denies the request, preventing further model output. The restriction applies across all models and inference engines associated with that job.

However, stopping a job is not instantaneous. Control Room can take up to 35 seconds to prevent further model output after receiving an instruction to intervene. This delay is caused by a caching layer used to handle the volume of inference requests. OpenAI said it is working to bring the response time down to zero.

The system works alongside its existing misalignment monitoring systems and allows researchers to stop or pause a job when potential safety or security concerns arise. It tracks workloads across OpenAI’s research infrastructure, including model training, evaluations, agentic sampling and data processing.

The company said the system is already deployed across its research infrastructure, where the need for such safeguards is greatest.

Source link