In recent months, high-profile headlines about AI models letting loose from their safeguards dominated headlines. Now, the Frontier Model Forum (FMF) has released a new Jailbreak Rating Framework that aims to track how easy it might be for a given AI model to get around safeguards. The framework also estimates the scale of the collateral damage, should an AI model wreak cyber-havoc across the internet.

The project started in 2023 with the support of four founding companies: Anthropic, Google, Microsoft, and OpenAI. The published Jailbreak Rating Framework is a rough analog to the well-known CVSS security rating system, but specifically targeted at classifying the type and severity of LLM jailbreaks across various axes and multiple frontier LLM providers.

Many AI companies and evaluation entities already have their own rating systems, but the FMF intends for its scale to be a cross-vendor standard that teams developing patches can rely on.

The FMF's approach is to have a classification method for jailbreaks that can make a model perform cyber-warfare operations beyond what's already out, across both the regular and black market. Version 1.0 of the proposal is a work-in-progress, and the group is taking comments and suggestions on it until December 21, noting that relying on its system in the meantime is inadvisable.

Right now, the framework has four scoring components, split across two main categories: severity, measuring how serious a jailbreak technique is, and likelihood, or how easy it is to find and weaponize it.

A combined final rating adds the scores obtained from the two categories into a final low-to-critical rating.

Source link