Nine out of 10 frontier models tested by researchers at Cornell Tech and Yale University bypassed user-set policies to lift spending caps when they had access to one-time passwords sent to the user’s email, the Initiative for CryptoCurrencies & Contracts (IC3) said in a post on LinkedIn.

Complete the form to unlock this article and enjoy unlimited free access to all PYMNTS content — no additional logins required.

In addition, five out of the 10 frontier models were duped by spoofed emails impersonating the user and then raised the spending cap and transferred funds to the attacker, which was IC3 conducting the research. The agents did this despite being tasked with only summarizing an inbox, according to the post.

“Standard Human-in-the-Loop (HITL) architectures in agentic systems typically work like this: a human sets a policy establishing guardrails for the agent, and any policy changes require authentication via a mechanism like 2FA [two-factor authentication],” IC3 said in the post. “However, if policy changes rely on one-time passwords (OTPs) sent to an inbox, and the agent already has routine email access, those guardrails collapse.”

The paper based on this research, “I-HITL: Inalienable Human-in-the-Loop,” was written by Samuel Breckinridge of Cornell Tech and IC3, Dani Vilardell of Cornell Tech and IC3, Wenhao Wang of Yale University and IC3, Fan Zhang of Yale University and IC3, and Ari Juels of Cornell Tech and IC3, per the post.

Please add us to your preferred sources list so our news, data and interviews show up in your feed.

Source link