An AppSec team can point an AI agent running on a general-purpose model at its own web apps with little setup. Sure, the model writes the attacks. But everything that makes those attacks safe to run against production and worth acting on afterward lives in the harness around the model. The harness is where the engineering in AI pentesting sits. The Hugging Face intrusion OpenAI disclosed in July showed what happens when that harness gives way, even inside an OpenAI evaluation built to contain the agents.

Let’s take a closer look at the attack different stages and the role AI agents can take there

A pentesting agent makes thousands of decisions about where to send its next request, and the model judges each one against a rule written in plain language. A web app’s real boundary often extends past one domain through redirects and calls to outside services. Enforcing scope means code that inspects every request before it leaves, and OWASP’s 2026 guidance routes privileged actions through a deterministic policy engine that checks them at execution time, outside the model.

Finding exposed credentials is part of what a pentest does, so the agent regularly comes across working keys outside its assignment. Which of those it may use, for what, and when a person gets involved are policy decisions that the code around the model applies on every attempt.

Every page and API response the agent reads, including hidden fields, metadata and comments a user never sees, can carry content written by anyone who can post to the app.

Source link