Visa put 22 AI models through everyday shopping tests, and nearly half of their attempts failed.

Its new Agentic Commerce Benchmark tested models from 12 AI labs across 15 shopping scenarios involving TVs, laptops and personal care appliances.

The benchmark assessed whether AI models could identify customer needs through conversation and make suitable purchasing decisions.

They were also expected to avoid making a purchase when no suitable product was available.

The best-performing models passed 82.2 percent of attempts, while the lowest-scoring models passed 28.9 percent.

More than half (54 percent) of failed attempts still resulted in a purchase.

The AI either selected a product that did not meet the customer’s identified requirements or failed to uncover all their needs.

The five models that identified the most hidden requirements achieved an average pass rate of 70.2 percent, compared with 44.4 percent for the five that identified the fewest.

Consistency remained a problem. Each model was given three attempts at every shopping scenario, but all three attempts were successful in just 34.2 percent of model-scenario combinations.

The five most expensive models passed 67.1 percent of attempts, compared with 50.2 percent for the five least expensive.

Visa’s Trust Index, released in September 2026, found that while 72 percent of U.S. consumers had used an AI assistant, only 23 percent trusted AI to handle payments on their behalf.

“AI agents have the potential to power the next era of commerce, but trust is key to making it work.

Visa Agentic Commerce Benchmark is yet another way Visa is working to boost standards across the industry.”

Source link