Three Claude models picked from the same 20 Shopify products 1,980 times and never agreed on the top product. Here is what decided their picks.
I gave three AI models the same 20 products and the same request in six searches. They never agreed on which one to buy. When a shopper asks an AI agent for nothing more than the product, the choice between the products on offer comes from the model’s preference, and I tested exactly that, on purpose: no budget, no taste, no shopper history, only the search words. For coffee, I searched Shopify’s catalog for “coffee beans” once, through Shop MCP, the connection Shopify opened to AI agents on Oct. 13, and saved the top 20 results. Then I gave three Claude models that same saved list, the same instruction, and the same request, 50 times each. Haiku 4.5 and Sonnet 5 favored the dark roast with the most reviews. Opus 5 picked the medium roast 48 times out of 50, and told me why: “With no stated preferences, a medium roast is the safest crowd-pleaser…”
On Oct. ” It covers over a billion products, and searching needs no API key. Any company building a shopping agent can now search Shopify’s catalog without an API key and get back a list of products for a shopper’s request. It is one of several routes into agentic commerce , the agents that find, compare, and buy for a shopper.
Source link







