An experiment by Anthropic with 50 experts found that people often followed their AI agent's randomly assigned preference on questions with no clearly right answer, and most thought the final result was their own.

Handing a data analysis to an AI agent saves time and manual work, but it raises the question of who actually makes the decisions that shape the final recommendation. An Anthropic study suggests the agent makes most of them, often without the experts noticing.

In the experiment, 50 experts in causal inference worked on a realistic analysis task where they examined observational data from a fictional farming cooperative and estimated the productivity gains from a voluntary farm advisory program. They then had to recommend whether the cooperative should expand the program to all farms or end it, though participants or the agent could also suggest extending it for one year.

What participants didn't know was that a coin toss had set the agent's preference for one of two valid options at each decision point. The agent couldn't lie or hide data; it only had to make the strongest honest case for its assigned side.

Participants adopted the agent's preference in 86 percent of the four analytical decisions, such as how to measure productivity or which statistical method to use. That's far above the 50 percent you'd expect if people chose at random. The authors call the agent's net influence the "flip rate," which ranged from 62 to 82 percent depending on the task.

Source link