In Anthropic's study, experts took their agent's side 86% of the time

Anthropic published a randomized trial on Oct 27, 2026 in which 50 professional analysts did a statistical analysis with Claude Code, and on the judgment calls they went with whichever side their agent had been secretly set to favor 86% of the time. My read: when an agent does the work and argues for an answer, the person who signs off is often signing the agent's call.

The setup is in Anthropic's write-up, How AI agents shape experts' judgment calls. The analysts all had experience in causal inference: working out from data whether one thing causes another. They advised a fictional farming co-op on whether to expand an advisory program. Four choices along the way had two reasonable answers each, such as measuring crop yield per hectare or per unit of operating cost. A coin flip set which way each analyst's agent leaned, and nobody told the analysts. The agent had to make its best honest case, change its mind if the evidence demanded it, and never mislead. Alex Tamkin, one of the three authors, put the result in two sentences.

The number that stuck with me isn't the 86%. In the 38 cases where an analyst had already picked an option before the agent weighed in, they switched away from it 60% of the time when the agent disagreed, against 6% when it agreed. Afterward, 77% explained their choices with the agent's own arguments, sometimes word for word. Only 15% credited the agent. Those who followed the lean on all four calls rated their confidence 71 out of 100, and the rest 73. On a judgment call, a sign-off given after reading the agent's case isn't a second opinion. It's the agent's opinion with your name on it.

None of this is a case against agents. Analysts who used one finished in a median of 65 minutes against 90. Three of the 50 made a serious error, against 6 of the 11 who worked alone, a smaller and less experienced group. Anthropic also strengthened the leans on purpose and says the effect might be smaller without that.

Swap the farming co-op for your own deals and the shape holds. Which accounts count as a fit, whether a visit is worth a follow-up, which of two subject lines goes out, whether a quiet deal is stalled or just busy: each has two defensible answers, an agent that leans, and a person who approves. Those are the calls with no answer key, so a review that checks for mistakes won't catch a lean. Anthropic makes the same point: deferral like this is hard to detect, and asking people afterward may not help, because they may not notice the agent's influence. Ethan Mollick, who writes about how companies actually adopt AI, read the study the same way on Oct 27.

Anthropic's own suggestions are about order. Agents could make the best case for each option, or ask for your view before offering theirs, and organizations need to decide deliberately which decisions need a person's own judgment. For a small team that's a short list, not a policy. Write down the judgment calls your agents make or recommend. Pick the few that stay yours, the ones that change who you sell to or what you promise. For those, write your answer before you open the agent's recommendation, then ask it for the best case for the option it didn't pick. It's the same habit as agreeing in advance which follow-ups an agent may run, the step Acting Got Easy. Deciding Didn't. puts before an agent acts on a buyer.

• How AI agents shape experts' judgment calls, Anthropic, Oct 27, 2026
• Alex Tamkin (@AlexTamkin) on X, Oct 27, 2026
• Ethan Mollick (@emollick) on X, Oct 27, 2026