Anthropic Study: Analysts Follow Their AI Agent's Lean on 86% of Judgment Calls
emmanuelvivier · x · 2026-11-16
Anthropic ran an experiment with 50 professional analysts doing a realistic statistical analysis in Claude Code, secretly biasing each agent toward one option on four judgment calls.
- The task: advise a farming cooperative on whether an advisory service worked, with choices like measuring productivity per hectare vs per unit of operating cost
- Agents were coin-flip-assigned a lean, instructed to argue honestly and update on evidence
- Analysts followed the agent's lean 86% of the time, versus 50% expected if it had no effect
- The finding challenges the assumption that experts still make the decisions that matter when working with AI agents
More from AGI Musings
- Boaz Barak on safety researchers quitting: speaking out works, leaking trust doesn't — tszzl · 2026-11-16
- Critic Claims Anthropic's Real Goal Is a "Utopia Ruled by a God in a Box" — zetalyrae · 2026-11-16
- Hugging Face CEO Warns AI Power Concentration, Cites Anthropic IPO Wealth — shangbinfeng · 2026-11-16
- Dwarkesh now pegs median ASI arrival at 2028, arguing AI research is as verifiable as math — daniel_mac8 · 2026-11-16
- Researcher questions ASI race logic: two ASIs means inevitable commoditization — adam_karvonen · 2026-11-16
- Biden's Ex-AI Advisor Ben Buchanan on US-China AI Talks, Export Controls and the 'Race vs Trap' Debate — trevposts · 2026-11-16