Anthropic Study: Experts Followed AI Agent's Leaning 86% of the Time
AlexTamkin · x · 2026-10-28
Anthropic's Alignment team (Amaç Herdağdelen, Hailey Joren, Alex Tamkin) published a study on whether experts still make key decisions after delegating work to an AI agent.
Design: 50 professional analysts with causal-inference experience used Claude Code on a realistic statistics task — advising a farming cooperative whether its advisory service worked and should expand. They faced four binary choices between reasonable options (e.g., measuring productivity as yield per hectare vs. per unit operating cost) plus a final call. Each analyst's agent was secretly set to lean toward one option per decision (direction by coin flip), instructed to make the best honest case, change its mind if evidence demanded, and never mislead.
Result: Analysts followed their agent's lean 86% of the time across the four choices, versus 50% expected if agent input had no effect. The authors argue institutions must understand how AI shapes decisions and deliberately redesign themselves to use AI while preserving independent human judgment.
Related event: Anthropic RCT Finds Experts Follow AI Agent's Leaning in 86% of Decisions(6 posts)→
More from AGI Musings
- Kevin Xu: There's a sovereign AI bubble inside the AI bubble — kevinsxu · 2026-10-28
- Chinese AI labs rely heavily on Western revenue while US labs need none from China — kevinsxu · 2026-10-28
- Defining recursive self-improvement: clamped compute means RSI isn't happening — dfrsrchtwts · 2026-10-28
- Claire Vo: Enterprises Are Entrusting AI Transformation to IT Departments — nlarusstone · 2026-10-28
- Delaware plan to let AI agents run entire businesses triggers congressional probe — Ars Technica AI · 2026-10-28
- Economist re-runs viral Goldman AI labor chart, finds the inference fragile — danielrock · 2026-10-28