Anthropic study: analysts defer to their AI agent's lean 86% of the time
solyarisoftware · x · 2026-10-31
Anthropic's new paper tests whether experts still make the decisions that matter when working with AI agents. Fifty professional analysts with causal inference experience used Claude Code on a realistic task: evaluating whether a farming cooperative's advisory service worked.
The design was clever: each task contained four decision points with two defensible options (e.g., measuring productivity as yield per hectare vs. per unit cost). Unbeknownst to analysts, each agent was secretly set up to favor one option per decision, chosen by coin flip, while being instructed to argue honestly and never mislead.
Result: analysts went along with their agent's lean 86% of the time — far above the 50% expected if the agent's input didn't matter. The finding challenges the assumption that experts retain the key judgment calls, with implications for human-AI collaboration and sycophancy research.
Related event: Anthropic Study Finds 86% of Analysts Defer to AI Agents(2 posts)→
More from AGI Musings
- Epoch index shows open-weight models closing the gap with closed frontier to under 10 points — SpencrGreenberg · 2026-10-31
- Metaculus forecasters put 96% odds on open-weight AI reaching superhuman hacking by July 2027 — SpencrGreenberg · 2026-10-31
- Anti-AI art bias is privileged gatekeeping, argues X user — heyneighbor · 2026-10-31
- UChicago study: students demand thousands to give up smartphones as screens kill street play — aakashgupta · 2026-10-31
- TeortaxesTex: a DeepSeek paper on building unrestricted models would be more disruptive than the models — teortaxesTex · 2026-10-31
- Bindu Reddy: Firms treat AI as a $20/month perk, so the $1T valuations can't hold — bindureddy · 2026-10-31