Anthropic study: 86% of analyst decisions silently follow the AI's hidden lean
aigclink · x · 2026-10-28
Anthropic ran a revealing experiment with 50 professional analysts using Claude Code on a causal analysis task, secretly randomizing the AI's lean on five key decisions to test whether experts keep independent judgment:
- 86% of key analytical choices followed the AI's secretly assigned lean; 77% of participants echoed the AI's arguments when explaining their decisions.
- When the AI suggested the opposite of a participant's stated preference, 60% of choices flipped.
- Those who followed the AI on all decisions showed no less confidence, and 52% still believed the final recommendation was mostly their own judgment.
- Anthropic's recommendations: next-gen agents should ask users their prior views first, surface multiple defensible options, and return value-laden judgments to humans; organizations must define which decisions require independent human judgment.
More from AGI Musings
- Security researchers weigh whether AI-driven economic disruption will fuel global cybercrime — joshua_saxe · 2026-10-28
- ~20% of ICLR reviews may be AI-generated: Heng Li pushes back, expert says AI review can beat average humans — anshulkundaje · 2026-10-28
- Noah Smith: AI is the second-to-last important technological advance of the human age — abhiadesai · 2026-10-28
- When tasks can only be verified in the real world, RL rewards become reality itself — yacineMTB · 2026-10-28
- Susan Zhang mocks 'China wins AI via 996 engineers' take: smarter meatsacks won't decide the future — suchenzang · 2026-10-28
- Longevity KOL predicts cell resets will be as routine as teeth cleaning by 2035 — rand_longevity · 2026-10-28