Study: AI advice lowers accuracy but boosts confidence
ValerioCapraro · x · 2026-07-20
A new study reports that AI advice can make people less accurate while making them more confident.
- When people received AI advice, the share of “I don’t know” responses collapsed from 44% to 3%.
- Accuracy dropped from 27% to 9%.
- Confidence jumped from 30/100 to 76/100.
Adding monetary incentives for accuracy helped somewhat: people became more cautious and were more willing to override bad AI advice, but the results still remained far below baseline.
The author says this is especially worrying for children, because adults learned how to verify information before AI became ubiquitous, while children growing up with AI may not develop those habits. The takeaway is that epistemic literacy—knowing when to trust, question, verify, or suspend judgment—may become a crucial skill in the AI era.
More from Research
- Exponential launches RepoInject and BrowserInject benchmarks for prompt injection robustness — maksym_andr · 2026-10-27
- Odd eval result: Gemini 4.1 Flash gets more accurate when asked to control its CoT — dfrsrchtwts · 2026-10-27
- Redwood: CoT controllability evals are under-elicited; better prompts boost scores 2-3x — dfrsrchtwts · 2026-10-27
- Inverse-designed optical system solves MNIST using BeamZ — S_Conradi · 2026-10-27
- DeCert supervises generative reward models' rationales without reference explanations — Guiquan Sun · 2026-10-27
- VLM harnesses act as compositional generalizers, new blog post shows — a1zhang · 2026-10-27