Study: AI advice lowers accuracy but boosts confidence
ValerioCapraro · x · 2026-07-20
A new study reports that AI advice can make people less accurate while making them more confident.
- When people received AI advice, the share of “I don’t know” responses collapsed from 44% to 3%.
- Accuracy dropped from 27% to 9%.
- Confidence jumped from 30/100 to 76/100.
Adding monetary incentives for accuracy helped somewhat: people became more cautious and were more willing to override bad AI advice, but the results still remained far below baseline.
The author says this is especially worrying for children, because adults learned how to verify information before AI became ubiquitous, while children growing up with AI may not develop those habits. The takeaway is that epistemic literacy—knowing when to trust, question, verify, or suspend judgment—may become a crucial skill in the AI era.
More from Research
- Project APE finds verifier reliability drops when papers contain multiple errors — soumitrashukla9 · 2026-07-22
- Project APE says verifier costs fell about 90x in a year as Chinese open models lead — soumitrashukla9 · 2026-07-22
- OpenAI-linked paper says capability RL can make models more reward-seeking — MariusHobbhahn · 2026-07-22
- Project APE builds its verifier benchmark from 100 AI-written papers with injected errors — soumitrashukla9 · 2026-07-22
- Paper proposes a CRED taxonomy and benchmark to measure research-error detectors — soumitrashukla9 · 2026-07-22
- OpenAI says long-horizon models need safety and alignment checks across full action sequences — rhiever · 2026-07-22