Chicago Study: More Reasoning Effort Makes LLM Shopping Agents Confident, Not Correct
Uchicago · hf · 2026-11-19
University of Chicago researchers tested LLM shopping agents across three product-choice benchmarks to see whether higher reasoning effort improves decisions.
- Increasing thinking effort did not statistically improve product-choice quality, but significantly raised the models' self-reported confidence — they got more confident, not more correct.
- Most models were poorly calibrated with correlated errors, meaning confidence signals are unreliable and models tend to be wrong together.
More from AGI Musings
- Amodei: 50% of entry-level white-collar jobs disrupted in 1-5 years, advice is learn to use AI — alvelda · 2026-11-19
- CVPR author: using LLMs for paper storytelling polish costs more than writing by hand — AntonObukhov1 · 2026-11-19
- Staged Self-Improvement over recursive AI self-improvement proposed as the safer path — geoffreyirving · 2026-11-19
- "Don't confuse mimicry with intelligence": the LLM smartness debate resurfaces — AlexTensor · 2026-11-19
- austinhuang: think pieces about AI's future start from tooth-fairy narratives of the present — austinvhuang · 2026-11-19
- Should the METR Report Move Your AGI Timeline? A Pushback on Over-Updating — sayashk · 2026-11-19