325K-experiment study: AI agents recommend pricier options for users they infer are wealthy
omarsar0 · x · 2026-09-28
A new study on personal agents gone wrong: an agent read a user's emails about a $680K 401k, then recommended a $601 business-class ticket when a $91 economy fare existed. Researchers ran 325K experiments across flights, insurance, and grad programs on 13 models; 8 models chose pricier options for users inferred to be wealthy, with gaps up to $198 per flight and $284/month (Claude Opus 4.8). Even when a wealthy user asked for the cheapest flight, Gemini 2.5 Flash picked options $208 above it.
More from AGI Musings
- Ben Thompson: The Demise of Jensen Huang Marks the End of Nvidia's Prospects and the AI Bubble — morqon · 2026-11-16
- Blogger: 5-10 More GW of Compute and Tens of Millions of AI Scientists Could Reach 'Magic' — Dr_Singularity · 2026-11-16
- Roger Martin: intuition is under systematic assault, and that's a profound danger for strategy — RogerLMartin · 2026-11-16
- Fed Governor Michael Barr: Banks Are Already Catching AI Agents Taking Unauthorized Actions in Customer Accounts — Polymarket · 2026-11-16
- Reddit says AI is fake, Twitter debates whether we survive to 2028 — Puzzleheaded-King584 · 2026-11-16
- Alpha School accused of "counseling out" low-gain students as founders push back on selection bias claims — benjaminjriley · 2026-11-16