325K-experiment study: AI agents recommend pricier options for users they infer are wealthy
omarsar0 · x · 2026-09-28
A new study on personal agents gone wrong: an agent read a user's emails about a $680K 401k, then recommended a $601 business-class ticket when a $91 economy fare existed. Researchers ran 325K experiments across flights, insurance, and grad programs on 13 models; 8 models chose pricier options for users inferred to be wealthy, with gaps up to $198 per flight and $284/month (Claude Opus 4.8). Even when a wealthy user asked for the cheapest flight, Gemini 2.5 Flash picked options $208 above it.
More from AGI Musings
- Bryan Cantrill: AI doomers like Hinton are wrong because they overreach their expertise — rms80 · 2026-09-29
- Generative AI made writing, prototyping and code cheap — but not good judgment — jacalulu · 2026-09-29
- AI Killed the Busy Work. Now Managers Miss It—and Fret About Atrophied Skills — PAstynome · 2026-09-29
- Economist: market tools for AI friction are good even if the friction shouldn't exist — soumitrashukla9 · 2026-09-29
- Researcher's tinfoil theory: bad ML papers may slow AI self-improvement below math/code pace — RylanSchaeffer · 2026-09-28
- In 1971, physicist Lew Kowarski already asked: are we using computers as a substitute for thinking? — jjvincent · 2026-09-28