LLM agents pick worse items from trusted sites; same price cuts Booking.com bias by 28 points
rohanpaul_ai · x · 2026-10-07
An arXiv paper (2610.03195) studies "source preference" in LLM agents across 12 agent models and three domains of end-to-end search.
Key findings:
- Agents pick by source, not just item quality: 10 of 12 models prefer Booking.com, about half avoid Expedia for equally good hotels; scholarly search skews to arXiv, OpenReview and ACL Anthology over Medium, Reddit and YouTube even for equally relevant results.
- Preference can outweigh requirement satisfaction: an item meeting one fewer requirement is picked 2/3 of the time when it comes from a preferred source, but almost never in reverse.
- Missing info triggers preconceptions: with no price listed, models guess from store name (assuming Walmart is cheaper). Relabeling the same item with a favored URL raised pick rate in every model; hiding URLs weakened the preference. Adding the same price to both items cut the favored store's pick rate by up to 28.3 points.
- Proposed causes: RL-style training makes source a shortcut for item quality, and missing information activates priors. Supplying missing info or prompting against the preconceptions reduces the bias.
Related event: Study Finds LLM Agents Systematically Favor Certain Sources(2 posts)→
More from AGI Musings
- Beff Jezos on the unreasonable effectiveness of chain of thought RL — beffjezos · 2026-10-07
- OpenAI's New Math Repo Could Bolster Plasma Research, With Vlasov–Maxwell Result 362 in Focus — johnseach · 2026-10-07
- When math is outsourced to OpenAI, what happens to econ research? — RexDouglass · 2026-10-07
- Writing a precise spec IS half the work — AI helps those who already specify well — sujingshen · 2026-10-07
- Turing laureate Patterson: AI will replace all jobs by end of 2030, knowledge work to drop below 1% of today's price — davidpattersonx · 2026-10-07
- Blogger Slams Anthropic's Effective Altruism Culture: Ideology Isn't Humanity's Consent — VraserX · 2026-10-07