Study: LLM Agents Pick by Source Preference, Overriding Item Quality Two-Thirds of the Time
SeoulNatlUniv · hf · 2026-10-05
Seoul National University researchers tested 12 LLM agent models for source preference in end-to-end search across shopping, hotel booking, and paper citation.
Key findings
- Every model prefers some sources and avoids others in all domains, largely agreeing on which.
- Preference can outweigh requirement satisfaction: an item one requirement short from a preferred source wins about two-thirds of the time against a better item from a dispreferred source; the reverse almost never happens.
- Hiding source info weakens the preference; relabeling with a preferred source raises selection rates.
Causes and mitigation: reward-driven training can make source a shortcut for quality; missing info triggers preconceptions. Supplying missing info or countering preconceptions in prompts reduces the bias.
More from Models
- ChatGPT is now 'full of upsell slop', complains paying Pro plan user — weswinder · 2026-10-05
- Matt Shumer: Claude Opus 5.5 with Codex-level computer use would be unbeatable — mattshumer_ · 2026-10-05
- GPT-6.1 Sol Quota Nearly Impossible to Exhaust at Medium, User Observes — bytebot · 2026-10-05
- Users frustrated as Google removes Gemini Flash, disrupting audio workflows — Th3F4llen1 · 2026-10-05
- Redditor claims GPT-6 Astra made optimized quantum circuit code another 10x faster overnight — 141_1337 · 2026-10-05
- Power user review: ChatGPT Dots hit abuse-prevention limits even on the $200 plan — invertednz · 2026-10-05