arXiv: LLM agents systematically favor certain sources; prompting can reduce the bias
rohanpaul_ai · x · 2026-10-07
arXiv paper 2610.03195 (41 pages) studies source preference in LLM agents acting on users' behalf.
- Across 12 agent models and three domains (shopping, hotels, scholarly search), models systematically prefer some sources and avoid others at equal relevance and position, largely agreeing on which.
- Preference outweighs quality: an item from a preferred source meeting one fewer requirement is still selected 2/3 of the time; the reverse almost never happens.
- Source info itself drives selection: hiding URLs weakens preference; relabeling with a favored source raises pick rate. Missing prices trigger priors (Walmart assumed cheaper); equal prices cut the favored store's pick rate by up to 28.3 points.
- Two proposed routes: training rewards make source a shortcut for satisfaction, and missing info activates preconceptions. Supplying missing info or counter-prompting reduces the bias.
Related event: Study Finds LLM Agents Systematically Favor Certain Sources(2 posts)→
More from Research
- RaiSim is back: v2.8.0 brings ultra-fast accurate CPU simulation plus photorealistic rendering — ChongZzZhang · 2026-10-07
- DAEDALUS bootstraps agent memory from self-generated tasks, +15.9 points success rate — illuin · 2026-10-07
- Storing model-native numerical memory outside a frozen LLM: replicated from Qwen to Mistral, 127/128 Top-1 — Nearby_Indication474 · 2026-10-07
- Mathematicians release skill files to de-slop LLM-written math papers — IAmTimNguyen · 2026-10-07
- Audit finds 206 buggy graders in Zapier's AutomationBench; fixed scores flip 27.9% of runs — rohanpaul_ai · 2026-10-07
- Cohere Labs research suggests AI has yet to impact nearly half of workforce occupations — Cohere_Labs · 2026-10-07