Semantic bandit experiments show irrelevant wording skews LLM exploration-exploitation balance

burkov · x · 2026-11-01

Andrey Burkov highlights research on LLM reliability in sequential decision-making: as LLMs increasingly choose among options under incomplete information, their ability to balance exploring alternatives against repeating known successes becomes a practical concern. The study asks whether language itself—not just rewards and choices—shapes that balance.

Key points:

The paper introduces a "semantic bandit" test: three options with different, initially unknown rewards, while option names and scenario framing vary without changing the underlying reward structure.
Three language models were tested across abstract, farming, and clothing-recommendation settings, comparing meaningful labels against random letter-and-number labels, and measuring exploration and missed opportunities.
Conclusion: seemingly irrelevant wording can systematically steer an agent toward either helpful or seriously mistaken decisions, with direct implications for the reliability of LLM-based autonomous agents.

Original post →

Semantic bandit experiments show irrelevant wording skews LLM exploration-exploitation balance

More from Research