Swapping option order changes the answer: an LLM prompt sensitivity demo
helloiamleonie · x · 2026-11-06
The author shares a concrete example: when presenting multiple options to an LLM in a prompt, simply reordering them causes the model to return a different answer each time. This order sensitivity means results from evaluation- or decision-style prompts are unstable and worth watching out for in practice.
More from Models
- scaling01: The cope for Chinese models is just distillation from Fable — scaling01 · 2026-11-07
- DeepSeek leads the accuracy-cost Pareto frontier "not even close", says casper_hansen_ — casper_hansen_ · 2026-11-07
- ChatGPT leak: SSE stream exposes per-chunk retrieval scores, quotes worth 100 points — lilyraynyc · 2026-11-07
- $50K live trading battles: AI bots currently edge out Claude and GPT — Mr_Hodlerr · 2026-11-07
- Brazil's Maritaca Releases S5-nano 4B, Claiming Best Open Weights in Brazil — ApprehensiveAd3629 · 2026-11-07
- Grok Bot's computer internet access issue is fixed — nima_owji · 2026-11-07