Decision models are shockingly sensitive to question order and random noise
antoine_chaffin · x · 2026-10-23
Hugging Face researcher antoinechaffin confirms observations that decision models are surprisingly sensitive to the order of questions and fluctuate when asked the same question twice. He notes that all model decisions are sensitive to "a bunch of random stuff," and while capability gains are still early, the community should soon work on making models less biased and more robust — "because it's a mess."
More from Models
- SEO testing: LLMs are bad at anchor selection but surprisingly good at picking target URLs — gaganghotra_ · 2026-10-23
- NVIDIA releases Nemotron 3.5 Bolt E4B, a 4GB open model for on-device conversational agents — NVIDIAAI · 2026-10-23
- NVIDIA releases open Nemotron 3.5 Bolt E4B 4GB with weights, data and recipes — NVIDIAAI · 2026-10-23
- Gemini 'Argon' bug reportedly fails to analyze uploaded audio files — Numerous-Campaign844 · 2026-10-23
- 2-bit quantized DeepSeek 4.1 Flash tested locally: KDL degrades but still works — bakawolf123 · 2026-10-23
- KOL predicts K3.1 benchmarks sight unseen: benchmaxxed, below Sonnet in practice — teortaxesTex · 2026-10-23