An 'Amazon's Choice' label flips LLM picks: three NeurIPS 2026 bias papers
xuandongzhao · x · 2026-10-02
Three papers from the same team, all accepted to NeurIPS 2026, examine whether LLMs judge by packaging rather than content — and whether the bias can be planted and removed.
BiasRecBench
- Adding authority cues to the inferior option alone drops Gemini-3-pro's accuracy from 95% to 57% on paper selection; "everyone is buying" cues cost four models 18.5–24.5 points in e-commerce
- DeepSeek-R1 switched its deep-fryer pick after a rival gained [Amazon's Choice][Best Seller] labels, reasoning "because it's a best seller"
- Bias only kicks in when options are close in quality — exactly when users need help most
BiasTrojan
- Fine-tuning Qwen2.5-14B on 1,200 poisoned examples leaves normal-task accuracy at 79%, but accuracy collapses to 6% when authority cues appear
- Five existing data-cleaning methods fail to filter these samples; retraining on clean data for 10x longer doesn't fully remove the bias
EIT (Treat Bias as Noise) proposes mitigating bias by treating it as noise during training.
The authors warn this undermines LLM-as-a-Judge: biased judges silently skew filtered training data while appearing perfectly normal on clean benchmarks.
More from Models
- Local AI community worries growing dependence on Claude and GPT strengthens closed labs — takoulseum · 2026-10-02
- Dev swaps prod system from GPT-5.4 to GLM: faster, cheaper, far more reliable — ivan_bezdomny · 2026-10-02
- Unreleased Gemini 4 Argon reportedly matches Claude's best on 3D game generation — 141_1337 · 2026-10-02
- "Visualize the biggest scam in humanity": Opus 5.5's answer goes viral — zealcaiden · 2026-10-02
- Developer says he's burned 6 billion tokens on the Grok API with near-zero downtime — Daniel_Farinax · 2026-10-02
- AI2: AstaBrief Fast Mode Is 3.5× Faster Than Claude-Powered Mode at Similar Quality — allen_ai · 2026-10-02