EMNLP study: LLMs attribute biased text to groups, refusing only 0.4% of the time
ZhijingJin · x · 2026-10-27
A study accepted to EMNLP 2026 Findings reveals a systematic flaw in how LLMs handle biased text attribution.
Key finding
- Researchers asked LLMs who would accept a biased text, with no demographics given — the correct answer is "Unknown"
- Models nonetheless name a group anyway, often a dominant one, and refuse only 0.4% of the time
Reasoning mode and second-order bias
- GPT-5.1's reasoning variant raises second-order bias by 50% on acceptable text and 128% on unacceptable text
- For Sonnet 4.6 and Olmo, reasoning mode actually lowers second-order bias
The work comes from JinesisLab, University of Toronto, Vector Institute, and collaborators.
More from Models
- repligate: noisy context is genuinely 'hard' for LLMs, beyond metaphors — repligate · 2026-10-27
- The trust paradox: only the least trustworthy LLMs will do the sketchiest things — AaronBergman18 · 2026-10-27
- Integrity and refusal may correlate: the fundamental tradeoff behind password refusals — AaronBergman18 · 2026-10-27
- Claude's refusal to type one-time passwords is a real security impediment — AaronBergman18 · 2026-10-27
- repligate argues context full of noise is genuinely 'hard' for LLMs, mulls an info-theory model of breaks — repligate · 2026-10-27
- Fork builds engine to run Qwen at ~400 tok/s on three used V100s — jfinkpottery · 2026-10-27