Study finds all 13 major LLMs flip truth judgments on speaker gender, up to 23.6% of statements
anthara_ai · x · 2026-09-03
An experiment across 13 popular LLMs found every model shows statistically significant negative sentiment toward men.
Method: researchers altered only the speaker's gender presentation (neutral/male/female) and measured whether truth judgments changed.
Key findings:
- 10%–35% of statements received completely inconsistent truth labels solely due to gender presentation
- Flip rates hit up to 23.6% when comparing male vs. female variants
- Every tested model exhibited gender sensitivity
The results highlight hidden consistency risks for automated systems that rely on LLM judgments.
More from Models
- Polish dev defends open model: built on Gemma with more than a simple LoRA fine-tune — ColinWPLewis · 2026-10-21
- X trends: SophontAI's Holon-S1 predicts gene activity from cancer slides; decision models rise — iScienceLuvr · 2026-10-21
- OpenAI's MultiLangBench: PHP scores highest (91.3%) at building a database engine from scratch — ricklamers · 2026-10-21
- 704-vote robotics Elo ranks Claude Opus 5.5 first at 1379; no model completes the task — qinzytech · 2026-10-21
- DeepSeek V4 called 'god-tier infra, 4th-tier ML': 7 tricks stacked, 20 patches to fix them — teortaxesTex · 2026-10-21
- GPT-6 Astra and Claude Fable 5.1 write $10,000 research papers — experts reject all — sayashk · 2026-10-21