ACL 2025 Study: Raters Can't Tell AI Text From Human, But Prefer Anything Labeled "Human" by 30%
ipeirotis · x · 2026-09-10
A Zhu et al. study (ACL 2025 Findings) ran three experiments on text rephrasing, summarization, and persuasive writing. In blind tests, raters could not distinguish AI from human writing — yet once labels appeared, they preferred text labeled "Human Generated" by over 30%, and the same pattern held when labels were deliberately swapped.
The authors argue this bias against AI systematically undervalues AI performance, with implications for human-AI collaboration, especially in creative fields.
Related event: Blind Tests Show People Can't Distinguish AI Writing—Until Labels Bias Them(2 posts)→
More from Research
- FrogNano: a 4B model trained purely with RL on synthetic tasks hits repo-level coding — burkov · 2026-09-10
- Stanford lab rebuilt as interactive 3D web scene in a day with GPT-6 Astra and Retriever — OfirPress · 2026-09-10
- VisionCoach: RL framework rewards correct visual attention for grounded video reasoning, SOTA zero-shot — mohitban47 · 2026-09-10
- Goodfire explains how probes can read model minds to catch cyber intent and reward hacking — leland_mcinnes · 2026-09-10
- Engineer deploys hundreds of parallel AI agents to work on a type 1 diabetes cure — Scobleizer · 2026-09-10
- SOFAIR lab launches with UCL, Cambridge, Oxford, Edinburgh to do Science for AI — latticecut · 2026-09-10