Humans Spot AI-Rewritten Text 64% of the Time in 1,636-Answer Test
dejanseo · x · 2026-10-09
- dejan.ai's AI vs Human test collected 1,636 answers in two days: 64% overall accuracy vs a 50% coin flip, with 18 perfect 10/10 rounds against 0.14 expected from guessing.
- Each round shows 10 news stories with matched excerpts (30–120 words, mid-article) from a journalist's original and an LLM rewrite; players pick the machine-written one, with answers verified server-side and a permanent leaderboard.
- Reading time was the strongest factor: answers under 5 seconds scored 55% correct, while answers after 40+ seconds hit 75% — the longer people read, the easier AI text is to spot.
- The corpus draws from 10,000 matched pairs, deliberately avoiding openings and sign-offs where AI giveaways cluster.
More from Research
- Could Looped Models Resist Distillation Attacks by Reasoning in Latent Space? — moyix · 2026-10-09
- LLM-as-a-Verifier: Weaker Model Verifies Stronger One, Hits 69.2% SOTA on Terminal-Bench 4 — Azaliamirh · 2026-10-09
- Salesforce's SRD distills hindsight into foresight, lifting 2B agent success from 0% to 60.6% — Salesforce · 2026-10-09
- Claude claims discovery of new binary red dwarf pair ~500 light-years away — nitarshan · 2026-10-09
- Prompt Tuning Is Forgotten Lore — Are We Massively Underusing Finetuned Tokens? — cephaloform · 2026-10-09
- Ricardo Baeza-Yates Lecture: When Will ML Evaluation Stop Fooling Itself? — PolarBearby · 2026-10-09