Study (n=504): suspicion doesn't improve AI-text detection; fake-news accuracy drops 10.2 points
bit3py · reddit · 2026-09-02
A human-subject study (n=504 participants, 2,438 judgments) had participants classify news fragments along two axes: origin (human vs machine) and veracity (real vs fake). Preprint is open access (arXiv).
Three surprising findings:
- Perception-accuracy gap: more suspicious participants were not better at detecting machine-generated text — "just be more skeptical" media-literacy advice doesn't translate into accuracy
- Modern LLM output was frequently indistinguishable from human text
- Asymmetric cognitive fatigue: under sustained exposure, fake-news detection degraded by 10.2 percentage points while AI-origin detection stayed roughly stable — the two judgments draw on different resources, and only one wears out
The study adapts the cybersecurity kill chain, framing disinformation as a staged lifecycle and arguing for earlier interventions instead of relying on a tired human at the end of the chain. The author notes that platform designs forcing people to evaluate more content could quietly worsen veracity judgment.
More from Safety
- AI detector startups accused of pushing "AI score = slop" misuse to survive — manoelribeiro · 2026-09-02
- US government tells OpenAI lawsuits: AI training is fair use, dilution theory 'deeply flawed' — Formal_Drop526 · 2026-09-02
- Ethan Mollick: AI can align flaws in complex systems, forcing a new defense philosophy — emollick · 2026-09-02
- a16z lobbyist calls AI-regulating state legislators "bugs" as Politico charts Silicon Valley's state law defeats — GarrisonLovely · 2026-09-02
- Provcheck v1.4.0 ships a ComfyUI node for in-graph watermarking and C2PA signing — ckn · 2026-09-02
- UK government doesn't know what its datacentres are used for, FoI request shows — nordicinst · 2026-09-02