72,000 Turing-style tests show AI now fools human judges — and AI detectors beat us
hpfister · x · 2026-10-29
A large-scale study by teams including Harvard's Pfister and Kreiman, published in Nature Human Behaviour, ran over 72,000 Turing-style tests across vision and language tasks to examine whether humans can distinguish AI answers from human ones.
Key findings:
- Modern AI models have become strikingly good at imitating humans in everyday language and vision tasks, often fooling human judges with ease as the gap continues to narrow;
- More humbling still, basic AI evaluators outperformed humans at detecting human-produced answers.
The study spans six task types—object recognition, color recognition, attention prediction, image captioning, word associations, and conversations—with code and data openly available from the Kreiman Lab website.
More from AGI Musings
- Ofir Press: AI's jagged frontier advances unintuitively — math robots beat self-driving cars — OfirPress · 2026-10-29
- Ofir Press thread: math is more concrete and RL-able than SWE — OfirPress · 2026-10-29
- Ofir Press: Millennium Problems is AI's best benchmark; 70% of open math could fall in a year — OfirPress · 2026-10-29
- Musk says AI and robotics output 'will dwarf everything else,' names Google and Nvidia as top picks — elonmusk · 2026-10-29
- levelsio urges X to adopt zero-tolerance lifetime bans for AI replies — damnGruz · 2026-10-29
- Researchers who turned down Anthropic and OpenAI: choosing moral ambition over lab wealth — AaronBergman18 · 2026-10-29