Pangram false positives are nearly nonexistent — Bender's high-FP prediction fails a cash-bounty test

FlorianGallwitz · x · 2026-11-19

Byrne Hobart examines Emily Bender's hypothesis that AI text detectors like Pangram should have high false-positive rates — yet known false positives are almost absent. A cash bounty for pre-LLM text scored as AI-produced was won only by a funny 2006 poem. Hobart argues LLMs have stacked statistical tells ("It's not X, it's Y", rule of three, sentence-length distributions) that make detection work, and that the episode shows how theory-driven predictions about AI should be tested against counterexamples.

Original post →

More from Models

Models channel →