Professor runs Pangram on 50 pre-ChatGPT self-published novels: zero false positives
TuhinChakr · x · 2026-10-31
Citing Vauhini Vara's Bloomberg report, the author shares that Chakrabarty, now a CS professor at Stony Brook, ran the Pangram AI detector on 50 randomly sampled self-published novels from before ChatGPT's introduction and found a false positive rate of zero.
The report also notes Pangram's cited false-positive rates come from tests across varied document lengths — errors are more common on short texts but drop significantly as text length grows. Internal testing puts the book-level false positive rate at essentially zero, reinforced by studies from Vrije Universiteit and the University of Chicago.
Related event: Professor tests 50 pre-ChatGPT novels, AI detector flags none(2 posts)→
More from Safety
- Wired podcast: a contractor's Claude agent got into a Pentagon system — Wired AI · 2026-10-31
- Tyler Alterman on the hacking panic: more Y2K than COVID, use 1Password and update software — jd_pressman · 2026-10-31
- Anthropic's VS Code extension on Cursor backdoored for nearly 12 hours, up to 5,000 devs exposed — Ars Technica AI · 2026-10-31
- Security researcher: vibe coding will force software to consolidate around mature products — jd_pressman · 2026-10-31
- Every Used Codex and Agents to Survive the Railway Credential Breach — every · 2026-10-31
- Researchers debate whether more inference compute can substitute for training compute — teortaxesTex · 2026-10-31