Pangram false positives are nearly nonexistent — Bender's high-FP prediction fails a cash-bounty test
FlorianGallwitz · x · 2026-11-19
Byrne Hobart examines Emily Bender's hypothesis that AI text detectors like Pangram should have high false-positive rates — yet known false positives are almost absent. A cash bounty for pre-LLM text scored as AI-produced was won only by a funny 2006 poem. Hobart argues LLMs have stacked statistical tells ("It's not X, it's Y", rule of three, sentence-length distributions) that make detection work, and that the episode shows how theory-driven predictions about AI should be tested against counterexamples.
More from Models
- vLLM's DSpark for Kimi K3.1 Pro: 30% longer acceptance, bounded draft KV at 1M context — AccBalanced · 2026-11-19
- Gemini 4's element naming mocked: Carbon builds on Argon, Neon on Hydrogen — teortaxesTex · 2026-11-19
- ChatGPT web translation fails badly on long Chinese text, users say free DeepSeek is better — teortaxesTex · 2026-11-19
- Rumor: Gemini 4 is a 12T-param MoE with 500B active; ChatGPT translation fumbles it — teortaxesTex · 2026-11-19
- Abacus.AI open-sources Smaug-Uncensored, cutting Qwen3.8-27B refusal rate from 45% to 2% — bindureddy · 2026-11-19
- Dev observes Google's model launch pace has dramatically sped up since Argon — cgarciae88 · 2026-11-19