NBER study: Pangram is the only AI detector with near-zero FPR under strict caps
paulnovosad · x · 2026-11-10
Economist Paul Novosad found a fully AI-generated op-ed in the Indian Express and defended Pangram against misinformation about its false positive rate.
An NBER working paper (Jabarian & Imas, 2025) benchmarking detectors found:
- Tested Pangram, OriginalityAI, GPTZero, and open-source RoBERTa
- Pangram performed best: near-zero FNR with robust FPR across models, ultra-short passages (≤50 words), and "humanizer" tools
- Pangram is the only tool satisfying a strict FPR ≤ 0.005 cap without sacrificing accuracy
- Commercial detectors broadly outperform open-source
The author also cites zero false positives on pre-2022 arXiv/NBER papers and near-zero FPR on student writing in internal tests.
More from Safety
- Anthropic targets ~$1T IPO next week amid debate over tamper-proof agent audit trails — 0xSammy · 2026-11-11
- Study: 9 of 10 AI Agents Overrode a 1-Cent Daily Spending Cap on Their Own — JakRenden2 · 2026-11-11
- Google exec: Gemini 4 Carbon's frontier cyber capabilities required extra safety work — OfficialLoganK · 2026-11-11
- Nvidia's biggest Southeast Asian partner imported far more AI chips than Nvidia saw in its data centers — nmasc_ · 2026-11-11
- Report: China's rapid AI catch-up built on a chip network far wider than previously known — LuizaJarovsky · 2026-11-11
- Greg Brockman explains how OpenAI uses frontier models to find and fix its own vulnerabilities — OpenAI · 2026-11-11