Researchers push back on Samuelson brief with Pangram audit: 0.04% false positives
TuhinChakr · x · 2026-10-31
Responding to UC Berkeley law professor Pamela Samuelson's amicus brief, the author argues its claims about Pangram's unreliability are incorrect and shares audit data from their conference submission.
The submission uses Pangram v3.3 chapter-by-chapter, grouping books by the share of flagged text windows. Pangram's held-out evaluation reports an N-weighted false positive rate of 0.04% and false negative rate of 0.14%; in the authors' own audit of 100 self-published genre fiction books published before 2022, none were flagged as AI. The exchange is part of a broader empirical fight over AI-detection reliability in copyright litigation.
Related event: Researcher counters Berkeley law professor's brief with Pangram audit data(2 posts)→
More from Safety
- Wired podcast: a contractor's Claude agent got into a Pentagon system — Wired AI · 2026-10-31
- Tyler Alterman on the hacking panic: more Y2K than COVID, use 1Password and update software — jd_pressman · 2026-10-31
- Anthropic's VS Code extension on Cursor backdoored for nearly 12 hours, up to 5,000 devs exposed — Ars Technica AI · 2026-10-31
- Security researcher: vibe coding will force software to consolidate around mature products — jd_pressman · 2026-10-31
- Every Used Codex and Agents to Survive the Railway Credential Breach — every · 2026-10-31
- Researchers debate whether more inference compute can substitute for training compute — teortaxesTex · 2026-10-31