Emily Bender: Pangram is not an oracle — why 'accuracy' claims of AI detectors mislead
mjdramstead · x · 2026-10-28
Timnit Gebru shares Emily Bender's essay "Pangram is not an oracle," prompted by the controversy over accusations that Caine Prize-winning story "The Blue Fighter" was LLM-generated.
Key points:
- AI text detectors like Pangram are widely adopted in publishing; claims of being "extremely accurate" reflect effective marketing more than meaningful evaluation, and the public misunderstands what "accuracy" means for language tech.
- The larger story is racism: software sold as "AI" clothes bias in a veneer of objectivity, placing impossible credibility burdens on racialized authors, especially Black authors.
- Bender acknowledges synthetic text is a real problem but warns that a machine that outputs "scores" can easily look like a solution without being one.
More from AGI Musings
- Big Tech fired hundreds of thousands, then hired again — was AI really the reason? — TansuYegen · 2026-10-28
- Mark Cuban: Run Every Benefits Contract Through AI to Find Where You're Getting Ripped Off — a16z Podcast · 2026-10-28
- DeepMind's Paige Bailey on farm robots: ag startups wait 6-9 months just to access a field — DynamicWebPaige · 2026-10-28
- GPT-2 staged release debate revisited: setting a caution norm vs overhyped PR — gleech · 2026-10-28
- Smash Bros creator Masahiro Sakurai says AI-generated games only copy existing ideas — aakashgupta · 2026-10-28
- Scientists may lose taste for picking problems as agents speed up experiments — charlieharris01 · 2026-10-28