Only 2 of 25 full AI rewrites flagged as 100% AI — detector experiment backsfires
paulnovosad · x · 2026-10-26
Economist Paul Novosad shared experiments with the AI detector Pangram:
- Asking the latest ChatGPT (G5) to fully rewrite 25 essays resulted in only 2/25 being flagged "100% AI-generated"
- The reason: AI rewrites preserve most of the words you supply, and the residual human phrasing keeps the detector from a full score
- The only reliable way to trigger "100%" was compressing essays into bullet outlines — and even that often fell short
The inversion is notable: if your work is consistently flagged 100% AI, you likely never wrote your own first draft. Starting from your own draft makes a pure-AI flag extremely unlikely. Commenters added that Word Copilot and Grammarly also run LLMs, so students may be using the same models unknowingly.
More from AGI Musings
- Frontier LLMs negotiate contracts poorly and frequently renege for selfish gain, study finds — xuanalogue · 2026-10-26
- Study of 133 patent lawyers: AI augments veterans but causes cognitive offloading in novices — xiaohu · 2026-10-26
- OpenAI computer-use agents near superhuman speed, say the team building them — a16z · 2026-10-26
- Kai-Fu Lee publishes AI Native, a roadmap for building AI-native companies — kaifulee · 2026-10-26
- RCT: ChatGPT temporarily boosts consultants' data science skills, gains vanish without it — EricTopol · 2026-10-26
- Dev on AI games backlash: reject AI slop, not the tool itself — ezshine · 2026-10-26