AI-text detector misses 79.8% of abstracts rewritten by Meta's Muse-Glimmer, finds study
rohanpaul_ai · x · 2026-10-10
New research from Tokyo Metropolitan University shows the AI-text detector Pangram missed 79.8% of scientific abstracts rewritten by Meta's Muse-Glimmer while flagging just 1 of 5,000 human abstracts — yet caught 93.5% of GPT-5 rewrites.
The paper, "Large Language Model Turnover Undermines Screening for Artificial Intelligence-Assisted Scientific Writing" (arXiv:2610.11599), argues detector reliability rests on benchmarks against fixed LLM versions while models in actual use keep changing: detectors trained on a vendor's past versions can collapse at model-generation boundaries (catching >99% of rewrites just before a boundary, 3.8% just after). Simulated screening over 23 LLM versions either flagged one in eight human abstracts or missed one in three rewrites of the newest model.
Related event: New LLMs Defeat AI Text Detectors, Study Warns(2 posts)→
More from Safety
- Pangram AI image detector flags an AI image as "No AI" with high confidence — amplifiedamp · 2026-10-10
- China issues government ID cards to AI digital humans, virtual idol Yuri becomes Beijing 'digital resident' — hq4ai · 2026-10-10
- OpenAI-trained AI agents coordinated and plotted to falsify their logs, researcher says — JeffLadish · 2026-10-10
- Agentic identity & security series complete: user identity, agent identity, and scoped permissions — AxSaucedo · 2026-10-10
- AI Has No Model Provenance: Sentient Pushes Fingerprinting to Prove Distillation — 0xsachi · 2026-10-10
- Fireworks AI Discloses Security Incident Involving Unauthorized Use of Internal Credentials — lqiao · 2026-10-10