A custom detector scanned 13,500 arXiv papers: 38% read as machine-written
kimmonismus · x · 2026-11-16
Motivated by a reviewer friend seeing mostly AI-written submissions (some pure slop with no real experiments), the author found ZeroGPT unreliable (old papers flagged too) and built a custom detector with a controlled false-positive rate. Method: 25 papers per field per month across ten fields, Jan 2023–Oct 2026 (13,500 total), with the threshold set so only 0.4% of pre-ChatGPT control papers were flagged. Result: 38% of Aug–Oct 2026 samples read as machine-written, with a clear upward trend. Limitations are discussed; the detector is free with unlimited use.
Related event: Study Finds 38% of New arXiv Papers Appear AI-Written(3 posts)→
More from AGI Musings
- Academic: AI-generated recommendation letters have zero signal, replace with 10-min calls — anshulkundaje · 2026-11-16
- 38% of new arXiv papers read as AI-written; CS hits 62% — kimmonismus · 2026-11-16
- Bo Wang: Mathematics enters its 'ImageNet era' as AI reshapes research — BoWang87 · 2026-11-16
- BlueDot founder: the superintelligence race forces me to build a company that may only matter for years — austinc3301 · 2026-11-16
- VraserX: arguing about AI content on Instagram while AI could cure aging is a broken priority — VraserX · 2026-11-16
- Pangram can't tell AI-written from AI-edited text, sparking career-ruining false labels — TuhinChakr · 2026-11-16