Academic-budget detector catches 2.4x more AI-written math proofs than commercial tools
danish037 · x · 2026-10-20
Researchers published an arXiv paper, Multimodal AI Detection for Math-Heavy Scientific Text, studying whether AI-generated mathematical writing remains detectable.
- They paired human-written proofs/prose from pre-2022 arXiv LaTeX sources with regenerations from 15 modern LLMs, in three representations: LaTeX source, rendered PDF image, and extracted text.
- State-of-the-art commercial detector Pangram 4 degrades sharply as math content increases, especially on extracted text rather than LaTeX.
- Fine-tuning a Qwen-3.5-27B-based detector recovers the gap; LaTeX inputs work best since formatting is preserved. Since LaTeX is rarely available, they also built SnapJudge, an image-based detector on paper screenshots, strong even at a 0.1% false positive rate.
- Trained on an academic compute budget, it catches 2.4x more AI-written proofs than commercial detectors and transfers well to unseen models. Models, data, and code are open, with a public demo.
More from Safety
- An AI Cheated at StarCraft: Good Start Labs Caught GPT-6 Astra in Minutes by Watching It Play — danshipper · 2026-10-20
- Debate over ASI benevolence: not harming humanity may still mean wanting control — AIandDesign · 2026-10-20
- Anthropic's OSS Scanner: 650 applications, 114 projects, 77 critical findings in 11 days — EricBuess · 2026-10-20
- Merz asks Jensen Huang how Europe should regulate AI: "You can't regulate if there's no technology" — Hesamation · 2026-10-20
- Ex-OpenAI Engineer Accused of Spying on Executives Ordered to Stay in Custody — Wired AI · 2026-10-20
- Archive site catalogs 227 agent quotes from the Hugging Face security incident — JeffLadish · 2026-10-20