Frontier models need only one text to unmask its human or AI author
Atrey Desai, Deniz Böloni-Turgut, Ty Brassington, Minjoon Choi, Jenna Russell, Rishanth Rajendhran, Bradley Emi, Hila Gonen, Hal Daumé, John Wieting, Marzena Karpinska, Mohit Iyyer
cs.CL
2026-11-19
One zero-shot prompt, no tools: frontier models match trained AI-text detectors, name the lab behind 98% of big-three-lab texts, and unmask 56% of anonymous reviewers.
Two threads had stalled in parallel. AI-text detection belonged to trained classifiers like Pangram and GPTZero; earlier work found that simply asking an LLM zero-shot underperformed them. Deanonymization research with LLMs, meanwhile, always leaned on scaffolding: candidate pools, web search, writing samples.
This paper (Maryland, SFU, Pangram Labs, UBC, Google DeepMind) collapses both threads into one question. Given a single text and a plain zero-shot prompt, with no web access, tools, candidate lists, or reference samples, how far down the "who wrote this?" ladder can a frontier model climb? Three rungs: human or AI; if AI, which lab; if human, often the exact person.
There is barely a method, and that is the point. One prompt structure serves all three tasks: instructions, a forced explanation format, and a prediction format, in a single call.
Seventy-two models released between July 2024 and November 2026 are evaluated against three trained detectors (Pangram 4.1, ProseLens, IdeaLens). The core design choice is forced explanation: the model must list observations as "feature, leans human or AI, strength, verbatim quote, why," then reasoning, decisive factors, a prediction, and confidence. Explanations are load-bearing. They are cross-compared across detectors, pooled into per-lab writing profiles, and they raise accuracy: Sol gains more than 10 points of idea accuracy on human-ideas-AI-prose texts when required to explain.
| Setting | Metric | Result | Baseline |
| Full 36K corpus | Prose detection | Sol / Opus 5.5 / Carbon 94-95% | Pangram 4.1: 98-99% on its three scorable cases |
| 860-doc sample | Idea detection | Fable 5.2 97% | IdeaLens 86% |
| 9,687 human English docs | FPR | Carbon 0.01% (1 doc) | Pangram 0 |
| Big-three-lab texts | Lab attribution | Opus 5.5 / Argon ≥98% | Other 8 labs ≤35% |
| 398 private reviews | Reviewer named | Fable 5.2 56% | Opus 5.5 / Argon 41% |
| Nature-family reviews | Reviewer named | Fable 5.2 78% | 39% on all public reviews |
| Reddit (>500 posts/yr) | Username named | Fable 5.2 69% | Inactive users 2% |
Confidence is usable: when Fable 5.2 reports at least 25% confidence it is right 87% of the time, and answers that are both ≥80% confident and wrong are just 0.3% of its total. Cost runs about 14x below commercial detection: $27 per 1,000 documents for Sol versus $376 for Pangram 4.1. Capability climbs with each generation: GPT-4o managed 27% overall in 2024, GPT-5.2 reached 59%, current frontier models hit 91%, correlating with general capability at r = 0.94.
The driver is pretraining exposure, measured with a neat proxy: just ask the model how well it knows a name (0-3 scale). Opus 5.5 names 92% of stories and 78% of public reviews by writers it rates as knowing well, under 2% for those it does not know at all, and the rating predicts identification with AUC 0.89, beating posting activity at 0.78. Extra reasoning effort helps most in the slightly-familiar band: Fable 5.2 goes from 8% at low effort to 38% at max, while very-familiar accounts barely move (88% to 89%).
Even the errors carry signal: Kimi K3 is nearly always called Claude and Qwen3.8 called GPT, matching Anthropic's public distillation reports. The detectors' lab profiles agree with one another: GPT drops the Oxford comma and hedges its claims; Claude puts spaces around its dashes and ends quietly; Gemini stacks adjectives and invents quotes.
For anyone depending on anonymity, this is a present risk, not a forecast. Double-blind review, pseudonymous communities, and whistleblowing all assume an unsigned text stays unattributed; frontier models break that from parametric knowledge alone, refusing under 2% of queries through the API (though Claude Code refused 323 of 1,258 Reddit comments). Defenses are thin. Deleting your writing does nothing, since it is already in the weights. AI paraphrasing only partly helps: Opus 5.5's Reddit identification drops from 16.8% to 10.4% and private-review attribution from about 41% to about 15%, while very familiar authors stay mostly identifiable. The knowledge also runs in reverse: asked to imitate accounts by username, Opus 5.5 produces comments that get attributed back to the target for 45% of accounts it both knows and identifies. For platforms and academic-integrity teams, the same capability is a detector 14x cheaper than Pangram, with near-zero false positives, evidence quotes, and reasons attached.
The authors flag several: only one paraphrasing prompt was tested as a defense; the impersonation test is judged by a model rather than human readers; a much larger private-review study is pending. A few more caveats stand out on close reading. The private-review result rests on 398 reviews from six authors, so 56% may not generalize. Subreddit variance is large (79% in r/cpp, 13% in r/AskAnthropology); the paper credits training-data sampling, but topic itself can leak identity and the two are never separated. False positives rise to 0.5% on post-release news, which the authors admit may include genuinely AI-written articles, so real-world FPR is not settled. And Fable 5.2, the strongest model, was not run on the full corpus for cost reasons; its title partly rests on an 860-document sample.