AI Writers Have a Consistent Stylometric Footprint, but AI Editors Do Not
Zhengyang Shan, Yukyung Lee, Sophie Hao
EMNLP Main 2026
cs.CL
2026-08-28
AI writing has a stable entropy-and-diversity footprint (91.74% linear acc. across 8 models, 5 domains). AI editing cuts lexical density (d=-3.10) and stays near the human source.
AI-text detectors assume a shared statistical stain: if a model touched the document, the same features should fire, whether the model wrote from scratch or edited a human draft. Detectors trained on fully generated text then get pointed at copy-edits, paraphrases, and expansions. That transfer was never tested on one compact, interpretable feature set.
Boston University asks two concrete questions. Does AI-generated text have a style that holds across models and domains? Do AI edits follow it?
They build GEN (Generated, Edited, and Naturalistic) in five English domains: Wikipedia, WikiHow, arXiv abstracts, Reddit QA, and Reddit stories. Human documents: 4,998, mostly from M4 and WritingPrompts. AI-generated: 39,916, written by eight open instruction models (Qwen-2.5 7B/72B, Gemma-3 12B/27B, Llama-3.1-8B, Llama-3.3-70B, GPT-oss-20B/120B). AI-edited: 273,420, from 1,000 human seeds, 31 prompts, eight edit types (grammar, fluency, clarity, tone, paraphrase, add detail, restructure, condense), three large editors, target edit ratios 30%/50%/70%. Actual edit ratio is word-level Levenshtein, measured after the fact.
Each document gets 14 interpretable features: lexical diversity and density, entropy, burstiness, readability, punctuation, parse-tree depth, and related surface stats. The detector is L2-regularized logistic regression on standardized features, averaged over ten class-balanced draws. Importance is both the absolute coefficient and the accuracy drop after shuffling that feature.
Generation has a thin, stable footprint. Versus human text, AI-generated documents sit about 1 standard deviation higher in entropy and about 2 standard deviations higher in lexical diversity. With only these 14 dimensions, human vs generated accuracy is 91.74%±0.61 globally, 92.57%–95.83% per domain, and 92%–99% for most generators. GPT-oss-120B is the outlier at 79.45%, the most human-like generator in the set.
Across 14 training conditions (global, per-domain, per-model), lexical diversity ranks in the top five 13 times and entropy 12 times; both have the highest stability scores. Burstiness ranks third globally and then fails to travel. Temperature 0.3/0.7/1.0 yields 82.2%/80.7%/79.5% accuracy, with mean Spearman ρ 0.89 on feature ranks. Proprietary add-on: Gemini-3-Flash 82.1%, GPT-5.4-mini 78.6%. Lexical diversity still helps; secondary features already change.
Editing takes a different path. Versus the human source, lexical diversity rises only slightly (Cohen's d=+0.24) and entropy falls (d=-0.28). The generation pattern of both going up does not appear. The dominant edit effect is a collapse in lexical density (d=-3.10): content-word share drops, function words rise. That same density feature had permutation importance of only 0.056 for generation detection. GPT-oss-120B as an editor is again the odd one out, raising entropy (d=+0.44) and cutting Linsear Write hard (d=-2.81).
Pairwise AUC:
| Method | Human vs gen | Edit vs gen | Human vs edit |
| Stylometric LR | 0.97 | 0.98 | 0.80 |
| EditLens-Llama | 1.00 | 0.85 | 0.97 |
| Score average | 1.00 | 0.99 | 0.94 |
Stylometric features separate edited from generated text and struggle to separate edited from human text. EditLens does the reverse. At 5% false-positive rate, EditLens-Llama's true-positive rate for human vs edit is 91%. Edited text hugs the source, not the generation cloud. It is not a midpoint.
"AI text" is not one label. A detector trained on fully generated documents will systematically miss "human-written, model-edited" drafts if it relies on this stylometric footprint: those drafts hug the source, not the generation cloud. Classroom checkers, journal screens, and platform labels that train only on pure generation will under-flag polish, paraphrase, and expansion. Stylometric models and neural edit detectors see different signals; combining them covers both jobs. This is a measurement result, not a new leaderboard detector.
Fourteen features were chosen to stay interpretable and cheap. Semantics, argument structure, and factual hallucination are outside the set. The main study uses eight open models; editing uses only three 70B-class open models; proprietary generation is a smaller add-on of two models. The corpus is English long-form. Chat, multilingual, and highly technical writing are untested. Feature importance is associational: high entropy need not be "because it is AI," it can travel with length, domain, and decoding. Edit prompts come from the EditLens suite; real user edits may look nothing like them. GPT-oss-120B already splits from the other models on both generation and editing, so the footprint is starting to crack by model family.