2026-08-24
Across seven datasets and 880k-plus texts, LLM polishing cuts writing-complexity variance 21-50% while keeping meaning, and shifts author-trait cues toward the models' priors.
Language carries identity, not only meaning. Age, gender, politics, personality, and moral stance leave traces in word choice and syntax. Diagnostics, personalization, hiring screens, and cultural research all mine those traces. Once LLMs sit in the writing loop, they prefer the statistically common continuation. The question is whether polishing keeps the message and strips the signature.
This Nature Human Behaviour paper ties observational time series to controlled rewrites. The journal PDF is paywalled; method detail below comes from the matching preprint (arXiv:2502.11266). The published abstract reports three studies, seven datasets, and more than 880,000 texts. The preprint splits the same material into four blocks: a shock analysis, a rewrite experiment, trait classification, and lexicon correlations. The write-up follows that finer split, and keeps only numbers that the abstract and preprint both support.
The observational slice uses three relatively unconstrained sources from about 2018 through 2023/2024: 318,490 Reddit r/WritingPrompts stories (replies under 200 words dropped), 379,583 Patch local-news articles, and 80,238 arXiv CS.CL / CS.CV abstracts. AI authorship is scored with Binoculars, a normalized-perplexity detector the authors cite at true-positive >0.95 and false-positive <0.01 on similar genres. Complexity is the monthly variance of five features (Simpson index, Shannon entropy, mean dependency length, type-token ratio, hapax), then averaged. ChatGPT's public launch (30 November 2022) is the shock. A discontinuous growth model estimates Time, ONSET, and POST; Granger tests ask whether lagged AI-usage rates predict later variance.
The rewrite slice draws 1,000 pre-GPT-3.5 documents each from Reddit and arXiv, then rewrites them with GPT-3.5, Llama 3 70B, and Gemini Pro under two neutral prompts (fix syntax, rephrase). Semantic drift is cosine similarity of text-embedding-ada-002 vectors, plus a four-rater human check on 20 Reddit pairs. Variances are compared with Levene's test.
The trait slice uses labeled authors: 710 congressional speakers (age, gender, party), 2,348 Pennebaker stream-of-consciousness essays (Big Five), 711 empathic-conversation essays (IRI), and 3,641 YourMorals Facebook users (moral foundations). Classifiers (SVM, logistic regression, random forest, gradient boosting, Longformer) train on original text with TF-IDF or the same embeddings, 5-fold times 40 seeds, keeping only runs that beat chance on the original with at least 20 successes, then score F1 on the rewrite. A Δ statistic tracks class imbalance; prediction flips show which way the model pushes authors. A top-down lexicon pass (LIWC and related dictionaries) checks whether textbook associations survive the rewrite.
The published abstract's headline effect: writing-complexity variance falls a statistically significant 21–50% across datasets and models (P ≤ 0.05). Core content stays; patterns tied to dominant attributes are amplified and others suppressed.
In the time series, variance drops after the ChatGPT shock in all three sources. On arXiv the POST term is β=−0.0014 (p<.001), on top of an already falling Time trend; Granger is significant at lags 5–8 months. Patch shows a large ONSET drop (β=−1.405, p<.001) but no Granger result, consistent with editorial process buffering a direct causal path. Reddit's POST is β=−0.0021 (p<.001), with Granger only at lag 14, which the authors read as slower consumer adoption.
In the rewrite experiment, 87% of cosine scores exceed 0.95; human raters average 2.97 / 3 (Gwet AC1=0.947). Most model-by-prompt cells shrink variance; the steepest Reddit cell goes from 0.0091 to 0.0045 (about 50%).
Trait classifiers lose about 6 absolute F1 points on average and remain above chance. Age is hit hardest: 0.351 to 0.260. Δ is larger on rewrites, so predictions bunch toward one class. Directionally, rewrites are judged as lower empathy (except personal distress), higher morality, older, more open, more agreeable, less extraverted, more Democratic, and more male. Lexicon links such as gender with negative-emotion words, extraversion with pronouns, loyalty with friend words, and age with future-focus wash out; neuroticism with negative emotion, purity with religion, and gender with social words remain.
| Trait | F1 original | F1 rewritten | Chance |
| Age group | 0.351 | 0.260 | 0.244 |
| Empathy | 0.657 | 0.603 | 0.541 |
| Personality | 0.658 | 0.602 | 0.514 |
| Gender | 0.694 | 0.623 | 0.495 |
| Morality | 0.640 | 0.578 | 0.521 |
| Party | 0.664 | 0.591 | 0.490 |
For anyone who still reads style as a proxy for a person (hiring screens, mental-health monitors, audience models), this is a measurement warning. LLM polish keeps the proposition and moves the signature, and the remaining signal is biased toward the model's prior: older, male, liberal, morally high, low-empathy. In hiring, candidates who sand their prose toward that prior can crowd out writers whose original style is more distinctive. The training-data loop is uglier: model-touched text re-enters the next pretraining mix, so variance can keep shrinking.
This does not prove that models will flatten every register. Granger already fails on Patch news, so institutional style guides still absorb some of the shock. They change the size of the effect, not its sign.
The Nature full text was not available; the journal write-up collapses the preprint's four blocks into three studies, and unpublished figure cells cannot be checked cell by cell. Observational AI labels come from Binoculars, and both the detector and some complexity features touch predictability; the authors split "level of usage versus variance of complexity," but collinearity remains a live worry. Multiple Granger lags raise Type I risk, which they note. Neutral prompts are not how people actually use writing assistants; persona-heavy prompts could shrink or grow the effect. Classifier drop is not the same as humans failing to read identity. Congressional speech, student essays, and survey users are not the world's writers. Army Research Laboratory and DARPA funding is acknowledged; the paper itself does not study military or intelligence use.