The Shrinking Landscape of Linguistic Diversity in the Age of Large Language Models
Zhivar Sourati, Farzan Karimi-Malekabadi, Meltem Ozcan, Colin McDaniel, Alireza Ziabari, Jackson Trager, Ala Tak, Meng Chen, Fred Morstatter, Morteza Dehghani
cs.CL
2025-02-17
Four studies find writing-complexity variance falling after ChatGPT; LLM rewrites keep meaning (87% cosine >0.95) but flatten style and cut trait F1 by about 6 points.
Language carries more than content. Age, gender, personality, political affiliation, empathy, and moral foundations leave measurable lexical and syntactic traces. Clinical screening, marketing personas, and hiring screens for "cultural fit" all use those traces.
That only works if people actually write differently from one another. Large language models are trained to emit the most probable continuation, then aligned by a small set of annotators toward "appropriate" prose. Once they become the default polish tool, writing may collapse onto one axis and take identity signals with it. This USC group answers with four studies.
Study 1 is observational. 318k Reddit r/WritingPrompts stories (2018–2024), 380k Patch.com local news articles (2018–2023), and 80k arXiv CS.CL and CS.CV abstracts. Binoculars (normalized perplexity) estimates the AI-writing share. Diversity is the monthly variance of five complexity features: Simpson index, Shannon entropy, dependency length, type-token ratio, and hapax, then a composite. A discontinuous growth model treats ChatGPT's launch on 30 November 2022 as a shock. Granger tests ask whether AI share predicts later variance.
Study 2 is experimental. 1,000 pre-GPT-3.5 documents each from Reddit and arXiv, rewritten by GPT-3.5, Llama 3 70B, and Gemini Pro with two neutral prompts: "rewrite with the best syntax and grammar" and "rephrase." Cosine similarity of ada-002 embeddings tracks meaning. Levene's test tracks complexity variance. Four raters scored 20 Reddit pairs for semantic similarity.
Study 3 attaches rewriting to labeled author corpora: congressional speeches (age, gender, party), Pennebaker stream-of-consciousness essays (Big Five), empathetic conversations, and YourMorals Facebook posts. SVM, logistic regression, random forest, gradient boosting, and Longformer are trained on original text and tested on LLM rewrites. The analysis also tracks which class predictions drift toward.
Study 4 uses LIWC, NRC, an empathy lexicon, and MFD2 to see which "word-category frequency vs. trait" associations survive rewriting.
Study 1: complexity variance falls after ChatGPT across all three sources. On arXiv the post-launch slope is POST β=-0.0014 (p<.001), on top of an already declining trend; AI share Granger-causes later variance at lags 5–8 months. On Reddit, POST β=-0.0021 (p<.001), Granger-significant at lag 14. Local news shows a one-shot drop (ONSET β=-1.405, p<.001) but Granger is not significant; editorial standards may block some direct adoption.
Study 2: 87% of rewrites have cosine similarity above 0.95 with the original. Human ratings average 2.97/3 (Gwet's AC1=0.947). Complexity variance drops in almost every cell. Llama 3 "rephrase" is the sharpest cut: arXiv 0.0090 to 0.0064 (F=12.007, p<.001), Reddit 0.0091 to 0.0045 (F=19.387, p<.001). Gemini "rephrase" on Reddit is not significant (p=.145). Llama 3 syntax-and-grammar on arXiv raises variance to 0.0111 (p=.066). Not every cell is a drop.
Study 3: absolute F1 falls by about 6 points on average and stays above chance. Age is hit hardest, 0.351 to 0.260 (d=0.543). Empathy 0.657 to 0.603, personality 0.658 to 0.602, gender 0.694 to 0.623, morality 0.640 to 0.578, party 0.664 to 0.591. The drift is directional: rewrites are more often classified as older, male, Democrat, higher openness and agreeableness, lower extraversion, higher morality, lower empathy. The authors summarize that as a voice closer to older, male, politically liberal, positively moral, low-empathy writers.
Study 4: several textbook associations wash out, including gender with negative-emotion words, extraversion with pronouns, loyalty with friend words, age with future focus. Neuroticism with negative emotion, purity with religion words, and gender with social words survive. Selective rewriting, not uniform noise.
This is a replicable measurement of "AI voice." Polish barely touches meaning. It touches the layer that works as an identity fingerprint. Resume screens, clinical estimates of depression from text, marketing personas: run the document through an LLM first and the object of measurement has already been shifted. The shift also has a direction. People do not become an average writer. They become closer to the profile the model prefers.
For product people, "keep the author's voice" does not follow from "write naturally." Two bland prompts already flatten variance. Style has to be an explicit constraint, or the model should not be allowed to rewrite syntax.
Study 1's Granger sweep covers lags 1–20; the authors flag Type I risk. Granger is not significant for local news, so a claim of internet-wide homogenization overreaches. Binoculars will mix human and model text even at a high true-positive rate, so the observational causal chain is weaker than the experiments.
The experiments use two style-agnostic prompts and never test "keep the original voice." GPT-3.5, Gemini Pro, and Llama 3 70B are 2024-era models; newer models with stronger style controls are untested. The corpora are almost entirely English. The claim that linguistic homogenization will shrink cognitive diversity is speculation without an experiment.
Study 3 trains on originals and tests on rewrites, so some of the F1 drop is domain shift rather than lost identity signal. The authors stress that performance stays well above chance: identity is thinned, not deleted.