Generalization and the Rise of System-level Creativity in Science
Hongbo Fang, James Evans
cs.SI, cs.DL
2025-09-23
Using tens of millions of papers, Fang and Evans find generalization rose 75% after 1991; 74% of so-called disruptive citations are cross-field modular reuse.
The loudest scientometric claim of the past few years is Park et al. 2023 in Nature: disruption scores of papers and patents have been falling, so science looks less "disruptive." Disruption asks only whether a citing paper also cites the focal paper's references. Cited alone, it counts as disruptive. That definition mixes two jobs: founding a research trajectory, and being imported as a tool from somewhere else.
Hongbo Fang and James Evans at the University of Chicago and the Santa Fe Institute invert the question. They stop asking what a paper cites, and ask how later papers use it. On tens of millions of OpenAlex and Web of Science records, they split downstream citations into three functional types.
For each citation a focal paper receives, count two co-citation tallies: how many of the focal paper's own references the new paper also cites, and how many of the focal paper's other citers it also cites.
A paper's F, E, and G indices are the shares of its citations in each class. The main analyses use a five-year citation window; the longitudinal charts use one year. Inclusion is at least one recorded reference and at least five citations in five years. That yields 23.45 million OpenAlex papers (1945-2019) and a 19.03 million Web of Science replication.
The running example is Attention Is All You Need. Its citation of LSTM is foundational: the introduction discusses LSTM with a cluster of follow-on work to position the Transformer. Its citation of Layer Normalization is generalizational: a building block mentioned in the encoder-decoder subsection, with almost no surrounding literature.
They check the labels three ways: embeddings of co-occurring venues and title tokens, citation placement in about 3.2 million Semantic Scholar full texts, and citation-purpose annotations from ACT2 and Jurgens et al.
Start with disruption. G correlates with disruption at r = 0.37; F is nearly unrelated (r = 0.05, U-shaped); E is negatively correlated at r = -0.37. At the citation level, 73.96% of "disruptive" citations (cited without the focal paper's references) are generalizational (cited without the references or the later papers that also cite it).
| Metric | Value | Baseline |
| Disruptive citations that are generalizational | 73.96% | original disruption definition |
| G vs disruption | r = 0.37 | F r = 0.05, E r = -0.37 |
| Mentions of a generalizing citation | 1.31 | foundational 1.59, extensional 1.82 |
| Mean G, 1991 to 2024 | 0.218 to 0.382 (+75.2%) | mean E 0.757 to 0.596 |
| G after a journal goes online | +0.043 SD | 95% CI 0.004-0.081 |
| Generalizing references after inferred LLM use | arXiv +0.050 SD | bioRxiv +0.117, SSRN +0.081 |
History splits in two. From the late 1940s to around 1990, disciplinary walls were going up: F fell from 0.059 to 0.025, E rose from 0.536 to 0.762, G fell from 0.405 to 0.214. After 1991, the year the World Wide Web launched, the signs flip: G rose 75.2% (0.218 to 0.382 in 2024), F fell another 12.0%, E fell 21.3%. Degree-preserving citation shuffles that hold fixed annual publication volume, citation counts, and venue or domain mix produce a shallower post-2010 G rise and a turning point about 10-15 years late.
What scientists write changed in parallel. Titles with review, dataset, software, and tool became more common; theory and hypothesis became less so. Within most keyword-defined categories, the tracks look like the randomized networks. The long-run shift is mostly a reallocation of output types.
Generalizing citations are the most likely to be labeled "uses" and show up more in Methods (12.1%); foundational citations land 45.7% of the time in the Introduction. After controlling for citation count, year, and domain, top-decile G papers are cited from more OpenAlex topics, with a smaller share from their single largest topic, and they spread farther across countries. Relative to the bottom group, high-G work sits 84.0% farther in reference space, 15.5% farther in semantic space, and is 114.0% more likely to draw a cross-domain citation. Nobel and major-award papers sit on the foundational side; generalization is negatively associated with those honors.
Teams of one or two authors produce more highly generalizing work; teams of about four have the highest foundation index. The United States and the United Kingdom lean foundational, France and Germany extensional, China and India generalizing since the early 2000s. The famous "small teams disrupt" result (Wu et al.) and the "aging scientists innovate less" result (Cui et al.) mostly move with G, not with the capacity to found a field.
This is a measurement defense of people who ship models, tools, datasets, and reviews. Being used as a brick, and rarely occupying the center of the citing paper, is still a scientific function. Award systems are calibrated to within-field landmarks, so generalizing work is systematically under-counted.
For readers of the "science is getting less disruptive" story, the alternative reading is that the locus of innovation moved from inside fields to between them. The web, search, preprints, and now large language models cheapened the cost of carrying a module across a boundary. Stacked difference-in-differences on journal online access and on inferred LLM use both point that way. The authors treat both as associations, not causal identification.
F, E, and G factorize disruption. They are not a new predictor of quality, and they should not replace peer review. They do explain why a review, a dataset, or a piece of software can look "disruptive" in a citation graph and still be missing from prize lists.
The authors list four. The three bins are coarse: a generalization cited in Methods is not the same contribution as one mentioned in Discussion. Citation coverage varies by year, field, and database, so cross-year and cross-domain comparisons need care. Online-access and LLM analyses have non-random treatment; LLM use is inferred from a text detector, not observed. The study covers scientific papers only, not patents or cultural production.
Two further gaps sit in the design. Dating the break at 1991 because that is when the Web launched is a temporal alignment, not an identification strategy. Funding shifts toward interdisciplinary work, publication inflation, and venue mix moved at the same time; the null models absorb some of that, not the policy channel. The main sample also drops papers with fewer than five citations in five years. Trends survive a zero-citation threshold, but the portrait of "scientific contribution" still excludes work that was never reused.