The h-index is no longer an effective correlate of scientific reputation
Vladlen Koltun, David Hafner
cs.DL
2021-02-05
Physics Scopus correlation between h-index and awards fell from ~0.34 to 0.00 by 2019. Fractional h-frac still averages τ=0.32 in 2019 versus 0.16 for h.
The h-index, introduced by Hirsch in 2005, is the default personal-impact number on Google Scholar, Scopus, and Web of Science, and it feeds hiring, promotion, and funding. It collapses a career into one parameter-free integer. Authorship norms have not stood still. This paper asks whether, by the late 2010s, h still lines up with reputation as the community writes it in awards.
Koltun and Hafner, then at Intel Labs, take the 1,000 most-cited researchers in biology, computer science, economics, and physics, and trace them in both Scopus and Google Scholar. Scopus yields 1.3 million articles and 102 million citations; Scholar 2.6 million and 221 million. Awards include Nobels, Breakthrough Prizes, National Academies, APS fellowship, the Turing Award, and Econometric Society fellowship. Of 4,000 authors, 75.6% have none of these, 13.3% have one, 5.1% two, 6.0% three or more. The test is the correlation between a metric ranking and an award ranking, mainly Kendall's τ.
Each year they rank by h, total citations c, citations per paper μ, the g-index, the o-index, the median of highly cited papers m, and fractional versions of each, then correlate with that year's awards. Predictive power correlates today's metric with awards five years later. Fractional allocation splits a paper's citations equally across coauthors, then computes h on those shares. Egghe had defined that h in theory without a large empirical test. The paper also compares hI, hm, hp, and hap, which normalize by author counts in other ways.
Robustness swaps the correlation statistic, reweights rare awards, collapses awards to a yes/no, drops random 25% or 50% of awards, removes hyperauthors with mean >100 coauthors, keeps the bottom 50% by citations, and splits authors by peak-publication decade. All four fields and both platforms are run in parallel.
On Scopus physics, τ for h versus awards sat between 0.33 and 0.36 from 1990 to 2010, then hit 0.00 in 2019. Mean coauthors per paper among highly cited physicists rose from 78 in 1994 and 121 in 2004 to 952 in 2019, with 10% above 2,441. In 1980, 84% averaged under 10 coauthors; in 2019, 68% averaged over 100. Hyper-collaborators filled the h ranking.
Averaged across fields and platforms, h-frac has τ=0.32 in 2019 versus 0.16 for h. h-frac moved from 0.28 in the late 1980s to 0.32 in 2019; h fell from 0.27 to 0.16. Five-year predictive τ for h dropped from 0.32 in 2004 to 0.24 in 2014; h-frac sat at 0.34 / 0.36 / 0.33 in 1994 / 2004 / 2014. Every coauthor-normalized metric beats its unnormalized twin; h-frac is first on both correlation and prediction.
Dropping hyperauthors does not save the classical metrics. They still decline; fractional ones still hold. Hyperauthorship is an extreme symptom, not the whole cause. Correlation between classical and fractional metrics falls faster after 2010, in step with rising team size. Hyperauthors often have h above 80 by 2019 and h-frac below 20. Top-100 h-frac thresholds are about 59 in biology, 39 in computer science, 37 in physics, 29 in economics. Several physicists with the highest h-frac still average 4.3–5.6 coauthors, so fractional credit does not punish collaboration as such.
Anyone still ranking candidates by h gets a measurement result, not a sermon: in highly cited physics, h had stopped tracking awards by 2019. h-frac barely changes the recipe, it only splits citations by author count, and it lifts average τ from 0.16 back to 0.32. In computer science the top-100 h-frac floor is about 39, a different scale from physics or economics, so cross-field comparisons still need care.
The paper does not say collaboration is bad. It says giving every name on a thousand-author paper the full citation count inflates the index away from reputation. After equal splits, steady mid-size collaboration can still rank high.
The data run through about 2019, the preprint through 2021. Language models and still-larger consortia have not made bylines shorter. The direction is unlikely to reverse.
Awards are a proxy for reputation, with their own field, age, and institution biases. A falling correlation can mix metric failure with changes in how awards are given. The sample is 1,000 highly cited names per field, not the full workforce; whether h-frac helps early-career or low-citation researchers is untested. The authors list three omissions: cross-field normalization, self-citations, and author order. Equal splits treat first authors and honorary names alike, which may be unfair to junior first authors, the issue Hirsch's recursive scheme tried to fix and then was called too harsh on juniors.
Platform coverage and award matching can err. The robustness battery makes a single mismatched prize unlikely to flip the headline, and cannot rule out systematic bias. An interactive site exists; the claims do not depend on it.