Stranded credentials: how a skill-signaling market absorbed generative AI
Song Yao
econ.GN, cs.CY
2026-08-18
An audit of 445k Kaggle entries: medals signal only in year one. Upload stocks lost 82% of informativeness; format exit, not AI, explains half to three quarters.
Generative models can now draft the analyses, code, and essays that many credentials were built to certify. Signaling theory since Spence says a credential dies when producing its artifact stops being costly. Written proposals on freelance platforms and unproctored exam scores have already shown that kind of collapse.
Kaggle is a rare place where the same people keep submitting objectively scored work before and after ChatGPT. Two formats ran side by side. Upload contests score prediction files computed on published data. Code contests execute the entrant's program on hidden data. Neither format bans AI. They differ only in whether unvalidated work fails in public.
Song Yao at Washington University in St. Louis audited the public Meta Kaggle snapshot of 29 July 2026: 444,698 participations by 197,561 users in 214 medal-eligible upload and code contests (deadlines 2018–2025; 2022Q4 dropped as the transition quarter). The outcome is one minus a team's hidden-test percentile. Predictors are pre-contest medal counts, split by earning format and age band, entered as log(1+m), with contest fixed effects and two-way clustering by user and contest.
Two reading lenses matter. A badge-count read looks at one medal in isolation. A full-profile read conditions on the other format's medals and prior participation, closer to how a hiring manager compares two CVs. Changes in the upload stock are then split into age mix, slope-curve shift, interaction, and an aggregation residual, so aging can be separated from revaluation.
Medals are fresh, short-lived signals. Before ChatGPT, a medal younger than one year had a slope of 0.15–0.18 per log medal: one fresh medal versus none predicts a 10–12 percentile gain. Medals older than a year fall to 0.01–0.03. The two under-one-year bands alone deliver 99% of the within-contest R² of the full age set before AI, and 95% after.
| Stock | Pre-AI slope | AI-era slope | Change |
| Upload medals | 0.067 | 0.012 | −82% |
| Code medals | 0.065 | 0.087 | +0.022 |
Of the upload drop of 0.055, composition (the stock aging in place) is −0.028 and the pure curve shift is only −0.015. Aging explains 43–87% of the collapse, depending on how interaction and residual are assigned. Fresh upload medals lose about a quarter of their slope under the full-profile read; fresh code medals do not.
Old upload medals look more informative in the AI era when read alone (+0.062 for the 1–2 year band). The gain vanishes under full-profile controls: the stale medal is proxying for the rest of the holder's record, especially fresh code medals. Official lifetime tiers discard 13–16% of the information in the same medal events. A recency-weighted index fit only on pre-AI outcomes reaches within-R² 0.053 on AI-era contests, against 0.047 for the official tier. An AI-like working-style index built from iteration telemetry has a near-zero extra return in code versus upload contests (+0.006, P=0.28). Failed-run rates track contests, not entry cohorts.
Verified credentials did not become cheap talk when AI arrived. They still decay within a year, and they die when the issuing format exits. Lifetime badge walls overstate stale signals by construction. Kaggle's own tier rule was already leaving information on the table before ChatGPT.
Every estimate is associational. Survivorship is severe: 7.8% of unmedaled pre-AI users appear in the AI era, against 56.5% of those with six or more medals. The AI window is only 2023–2025. Kaggle is one platform, and actual AI use is never observed. The working-style null does not test the return to AI use itself.