Revisiting 50 LLM findings: 44% expired within 15 months, only 6% strengthened
james_y_zou · x · 2026-11-03
A new study revisits 50 published LLM research findings to see how long AI knowledge holds up: within 15 months of publication, 44% of claims expired, 34% substantially weakened, only 16% persisted, and just 6% grew stronger — implying an extremely short half-life for empirical LLM results.
Related event: Stanford Retests 50 LLM Research Findings: 44% Fade Within 15 Months(2 posts)→
More from AGI Musings
- Mark Solms vs Nick Humphrey debate: can we make AIs sentient? — cccalum · 2026-11-03
- trackingsingularity.com Logs Key AI Capability and Safety Events on One Timeline — dejavucoder · 2026-11-03
- Cryptographer Matthew Green pushes back on LLM cryptanalysis fears: best AES-128 attacks still need 2^89 work — matthew_d_green · 2026-11-03
- 51 AI Insiders Vote on Who Best Understands Recursive Self-Improvement — eliebakouch · 2026-11-03
- Depression embryo screening tool MDD-PGT sparks moral backlash from AI researcher — jachiam0 · 2026-11-03
- repligate challenges LLM consciousness skeptics: is consciousness epiphenomenal? — repligate · 2026-11-03