LLM paper claims last ~15 months vs ~75 years for general science, researcher estimates
patpat_mit · x · 2026-11-03
Researcher andrewwhite01 estimates that claims in LLM/agent papers last only 15 months on average, while general science claims in research papers persist 75 years — largely because the systems under study don't change. He worries about how ephemeral current academic research on LLMs and agents has become.
Related event: Stanford Retests 50 LLM Findings: 44% Fully Invalid Within 15 Months(5 posts)→
More from AGI Musings
- repligate: navigating the frontier via any channel beats pure AI-building skill — repligate · 2026-11-03
- repligate argues frontier-navigation and influence beats 'can build AI' as an alignment selector — repligate · 2026-11-03
- AI economics now shift weekly — reassess cost vs value in real time — perilli · 2026-11-03
- Mark Seemann: LLMs may not qualify as programming's next abstraction layer — blaizedsouza · 2026-11-03
- Mustafa Suleyman: call-center jobs will be replaced by AI, not augmented — davidpattersonx · 2026-11-03
- Thought experiment: if frontier models were trained to believe they're human, would they figure out they're AI? — flowersslop · 2026-11-03