Stanford Re-tests 50 Widely Cited LLM Findings: Most Expire or Fade Within 15 Months
ziv_ravid · x · 2026-11-03
Stanford's Owen Queen and James Zou re-tested 50 widely cited LLM behavior findings across 20 models from 8 developers. Within 15 months of publication, most claims no longer held or showed substantially diminished effects: 3 strengthened, 8 persisted, 17 weakened, and 22 expired. Classic example: GPT-3.5's 55% fabricated-citation rate has essentially vanished in modern models.
Related event: Stanford Retests 50 LLM Research Findings: 44% Fade Within 15 Months(2 posts)→
More from Research
- LLMs are getting good at directly writing neural network weights—what does that mean for gradient descent? — RichmanRonald · 2026-11-03
- UCSB awards IEE seed funding to three ECE projects on energy-efficient AI and quantum hardware — ninamiolane · 2026-11-03
- OpenAI deploys activation classifiers (probes) on frontier models starting from GPT-5.6 — bobabowen · 2026-11-03
- Google's ScientistTwo runs the full research loop autonomously, out-rating human papers — udmrzn · 2026-11-03
- ML4 tech report drops, detailing pretraining architecture and agentic performance interplay — Fluke_Ellington · 2026-11-03
- TuneJury: a model trained on human judgments predicts music preferences — mittu1204 · 2026-11-03