Meta Researcher's Advice: Treat Published AI Results with a Hierarchy of Evidence
fredahshi · x · 2026-08-06
In response to the prevalent overclaims and fraud in academia, particularly within big labs, Meta researcher Yuntian Deng advises new students against treating published results as ground truth. He proposes a hierarchy of evidence for evaluating AI research.
From most to least credible, his tiers are:
- A public demo that accepts user-chosen inputs
- Code, checkpoints, and eval scripts
- Code without checkpoints
- No code
- "Code coming soon" that never arrives
More from AGI Musings
- Former OpenAI Advisor Miles Brundage Co-founds Agent Ecology Research Org — Miles_Brundage · 2026-08-06
- Ex-OpenAI Researcher: GDM Leadership Shifts Are Bullish as AGI Looks Overdetermined — jachiam0 · 2026-08-06
- Ex-DeepMind Researcher: Executive Exits Are Bets on the Next Tech Frontier — jachiam0 · 2026-08-06
- Ai4 2026 Panel: What It Takes for AI to Rigorously Reason About Physical Reality — AnneliesGamble · 2026-08-06
- AI Solves Decades-Old Math Problems Overnight, Ushering in Era of Proof Indigestion — TheTuringPost · 2026-08-06
- Countering AI Utopia: AI Won't Solve Power and Social Competition — JOBhakdi · 2026-08-06