AI Agents Replicate 168 ICML Papers, Only 7 Mostly Hold Up
ChenhaoTan · x · 2026-07-23
A developer utilized AI agents to review and replicate all 168 oral papers from ICML 2026, conducting 105 full replications.
The results reveal that only 7 papers mostly held up. This stark outcome has sparked discussions about a potential "replication crisis" in AI research: whether there is a systemic flaw in the reproducibility of academic papers, or if current AI agents simply lack the capability to effectively execute complex code.
Related event: AI Agents Reproduce ICML Papers: Only 7 Out of 168 Fully Hold Up(5 posts)→
More from coding & agent
- Dev argues there's still alpha in working with your coding agent interactively — alex_frantic · 2026-10-27
- Researchers launch AMA on patterns for safer AI agents used by citizen developers — Loose_Rip359 · 2026-10-27
- Gradio Adds In-Chat Human-in-the-Loop So ML Intern Pauses Before Training — Gradio · 2026-10-27
- Building Halo Infinite Forge maps with local LLMs, DCjson and Blender MCP — swagonflyyyy · 2026-10-27
- Ex-Unity dev Lucas Meijer says OpenClaw looks cleaned up and worth another run — lucasmeijer · 2026-10-27
- Agent skills started contradicting each other after months — how to keep them clean? — Successful-Tax1306 · 2026-10-27