AI Agents Replicate 168 ICML Papers, Only 7 Mostly Hold Up
ChenhaoTan · x · 2026-07-23
A developer utilized AI agents to review and replicate all 168 oral papers from ICML 2026, conducting 105 full replications.
The results reveal that only 7 papers mostly held up. This stark outcome has sparked discussions about a potential "replication crisis" in AI research: whether there is a systemic flaw in the reproducibility of academic papers, or if current AI agents simply lack the capability to effectively execute complex code.
Related event: AI Reproduces 168 ICML Papers, Only 7 Fully Pass(2 posts)→
More from coding & agent
- Serverless Framework v4.39.0 adds Lambda microVM sandboxes for agent code — DavidWells · 2026-07-23
- HeyGen’s HyperFrames adds a storyboard-first workflow for AI video generation — HeyGen · 2026-07-23
- W&B adds stronger agent tracing, harness integrations and automated evals in Weave — _ScottCondron · 2026-07-23
- Cisco's Antares AI Finds Vulnerabilities Cheaply Amid Access Deliberations — shashib · 2026-07-23
- A high-school builder’s “Taste Skill” for AI frontend design hits 67,000 stars — soumitrashukla9 · 2026-07-23
- Multica says community PR review is so costly it still has 400+ unmerged requests — jiayuan_jy · 2026-07-23