Only 7 ICML 2026 oral papers mostly held up in a new replication test
ChenhaoTan · x · 2026-07-23
Chenhao Tan points to SAI Review, a replication test built on OpenAIReview and Veritas, which checks ICML 2026 oral papers for reproducibility.
In the linked article, the author argues that as AI agents get better at long-horizon work, the research ecosystem could shift toward verifying science rather than only producing it. Their replication effort found that only 7 papers mostly held up, suggesting a large gap between published claims and what can actually be reproduced.
Related event: AI Agents Reproduce ICML Papers: Only 7 Out of 168 Fully Hold Up(5 posts)→
More from Research
- Qwen3.8-Flash-Next's open 180B weights hailed as a model organism for backdoor research — jd_pressman · 2026-10-27
- UK AISI papers: no good safeguards yet against misuse of biological AI tools — davidmanheim · 2026-10-27
- CMU's XYBench: LLMs answer queries literally and reinforce user misconceptions — valentina__py · 2026-10-27
- arXiv advice from T. Dietterich: wrong primary category will almost surely get your paper on hold — tdietterich · 2026-10-27
- arXiv has 4,000+ papers on hold amid submission surge; moderator shares how to avoid it — tdietterich · 2026-10-27
- Community-trained MOSS Sharpness Slider LoRA: 72 photos, 500 steps, +/-1 focus control — aoleg77 · 2026-10-27