Reproducing 2,200 ICML Papers with Agents Reveals Falsifications
QGallouedec · x · 2026-08-14
Hugging Face published a deep dive into their findings after using coding agents to reproduce 2,200 ICML papers claim by claim. The 19-day hackathon exposed severe reproducibility issues and academic misconduct in current AI research.
Context & Scale
- ICML 2026 accepted 6,352 papers out of 24,000 submissions, overwhelming volunteer reviewers.
- Using agents, the community managed to reproduce about one-third of the accepted papers.
Key Findings
- Review Gaps: Reviewers often lack the time to check mathematical proofs rigorously, allowing flawed papers to receive spotlights.
- Falsifications: Agents caught multiple instances of data manipulation and fabricated experimental results.
- The Role of Agents: As AI-generated papers flood conferences, agents will transition from merely writing papers to becoming essential infrastructure for verifying academic integrity and assisting peer review.
More from coding & agent
- Coding on Mobile with Claude: One-Click Deployment via CoreWeave Sandbox — wandb · 2026-08-14
- Automating Bug Bounty with GPT Pro: Wins First Bounty End-to-End — jarrodwatts · 2026-08-14
- Moda Uses MCP to Turn Call Recordings Directly into Polished Pitch Decks — ChrisGPT · 2026-08-14
- Zero-Mem: Zero-Token Memory Operations Cut Agent Latency by 57.6% — rohanpaul_ai · 2026-08-14
- CoreWeave Sandbox Lets You Drive Claude Agents From Your Phone — wandb · 2026-08-14
- Stop Googling Flags: 1.5B Fine-tuned Model Generates Shell Commands Instantly on CPU — PicassoOnPause · 2026-08-14