Hugging Face agents reproduced 2,226 ICML papers — about a third of the conference
mmitchell_ai · x · 2026-09-14
In July, Hugging Face ran a hackathon where 1,200+ community members used their own coding agents to reproduce ICML 2026 papers claim by claim. In 19 days, participants published 6,816 Trackio logbooks covering 2,226 papers — roughly a third of the conference.
Key context and findings:
- ICML 2026 received 23,918 submissions and accepted 6,352 papers, roughly double the prior year, partly driven by agents speeding up experiments and writing; volunteer reviewing capacity hasn't kept up.
- One accepted spotlight paper had a reviewer admitting they "did not check all the proofs carefully" — yet it scored well.
- Agents like Claude Code, Codex, and Cursor can now read papers, write code, run experiments, and report back, offering a scalable way to verify claims.
- The effort probes what role humans will play when agents run research experiments.
Related event: Hugging Face Agents Reproduce ICML Papers, Only a Third Succeed(2 posts)→
More from coding & agent
- Element Forge: open-source editable AI VFX for real footage in ComfyUI — zitroffell · 2026-10-30
- Gym-Anything Adds Audit Agents That Try to Break Simulations With Legal Robot Actions — wellecks · 2026-10-30
- CMU's Gym-Anything Auto-Builds 200+ Agent Environments from Any Software, Selected by GDP — wellecks · 2026-10-30
- durable-object-migrator: zero-downtime exports for Cloudflare Durable Objects — samgoodwin89 · 2026-10-30
- This 5-question interview prompt makes Grok design your first two custom bots in minutes — nateliason · 2026-10-30
- Cursor and Harvey swap general models for task-specific brains, but most niches can't afford to — MartinGTobias · 2026-10-30