Hugging Face agents reproduced 2,226 ICML papers — about a third of the conference
mmitchell_ai · x · 2026-09-14
In July, Hugging Face ran a hackathon where 1,200+ community members used their own coding agents to reproduce ICML 2026 papers claim by claim. In 19 days, participants published 6,816 Trackio logbooks covering 2,226 papers — roughly a third of the conference.
Key context and findings:
- ICML 2026 received 23,918 submissions and accepted 6,352 papers, roughly double the prior year, partly driven by agents speeding up experiments and writing; volunteer reviewing capacity hasn't kept up.
- One accepted spotlight paper had a reviewer admitting they "did not check all the proofs carefully" — yet it scored well.
- Agents like Claude Code, Codex, and Cursor can now read papers, write code, run experiments, and report back, offering a scalable way to verify claims.
- The effort probes what role humans will play when agents run research experiments.
More from coding & agent
- pi-vcc vs pi-blackhole: Sub-second Lossless Context Compaction or 3-LLM Recall? — mailto_devnull · 2026-09-14
- Homebrew 7.0.0 ships with native macOS app, sandboxing and vulnerability checks — MABUwaAI · 2026-09-14
- CS professor bans coding agents on assignments, students' reaction becomes a meme — CSProfKGD · 2026-09-14
- Teknium Spotlights Hermes Telephony Skill Turning Agents Into Phone Operators — Teknium · 2026-09-14
- Dev puts a 4,000-customer support inbox on autopilot with agents, stops reading code — natesiggard · 2026-09-14
- Every editor builds a three-skill AI agent pipeline for daily AI briefing — danshipper · 2026-09-14