Data leakage hit 648 papers across 30 fields; autoarxiv now auto-reproduces arXiv papers
pandeyparul · x · 2026-10-30
A 2023 study by Sayashk and Arvind Narayanan found data leakage across 294 ML papers in 17 fields; the continuously updated list (image) now spans 41 methodology papers across 30 fields, affecting 648 papers — common pitfalls include missing train-test splits, leakage-prone preprocessing, and temporal leakage.
alphaxiv's new autoresearch tackles verification at scale: change "arxiv" to "autoarxiv" in any paper URL and an agent sets up an environment on your compute, runs a minimal reproduction, and tracks logs, visualizations, and artifacts in a fully local closed loop.
More from coding & agent
- OpenClaw's Trail of Bits audit confirms 23 vulnerabilities, all fixed — steipete · 2026-10-30
- Supabase hits 13M users as ~70% of new activity comes from AI agents — ycombinator · 2026-10-30
- Zero evals is fine at first: nail UX, then build evals with domain experts — Al_Grigor · 2026-10-30
- DeepMind essay: agent coordination, not solo competence, will flip the economy — krishnan · 2026-10-30
- Tessl framework: right skill lifts agent accuracy 20%, but activation drops to 40% — bibryam · 2026-10-30
- 7 free open-source tools that replace paid apps, from local voice cloning to spyware checks — dr_cintas · 2026-10-30