LLM-mined commit histories reveal ~24% of research decisions in 599 NeurIPS projects were abandoned
ChenhaoTan · x · 2026-10-15
A new study uses LLMs to read commit histories of 599 NeurIPS papers' code repos, grouping 13,271 commits into research decisions (method changes, new experiments, ablations). About 24% of decisions were later replaced or dropped — the kind of winding path papers never show. ChenhaoTan noted his team is working on related directions.
More from Research
- Priming Models: Teacher-Free One-Step Generation Hits FID 1.19 With 12x Less Compute — du_yilun · 2026-10-15
- vLLM Achieves Bitwise-Consistent RL Training, Fixing KL Blowups and Score Collapse — vllm_project · 2026-10-15
- David Deutsch: Proofs Are No Longer a Useful Proxy for Mathematical Progress — francoisfleuret · 2026-10-15
- STEMBench launches: 6 STEM domains, zero full-target successes across all runs — mariofilhoml · 2026-10-15
- OpenAI gives $5M to Simons Foundation's $15M push to help mathematicians navigate AI-era results — OpenAI · 2026-10-15
- Qwen releases BlenderWorld, a 223-task benchmark for 3D creation in Blender — yshan2u · 2026-10-15