TMLR experiment shows spam paper authors don't understand their own work
anmarasovic · x · 2026-09-17
A discussion on peer review collapse: citing Nihar Shah's heroic TMLR experiment — he spent 20-25 hours over two weeks interviewing authors of seemingly low-quality submissions and confirmed that people submitting these papers have no idea what's going on in them. DynamicWebPaige notes the marginal cost to generate and submit a paper is essentially zero while human evaluation cost remains brutally high, half-seriously suggesting reviving viva voce as an anti-spam CAPTCHA. Peer review is a gift economy built on good-faith assumptions, and it can't survive when that faith is asymmetric.
Related event: TMLR Quizzes Authors Who Can't Explain Their Own Papers(14 posts)→
More from AGI Musings
- A kill switch is not a safety architecture: preparing for AI smarter than humans — williamtp · 2026-09-19
- Noam Brown clarifies clip: isolated agents can coordinate with just a few bits — jachiam0 · 2026-09-19
- Robotics AI can't copy the LLM playbook: 5% error rates kill the product — carlosdponx · 2026-09-19
- Anthropic Researcher Quits Over Extinction Risk; Newsletter Writer Pushes Back — alex_verem · 2026-09-19
- The Palantir Frame: Institutional Fitness in the Age of Accelerating AI — curious_vii · 2026-09-19
- Dario Amodei pitches a plan to 'Pace the Frontier' of AI — Nvidia's Huang pushes back — TechCrunch AI · 2026-09-19