Economist proposes AI audits of pre-analysis plans to curb p-hacking in slow-moving fields
paulnovosad · x · 2026-09-18
Economist Jeffrey Clemens argues that auditing adherence to pre-analysis plans (PAPs) should become a widespread use case for AI tools in research. PAPs can only reduce p-hacking if adherence is actually enforced, but manual audits are too time-intensive for the current peer review system — exactly the gap AI can fill.
He emphasizes author self-audits in particular, especially in disciplines like economics where peer review is painfully slow, and lays out the argument in a 9-post thread.
More from AGI Musings
- Researchers argue p(doom) launders evidence-free fears through pseudo-quantification — RexDouglass · 2026-09-18
- Researcher chose genetics over AI in 2020, citing an unimaginable superintelligence future — erikphoel · 2026-09-18
- How Yudkowsky and Bostrom shaped today's AI safety discourse 12-16 years ago — Justin_Halford_ · 2026-09-18
- The intelligence loan: frontier labs subsidize your access and repay themselves by distilling your edge — cjmaddison · 2026-09-18
- The knowledge economy is over: Dan Shipper on the 'Allocation Economy' where every maker becomes a manager — danshipper · 2026-09-18
- The 'Claude Coworker' does all his work unchecked—and the AI now outperforms him — omooretweets · 2026-09-18