Automated peer review at $0.10 per paper: dev says $500M could audit all 100M scientific papers
clarejtbirch · x · 2026-10-12
CJ Handmer shares numbers from his automated peer review prototype: current models deliver human-level peer review for roughly $0.10 of tokens per paper, and $5 buys a full rederivation of results, statistical checks, and bidirectional citation audits.
The scientific corpus holds about 100 million papers, most unimportant and many sloppy — and for structural reasons published work is essentially immutable, even for typo fixes. At these costs, roughly $500M today would be enough to thoroughly audit every paper ever written for p-hacking, plagiarism, fabrication, and citation clubs.
Since he lacks that budget, he builds a tree instead: targeting the 1% of papers carrying 99% of the signal with a cheap first pass, claiming to catch 90% of the nonsense for a few hundred dollars of tokens.
More from AGI Musings
- 'The most important chart for the AI economy': models commoditize, compute wins — FinanceYF5 · 2026-10-12
- AI Has Come For Call Centers: Job Losses Now Spread to Lower-Middle-Income Countries — FinanceYF5 · 2026-10-12
- Re-evaluating Steve Jobs: iPhone's net impact on society — granawkins · 2026-10-12
- From nuclear safety to AI: containing agents with signals they can't produce — WeldPond · 2026-10-12
- Personal Agents Nail First Impressions but Lack Daily Use Cases for Retention — sujingshen · 2026-10-12
- willcb: every non-RL algorithm is a proxy — policy gradients are the minimum-variance unbiased estimator — willcb · 2026-10-12