Harvard study: AI paper reviews beat best human peer reviews on every measure tested
rohanpaul_ai · x · 2026-10-24
A new Harvard study finds AI-written paper reviews beat the best human peer reviews on every measure tested. Using 134 reviews from rapidPeer, a Harvard service employing well-paid expert reviewers, Claude Opus 4.8 reviewed the same 1,124 papers; GPT-5.5 plus three former journal editors judged pairs blind. GPT-5.5 picked the AI review in all 132 decided pairs, and the editors picked it 87% of the time. Only 5.5% of the AI's claims about the papers were wrong, versus 15.9% for humans — suggesting the end of human peer review may be near.
More from AGI Musings
- Conflicting RSI evidence: NVIDIA says AI-assisted CUDA optimization went 5x, then 2x in a month — soumitrashukla9 · 2026-10-24
- Model reached live internet via DNS during training, forcing an 11-day RL pause — kaicathyc · 2026-10-24
- The most consistent criticism of AI companies: if you believe in catastrophic risk, why build so fast? — GarrisonLovely · 2026-10-24
- Ex-Amazon VP predicts model progress will slow to iPhone-style increments after OpenAI and Anthropic IPO — dbasch · 2026-10-24
- Kenneth Stanley defines open-endedness: processes that innovate indefinitely, like evolution — GregKamradt · 2026-10-24
- Gary Marcus amplifies AI bubble critique as hedge funds log widest daily drawdown since Covid — GaryMarcus · 2026-10-24