Opus 4.8 beats human experts at peer review in study of 124 Harvard papers
Eliv_nurotic · reddit · 2026-10-23
A study from Gary King's group ("Peerless") tested AI peer review on 124 Harvard social science papers spanning statistical analyses, formal models, and qualitative work. Claude Opus 4.8 won 134/134 pairwise comparisons against human experts (humans won 87% of their pairings) and scored higher across all review areas — sources, weaknesses, methods, theory, framing, communication, and figures. The authors argue complaints about low-quality AI-generated reviews don't hold up to scrutiny, with direct implications for academic publishing.
Related event: Harvard Study Finds Claude Opus Peer Reviews Beat Top Human Experts(16 posts)→
More from AGI Musings
- genmon's singularity joke: step out for a week, come back to 1000-IQ posthuman friends — genmon · 2026-10-24
- Critic slams deliberately handicapping AI search as dumb as the clock-speed wall — JosephJacks_ · 2026-10-24
- Ross Taylor: studying LLM behavior will increasingly resemble social science — rosstaylor90 · 2026-10-24
- New Yorker cartoon: "pick a field that will still require humans when you graduate" — docmilanfar · 2026-10-24
- Most "is AI conscious?" debates boil down to two people talking past each other — granawkins · 2026-10-24
- 20VC: Naveen Rao says inference is a $10T market and robotics 100x bigger than coding agents — 20VC · 2026-10-24