Study: Claude Opus 4.8 beats human experts on every measure as peer reviewer of 124 Harvard papers
emollick · x · 2026-10-23
Ethan Mollick highlights a new paper by Harvard's Gary King and Steven Worthington: across 134 paired reviews of 124 Harvard social science papers, AI-generated reviews outranked unusually high-quality human expert reviews on every constructed measure.
- Pairwise rankings: AI reviews won 100% of head-to-head comparisons; per-dimension win rates ranged 92%–100% (sources 97%, figures & tables 95%, methods 85%)
- Individual ratings: AI scored higher on all seven review characteristics
- The authors argue journals could use AI reviews to speed publication, raise quality, and reduce adversarial interactions
Mollick: academia needs to adapt the practice of science to AI, fast.
Related event: Harvard study finds AI reviews outperform human peer reviewers(2 posts)→
More from Models
- Gemini 4 Argon spotted testing apps autonomously, and 'topping humor benchmarks' — andrew_n_carr · 2026-10-23
- Gemini 4 reportedly far less sensitive in refusals than other models — haider1 · 2026-10-23
- SemiAnalysis: Anthropic subscriptions offer 5x+ more value than OpenAI as $200 plan value halves — RexDouglass · 2026-10-23
- Google Cloud's Physical-Engineering Last Exam: all frontier models score under 50% — Afinetheorem · 2026-10-23
- LLM copyright guardrail absurdity: refuses to scrape a book but happily zips it for you — burkov · 2026-10-23
- 'Wait' markers are inherited from training data, not invented by RL, HKUST study finds — huawei-noah · 2026-10-23