Study: Opus 4.8 beats human experts as peer reviewer across 124 Harvard social science papers
Eliv_nurotic · reddit · 2026-10-23
Harvard's "Peerless" study (Gary King's group) tested Opus 4.8 as a peer reviewer on 124 Harvard social science papers spanning statistical analysis, formal models and qualitative work. In pairwise comparisons the AI judge won 134/124 matchups (100%), while human experts won 87% of theirs. AI also led every review area: Sources 97%, Figures & Tables 95%, Weaknesses 92%, Methods 85%, Framing 80%. The authors argue common complaints about low-quality AI-generated reviews don't hold up. Paper: gking.harvard.edu/peerless
Related event: Harvard Study Finds AI Peer Reviews Beat Human Experts Across the Board(6 posts)→
More from Models
- TypeSafe's Jev: a "Decision Model" LLM That Returns Floats at $0.042/M Input Tokens — ariG23498 · 2026-10-23
- Microsoft's MAI-Base-2 scaling ladder pegs the L42 run at ~1.8×10^23 FLOPs — teortaxesTex · 2026-10-23
- Local model agentic coding benchmark: 27B matches 125B, tool-calling is the lever — Joralk_ · 2026-10-23
- NVIDIA open-sources PersonaPlex-7B, a full-duplex voice AI model that listens and talks simultaneously — solyarisoftware · 2026-10-23
- NVIDIA's KumoRFM-2: a foundation model that predicts churn, fraud and demand straight from relational databases — solyarisoftware · 2026-10-23
- Gemini Analyzes Whether a Simple 'Hi' Is a Jailbreak Attempt in Its Thinking Steps — OldIntroduction2909 · 2026-10-23