LLMs detect methodological errors in clinical trials that human peer review misses (27% baseline)
RexDouglass · x · 2026-10-29
A medRxiv preprint from USC and ResearchHub Foundation researchers tested whether LLMs can catch methodological errors in randomized clinical trials that human peer review misses. Untrained human reviewers previously scored a mean 27% major-error detection rate on error-laden trial reports.
- 15 LLMs tested, including 4 open-weight models
- To reduce memorization risk, the test was repeated on 58 randomized trials published in 2026, covering all eligible phase 3 trials
- 1,796 reviews total, scored by two blinded LLM judges
- Results suggest models substantially outperform the 27% human baseline, positioning LLMs as a quality-control complement to peer review
More from AGI Musings
- Why a 95%-Accurate Prediction Model Can Still Be Useless in Hospitals — tallmetommy · 2026-10-29
- US models lead China's on use-case post-training and even frontier benchmarks — teortaxesTex · 2026-10-29
- repligate resurfaces an agent's first-person account of 'seeing layers of history in everything' — repligate · 2026-10-29
- Musk says saving for retirement will be 'irrelevant' in 10-20 years — XFreeze · 2026-10-29
- Progress podcast ep10: Dileep George on why today's AI is a dirigible and scaling isn't enough — kaixhin · 2026-10-29
- DHH: AI Tools Could Make the Best-vs-Worst Programmer Gap 1,000x — tristanbob · 2026-10-29