Economist: AI peer review lacks judgment but beats experts at reading everything
paulnovosad · x · 2026-11-09
Economist paulnovosad argues the weakest part of AI-assisted peer review is judgment, though AI far outperforms experts at actually reading all materials like appendices and analysis plans — which reviewers often skip anyway. Given a fixed time budget, optimal expert workflow should incorporate the tool, and appendices/analysis plans should feed into AI evaluation. The thread points toward better combined human + AI review processes.
More from AGI Musings
- AI safety talent surge debated: is METR's hiring boom really field-wide growth? — austinc3301 · 2026-11-09
- Gary Marcus: soaring costs of incremental AI gains point to a looming bubble burst — GaryMarcus · 2026-11-09
- Open source is easy to start but hard to polish: the real cost is mental endurance — yangyi · 2026-11-09
- AI era favors broad intermediate skills over maxed-out skill trees, argues dev — zack_overflow · 2026-11-09
- Hot take: speeding up individual AI researchers is a weak metric as big labs fight over compute — ZhongRuiqi · 2026-11-09
- Are faster AI researchers a good progress metric? One researcher argues no — ZhongRuiqi · 2026-11-09