Economist Novosad on AI Peer Review: Cheap Laziness, Weak Judgment, and a Hybrid Workflow

On November 9, development economist Paul Novosad laid out a systematic thread on the risks of AI in academic peer review and a workable process. His core take: anonymous review is the cheapest place to slack off with AI, and while AI's judgment is weak, its ability to read all the material exceeds human experts—so a sensible human-AI division of labor is more realistic than blanket bans or laissez-faire.

Confirmed

Novosad noted reviews are anonymous; the worst case for a bad report is losing a bit of the editor's trust, and editors drowning in reports can't effectively police them, so the reputational cost is minimal. He worries lazy or dishonest academics will submit AI-generated reviews while claiming "I genuinely thought this through myself"—such plagiarized AI reviews are hard to formally prove but often obvious at a glance.
On capability, he believes AI reviewing is currently weakest at "judgment," for unclear reasons; but at "reading all the material" it far outpaces human experts—human reviewers skipping appendices and pre-analysis plans is precisely one of the current system's problems, which AI can fix.
For time-constrained cases, he proposed a four-step review process: the expert reads only the introduction to form an initial judgment; a strong AI model offers its top three criticisms; the expert discusses with the AI and cross-checks against the paper; finally the expert decides which criticisms have substantive merit and finalizes the review.
At the thread's end he suggested a constructive direction: build an AI system that reliably flags reviews with "no human increment"—screening out perfunctory reviews that show no clear added value over a directly AI-generated report—using AI to counter AI laziness.

Why it matters

Peer review decides what research survives; Novosad's analysis exposes a structural hole between anonymity and editors' oversight capacity: AI has slashed the cost of perfunctory reviews while the current system lacks the means to detect them.
His four-step method offers a pragmatic middle path—AI doesn't make the judgment call for humans, but its exhaustive reading and fault-finding are leveraged while final discretion stays with the expert, giving journals and conferences a reference framework for AI review policies.
The proposal to "use AI to identify reviews with no human increment," if realized, would directly answer the evidentiary difficulty of proving AI ghost-written reviews.

2026-11-09 ~ 2026-11-09 · 7 related posts

Primary sources

Economist proposes AI-assisted peer review: model reads appendices, expert judges critiques — paulnovosad · 2026-11-09 07:59 (UTC+08:00)
Economist: AI peer review lacks judgment but beats experts at reading everything — paulnovosad · 2026-11-09 08:04 (UTC+08:00)
Anonymous peer review has the lowest cost of AI shirking, economist argues — paulnovosad · 2026-11-09 08:20 (UTC+08:00)

Economist Paul Novosad proposes a 4-step AI-assisted peer review workflow — paulnovosad · 2026-11-09
[source] Economist proposes AI-assisted peer review: model reads appendices, expert judges critiques — paulnovosad · 2026-11-09
[source] Economist: AI peer review lacks judgment but beats experts at reading everything — paulnovosad · 2026-11-09
Scholars may pass off AI reviews as their own — obvious but hard to prove — paulnovosad · 2026-11-09
[source] Anonymous peer review has the lowest cost of AI shirking, economist argues — paulnovosad · 2026-11-09
Proposal: build AI that detects lazy AI reviews with no added human value — paulnovosad · 2026-11-09

1 near-duplicate retellings: paulnovosad