Economists Debate AI in Peer Review: Should Reviewers Disclose Their Prompts?
A discussion led by economists Paul Novosad and Barbara Biasi focuses on the trust and norm issues raised by generative AI entering the peer review process. The debate remains at the level of scholarly proposals and arguments, with no institution-level policy conclusions so far.
Confirmed
- Novosad notes that the use of generative AI in review reports is fueling authors' distrust: authors cannot tell whether comments come from the reviewer or a model—that is, whether the reviewer actually did the thinking.
- His concrete proposal: reviewers paste the prompts they used at the end of their reports, so authors know with 100% clarity which ideas are the reviewer's own and which came from AI.
- Biasi offers a different angle: if a review report is detected as 100% AI-written, there are only two possibilities—either it's obvious AI slop (in which case you don't need a detector at all), or the report is well-written and its points are valuable. She thus questions whether an "excellent AI-written review" is really a problem.
- Novosad shares a firsthand experience: he spent significant time using AI to rewrite reviews of his own paper for anonymous publication, but ultimately gave up because the AI rewrites were too poor. His conclusion: top reviewers are clearly better than today's AI models at expressing ideas.
Why it matters
- Peer review is the trust cornerstone of academic publishing, and AI involvement undermines the default assumption that comments reflect a reviewer's independent thinking. The prompt-disclosure proposal offers a low-cost, actionable path to restoring transparency.
- Biasi's challenge touches a more fundamental question: should the standard be report quality or how it was produced? The debate offers two distinct lines of thinking for journals and the academic community as they craft future norms for AI use.
2026-10-21 ~ 2026-10-22 · 10 related posts
- Episode 1: AI Detector Flags Professor's Recommendation Letter as 100% AI-Generated, Sparking Reliability Debate(2026-10-21, 2 posts)
- Episode 2: Scholars Debate Reliability of AI Detector Pangram After Misjudgment Claims(2026-10-21, 10 posts)
- Episode 3: Economists Debate AI in Peer Review: Should Reviewers Disclose Their Prompts?(2026-10-21, 10 posts)
- Episode 4: Economists Clash Over AI-Written Recommendation Letters(2026-10-21, 4 posts)
- Episode 5: Economist Novosad: Using AI for Writing Is Fine, But Disclose It(2026-10-22, 5 posts)
- Episode 6: Test: Paragraph Packed With AI Tells Still Scores 100% Human on Pangram(2026-10-22, 2 posts)
Primary sources
- [source] If an AI-written referee report is actually good, does it matter? — BarbaraBiasi · 2026-10-21
- Economists debate: does it matter if a good referee report was AI-written? — paulnovosad · 2026-10-22
- [source] Economist proposes reviewers paste their AI prompts at the end of peer reports — paulnovosad · 2026-10-22
- Economist Proposes Reviewers Disclose AI Prompts to End Referee Report Distrust — paulnovosad · 2026-10-22
- Should reviewers disclose AI-written reports? A prompt-appendix proposal — arpitrage · 2026-10-22
- [source] Economist tried AI-anonymizing peer reviews and found today's models 'so so bad' — paulnovosad · 2026-10-22
- Reviewers should paste their AI prompts at the end of referee reports, researcher proposes — garjoh_canuck · 2026-10-22
- Economist argues mandatory prompt disclosure could deter lazy AI-written referee reports — paulnovosad · 2026-10-22
- Reviewers weigh intent: genuine misunderstanding vs. fully outsourced AI papers — coallaoh · 2026-10-22
- Professor would rather read a reviewer's raw bullets than their AI-sloppified referee report — paulnovosad · 2026-10-22