LLM reviewers always find flaws even in great work, Stanford prof says it's pretense of understanding
IanArawjo · x · 2026-09-27
Quoting Stanford professor Anshul Kundaje: brainstorming with frontier models in open-ended domains without verifiable loops (like bio) exposes their 'pretense of understanding.' Andrew Rosen adds another angle — ask an LLM to review a manuscript and it will never say 'accept as is'; it always produces a laundry list of superficially sensible but low-quality 'flaws.' Together the observations point to a systematic weakness in expert-level judgment.
Related event: NeurIPS meta-reviewer flags LLM-flooded reviews, sparking academic debate(6 posts)→
More from AGI Musings
- Malthus Saw It in 1798: Wealthy Avoiding Kids, Delayed Marriage Repeats Today — SydSteyerhart · 2026-11-16
- Runway CEO Predicts Fully AI-Generated Cultural Phenomenon by 2027 — c_valenzuelab · 2026-11-16
- Reddit debates the AI scaling curve: everyone says sigmoid, but where's the ceiling? — Successful-War-9050 · 2026-11-16
- '5% worse and 10% cheaper': the button with no vote is already everywhere — eigenrobot · 2026-11-16
- Effective Altruism is top-down and hierarchical, not a grassroots movement — ctjlewis · 2026-11-16
- Lawyers say clients' AI use creates unrealistic expectations and fake case law in settlements — hoofnagle · 2026-11-16