LLM reviewers always find flaws even in great work, Stanford prof says it's pretense of understanding
IanArawjo · x · 2026-09-27
Quoting Stanford professor Anshul Kundaje: brainstorming with frontier models in open-ended domains without verifiable loops (like bio) exposes their 'pretense of understanding.' Andrew Rosen adds another angle — ask an LLM to review a manuscript and it will never say 'accept as is'; it always produces a laundry list of superficially sensible but low-quality 'flaws.' Together the observations point to a systematic weakness in expert-level judgment.
More from AGI Musings
- Stochastic Strawmen: Can LLM Critics Hold a Consistent Argument for More Than a Year? — Zergylord · 2026-09-27
- Radiologist's blog helps young doctors pick residencies as AI reshapes medicine — DrDatta_AIIMS · 2026-09-27
- David Patterson pushes back on ASI power fears: humans have no reason to cede control — davidpattersonx · 2026-09-27
- Article argues AI agents will build, test, and ship software from plain-language specs — blaizedsouza · 2026-09-27
- Which books and shows actually explain our AI eschaton? A writer's shortlist — gleech · 2026-09-27
- Opinion: AI amplifies expertise, but can't replace the struggle of learning it — Abhishekcur · 2026-09-27