AI Audit Finds 2,035 Errors in 138 Papers by Economists Saez and Zucman
sebkrier · x · 2026-10-29
Tyler Cowen reports on an AI audit of economists Emmanuel Saez and Gabriel Zucman: 138 papers yielded 2,035 problems, 147 serious enough to change a headline number; 69% of papers had at least one issue, and 97% of directional major/moderate errors overstated the paper's claim.
Method:
- Dual blind audits by Claude Opus 5.5 and GPT-6 Astra: each recomputed headline numbers from the authors' own files and original sources (tax statistics, national accounts, Fed data), verified citations, and probed assumptions
- A third adjudicator agent re-verified every claimed problem, tried to refute each, kept only surviving findings, and graded severity
Cowen calls for a radical revaluation of this body of work and frames it as an early step in AI revising economic science.
More from AGI Musings
- Why the "human in the loop" can't be removed: a Thomist metaphysics take on AI transformation — curious_vii · 2026-10-29
- AI exploit agent's API bill revealed: ~$40K per attack chain, slides two months apart — moyix · 2026-10-29
- What happens when AI can identify closeted people from public data? — PierceLilholt · 2026-10-29
- AI Social Scientists in Secure Enclaves Proposed to Reconcile Privacy and Discovery — RobbWiller · 2026-10-29
- Designers rue how empathy maps and personas gave way to one-shotting UIs with AI — round · 2026-10-29
- Rereading Xerox PARC's 1982 Star UI paper for the generative UI era — round · 2026-10-29