Physicist audits 158,000 papers with Claude, finds ~1,000 major uncorrected errors
IgorCarron · x · 2026-11-17
Casey Handmer (Caltech PhD) used Opus to machine-audit the entire field of theoretical astrophysics: 158,059 papers and 554,670 citations indexed, 50,000 audited, thousands more deeply checked, and 659 key papers recomputed.
Key findings:
- Nearly 1,000 major errors cited many times, mostly still uncorrected — including mistakes in his own papers
- The subfield's foundational 1997 paper had known sign errors; fixes lived in 10,000 lines of contradictory email threads and were never published, so every student pays the same tax
- Peer review is ineffective at correcting errors; nobody is paid to fix originals
- The survey yields dozens of open questions (with .md context files) anyone with spare Claude Max tokens could tackle today
Bottom line: AI can speed up quantitative science by >1000x with no further model improvement needed.
Related event: AI Audit of Astrophysics Papers Finds Nearly 1,000 Repeated Citation Errors(2 posts)→
More from AGI Musings
- Five surveys, one verdict: negative views of AI outweigh positive across the US — connoraxiotes · 2026-11-17
- Oxford philosopher Carissa Véliz's Prophecy makes FT Best Books of 2026, challenging AI prediction myths — CarissaVeliz · 2026-11-17
- Dwarkesh argues math's avalanche is coming for AI research, and a viral summary distills his three-point case — Dan_Jeffries1 · 2026-11-17
- Insiders' AI timelines keep shrinking; capabilities now projected in 6-12 months — mallow610 · 2026-11-17
- Blogger argues total compute controls create blind spots and accelerate adversaries — TinfoilTricorn · 2026-11-17
- Ellul: the thinker who answered Nietzsche head-on, and Ted K's selective reading — StewartalsopIII · 2026-11-17