The evaluation paradox: how do you prove AI improves where it's hard to evaluate?
davidmanheim · x · 2026-11-22
AI safety researcher davidmanheim highlights the self-referential paradox in eval debates: claiming AI capabilities improve even where they're hard to evaluate can't be backed by robust evaluations — because "hard to evaluate" is precisely what makes them impossible to demonstrate.
More from AGI Musings
- Ex-NSA's Perlroth: AI is multiplying cyber risk, exploits now beat patches by 7 days — chrisrohlf · 2026-11-22
- Szegedy predicts proofs without formal verification won't count within a year — aran_nayebi · 2026-11-22
- Two compute costs that decide AI takeoff: training the improver and paying for its exploration — 1a3orn · 2026-11-22
- Largest Lean proofs cost at most ~$50 in compute, per Claude's estimates — 1a3orn · 2026-11-22
- a16z: LLMs update predictions in mathematically precise ways — but AGI needs continual learning — vishalmisra · 2026-11-22
- "Swarms are anti-human": researcher escalates critique of multi-agent paradigm — shakoistsLog · 2026-11-22