ICM slide says most AI capability claims lack controlled scientific evidence
AlexKontorovich · x · 2026-07-25
At ICM 2026, Alex Kontorovich posted a slide emphasizing that many claims about AI capability are not backed by controlled scientific conditions. The slide argues that public evidence is often affected by reporting bias and non-scientific incentives, and that one should separate the truth value of a conjecture from whether it is desirable.
The post is a concise critique of how AI capability claims are argued in public: there may be many data points, but most are not gathered under experimental conditions that would support strong scientific conclusions.
More from AGI Musings
- Conference slide warns AI optimization could pull mathematics’ goals apart — AlexKontorovich · 2026-07-25
- Goodhart’s law makes generative-AI metrics especially easy to game, slide says — AlexKontorovich · 2026-07-25
- Bronze Age Collapse Analogy Frames a Future Where Civilization Becomes External Compute — dylan522p · 2026-07-25
- Tao’s ICM 2026 hypothesis: AI may soon handle a meaningful share of research math — AlexKontorovich · 2026-07-25
- Researcher forms group to study frontier AI’s economic impacts — RishiBommasani · 2026-07-25
- Gary Marcus says AI adoption still looks shallow next to hype and data-center spend — GaryMarcus · 2026-07-25