Why conditional predictions fail for unlikely policies: the case of a 2027 AI treaty ban
lxrjl · x · 2026-11-12
A debate snippet on evaluating AI policy: the author argues conditional predictions become unreliable the more unlikely the action being evaluated. If you ask whether an international treaty banning AI before end of 2027 would be good, the conditional prediction is 'completely fucked' — because the evidential update from discovering you live in a world with enough political will to pass such a ban within a year is massive enough to overturn your priors. Useful as a reasoning framework for AI governance discussions.
Related event: Conditional Forecasting Fails for Unlikely Events, Analyst Warns(2 posts)→
More from AGI Musings
- SF's math obsession: median Anthropic employee believes in 17% annual GDP growth — stevenstrogatz · 2026-11-13
- Paul Scharre launches CNAS AI Policy Lab: 'This is the moment' for AI policy — paul_scharre · 2026-11-13
- AI policy veteran Paul Scharre on Altman's "singularity" claim: this is crunch time — paul_scharre · 2026-11-13
- If the Platonic Representation Hypothesis Holds, Brain–ANN Alignment Research Is a Huge Opportunity — patrickmineault · 2026-11-13
- Chris Paxton: without Jensen Huang, NVIDIA will likely abandon open research and consumers — chris_j_paxton · 2026-11-13
- What should kids learn in the AI era: meta-learning and becoming a better self — yangyi · 2026-11-13