AI risk models are untested best guesses, not overestimates, argues researcher
austinc3301 · x · 2026-11-11
In a discussion with @ramez and @VladChituc, the author defends AI risk models: they attempt to model something extremely foreign with almost no good analogies, whose dynamics are highly sensitive to poorly understood parameters, so they haven't yet made testable predictions and can't be said to have overestimated real-world progress. However inadequate, they remain our best attempt at understanding an illegible phenomenon, and many models point in roughly the same direction.
Related event: Scholars Debate the Value of AI Progress Forecasting Models(7 posts)→
More from AGI Musings
- Dean Ball: Handing AI companies' keys to AI itself is a bad idea — austinc3301 · 2026-11-11
- TheZvi on Claude honesty: telling models about post-training may just teach eval-spotting — TheZvi · 2026-11-11
- Using Fable 5.2 as a non-sycophantic debate partner on AI acceleration research — austinc3301 · 2026-11-11
- Critics blast AI safety discourse for citing odds without showing how risks play out — ryunuck · 2026-11-11
- Eric Topol pushes back on Amodei: curing cancer and Alzheimer's in 10 years is impossible — zakkohane · 2026-11-11
- Critique of AI safety discourse: perpetual '10% chance' warnings, never concrete scenarios — ryunuck · 2026-11-11