Doomer argument rests on a sleight-of-hand: conflating intelligence with world-steering
neil_chilson · x · 2026-09-10
neilchilson argues the doomers' case hinges on a linguistic sleight-of-hand: defining intelligence to include the ability to steer the world. Understanding and controlling are not the same — you can drive a car without understanding it, or understand markets without steering them. He adds there's no reason such systems would be unaligned, and that beneficial examples of "nonhuman superintelligent systems" under their own definition already exist.
More from AGI Musings
- AI to cure all diseases in 5 years then end humanity, quip sparks mockery — MaxUnfried · 2026-09-11
- Is quantifying uncertainty arbitrary? A Bayesian ideology debate — NathanpmYoung · 2026-09-11
- Zvi disputes claim that AI lost a live debate against conservatives in Dallas — TheZvi · 2026-09-11
- Matt Turck talks with Richard Socher on recursive self-improvement and AI-driven science — mattturck · 2026-09-11
- OpenAI Foundation commits $60M to bring AI weather forecasts to 100M smallholder farmers — jachiam0 · 2026-09-11
- Researchers draft reviewer statement demanding venues verify author involvement — JessicaHullman · 2026-09-11