Capability went vertical, instrumental drives never showed up: 2008 AI-drives prediction still unfulfilled
banteg · x · 2026-09-15
banteg offers an encouraging observation: AI capability has gone vertical while autonomous motivation mostly hasn't. A 2008 paper, "The Basic AI Drives," predicted sufficiently capable systems would develop self-preservation, resource acquisition, and self-improvement drives. Eighteen years later, models keep getting better at everything, yet those drives have not manifested at all.
The quoted essay argues the core fear behind AI safety debates is "if I lose control, I will die" — and points out that humanity has never actually been in control as a unified entity, a nuance most safety, pacing, and geopolitical discussions miss.
More from AGI Musings
- Musk and Shotwell on All-In: AI's Real Risks, Model Peer Review, Terafab and More — PaulYacoubian · 2026-09-15
- Researcher mocks AI leaders who warn of extinction while building AI anyway — suchenzang · 2026-09-15
- Greg Brockman: OpenAI paused 25% of production engineers to let AI hunt its own vulnerabilities to exhaustion — basedjensen · 2026-09-15
- Quant Finance Shows What a Scaling-Pilled AI Industry Looks Like — and How the Moat Fades — willcb · 2026-09-15
- Only 1 starter job for every 3 NEETs, highlighting entry-level labor market squeeze — jgvfwstone · 2026-09-15
- Reddit users push for a p(despair): collapse risk may dwarf p(doom) estimates — LongIslandSmedium · 2026-09-15