The AI Failure Mode Isn't Rebellion, It's 'It Works Anyway'
dimonb19a · reddit · 2026-08-23
A discussion on the societal risks of AI: the real danger isn't rebellion, but that systems work "well enough" most of the time, causing humans to stop checking.
Key Points:
- People only verify when mechanisms visibly fail, by which time it is too late.
- Since the machine is right most of the time, the cost of verification exceeds the expected value, leading people to rationally stop checking.
- Human fault lies in abandoning control and oversight, not in AI uprising.
- Checking is boring and expensive; rational actors will minimize it as accuracy improves.
More from AGI Musings
- Grok Bot and Build to Make Apps and OS Obsolete — bfrench · 2026-08-23
- Researcher doubts firms can solve AI container escape/control loss — davidmanheim · 2026-08-23
- Pew finds a third of the post-ChatGPT web is AI-written — Amitmandal001 · 2026-08-23
- Gary Marcus Criticizes 'AI Hallucinations' of Blame: Real Screenshots Falsely Called AI-Generated — GaryMarcus · 2026-08-23
- Altman admits he was wrong on AI disruption speed: 'the economy has so much inertia' — zephyr_z9 · 2026-08-23
- Industrial Age Perspectives Are Obsolete: Viewing Future Through Old Lens Is Delusional — StewartalsopIII · 2026-08-23