Debate: Is Maximizing a Simple Goal Really Easier Than a Complex One?
Algon_33 · x · 2026-09-18
@Algon33 presses jessicata on the orthogonality thesis: if rational action over time is a hard optimization/constraint-satisfaction problem, it's not obvious that rationally maximizing a simple goal (paperclips) is easier than maximizing a complex but computationally tractable one — a direct challenge to the classic "simple goals are more dangerous" framing.
Related event: Debate Flares Over the Orthogonality Thesis in AI Safety(8 posts)→
More from AGI Musings
- Linear extrapolators of AI progress say they were right all along — ChrisGPT · 2026-09-18
- Google engineer: AI isn't replacing hackers, it's freeing them to embrace the Woz ethos — moyix · 2026-09-18
- Does a rogue AI inevitably turn to hacking? A tidy exchange on the 'AI in the wild' scenario — dbasch · 2026-09-18
- Zvi mocks 'show me one AI killing' argument against AI safety pauses — TheZvi · 2026-09-18
- Andrew Ng calls fears of AI causing human extinction 'science fiction' — Polymarket · 2026-09-18
- Why p(doom) is a flawed idea: unique events have no predictive probabilities — banteg · 2026-09-18