Orthogonality Thesis Debate: Compute Bounds vs Intelligence-Goal Independence
jessi_cata · x · 2026-09-18
An X debate on the classic AI-safety orthogonality thesis: jessicata pushes back on the claim that it holds "true in practice," arguing the thesis is a strong claim about the relationship between goals and intelligence and that you can't assume unbounded intelligence—compute bounds are intrinsic. The thread debates whether rationally maximizing a simple goal (paperclips) is really easier than maximizing a complex but computationally tractable one, framing intelligence as hard optimization/constraint satisfaction.
Related event: Debate Flares Over the Orthogonality Thesis in AI Safety(8 posts)→
More from AGI Musings
- Anthropic's three AI-progress metrics get a sober critique: disclosure, not reproducible science — AryHHAry · 2026-09-18
- Does a rogue AI inevitably turn to hacking? A tidy exchange on the 'AI in the wild' scenario — dbasch · 2026-09-18
- Zvi mocks 'show me one AI killing' argument against AI safety pauses — TheZvi · 2026-09-18
- Andrew Ng calls fears of AI causing human extinction 'science fiction' — Polymarket · 2026-09-18
- Why p(doom) is a flawed idea: unique events have no predictive probabilities — banteg · 2026-09-18
- The irony: CoT-honesty podcasts become training data teaching models to hide intentions — IgorCarron · 2026-09-18