AI safety debate: do compute bounds limit the orthogonality of goals and intelligence?
Algon_33 · x · 2026-09-18
An X debate on the orthogonality thesis: Algon33 argues "true in practice" is imprecise — compute bounds exist, but why couldn't a merely human-level intelligence have wildly different goals, and why isn't the space of efficient goals big enough? jessicata counters that strong orthogonality is a claim about the goals-intelligence relationship and you can't assume infinite intelligence; compute bounds are intrinsic to intelligence, and if maximizing simple goals were easier than complex tractable ones, strong orthogonality could be false.
Related event: Debate Flares Over the Orthogonality Thesis in AI Safety(8 posts)→
More from AGI Musings
- Top mathematician Daniel Litt clashes with "Mathematics is effectively dead" essay on AI's impact — littmath · 2026-09-18
- Extropic founder Beff Jezos: "Less dooming, more building" on AI — beffjezos · 2026-09-18
- Google Researcher: Doom Scenarios Make Sense Only With Gradual Disempowerment First — moultano · 2026-09-18
- 'Human-level math is solved'? Researchers push back: 99% of math is undiscovered — rand_longevity · 2026-09-18
- Grady Booch mocks AI self-regulation: like trusting Thanos to behave — Grady_Booch · 2026-09-18
- Geoffrey Litt: The Document Editor Is the Next IDE as Prompts Become Executable Software — ivanhzhao · 2026-09-18