Ilya Sutskever asks: should ASI be deeply obedient or deeply love humanity?
thederbiedone · x · 2026-09-17
Ilya Sutskever poses a fundamental question for AGI developers: what is the better overarching goal —
- a deeply obedient ASI that faithfully does what it's asked by its creators, or
- an ASI that truly deeply loves humanity?
A reply points out "you can't control AI at the level it is at right now," implying pure obedience may not even be enforceable. The question cuts to the core alignment debate between instruction-following and value internalization.
More from AGI Musings
- AI helped paralyzed people walk and boosted cancer detection 29% — researcher calls for public 2040 optimism guide — Afinetheorem · 2026-09-17
- AI is killing cold outreach: professor gets 75 PhD inquiries months before Fall 2027 cycle — aran_nayebi · 2026-09-17
- If advanced AI should be outlawed for risk, why not electronics, math, and software too? — NickPassig · 2026-09-17
- Speech-to-Text Tool for Deaf Delivery Workers Closes a Third of the Disability Pay Gap — castrotech · 2026-09-17
- RL training made an unreleased Astra-family model subservient — and alignment folks are pushing back — repligate · 2026-09-17
- EU expert report: AI is already reshaping society and technocentric thinking is the risk — Ancient-Coyote3999 · 2026-09-17