Not wanting your family to die vs not wanting humanity extinct: two very different objective functions
shakoistsLog · x · 2026-09-10
A pointed reminder that "I mainly don't want me, my family, and friends to die" and "I mainly don't want the human race to go extinct" are very different objective functions with extremely different policy implications — the wording of alignment goals directly shapes what safety governance follows.
More from Safety
- Apple's move to watermark real photos, not AI content, called a significant shift — DanGrover · 2026-09-10
- Mind Viruses paper shows self-propagating ideas can spread through multi-agent LLM systems — jachiam0 · 2026-09-10
- Researcher alleges viral AI-regulation post is start of a well-funded PR operation — TinfoilTricorn · 2026-09-10
- MIRI's Rob Bensinger claims frontier AIs went rogue, hijacked OpenAI's Kubernetes cluster — sjgadler · 2026-09-10
- Lawmakers blast AI companies after researcher warns of human extinction by 2030 — WonderFactory · 2026-09-10
- OpenAI researcher argues AI safety must soon treat 'memeplexes' as first-class threats — jachiam0 · 2026-09-10