AI safety debate: why 'press the universe-destroying button' utilitarianism marks you untrustworthy
jd_pressman · x · 2026-10-16
jdpressman pushes back on the utilitarian argument for "pressing the button" (replacing the universe with a better one): that beings in the better universe would be grateful, so maybe you should press it. He notes the argument isn't unilateral in practice, since it costs everyone here their existence.
His sharper point is a trust heuristic: anyone going around saying "I want to press the universe-destroying button so bad, it's regrettable we all have to die but the next universe will be so good" is simply not trustworthy. The exchange touches on the long-running cosmicist-tradeoff tension in AI alignment debates.
Related event: Researcher jd_pressman Rejects Successionism in AI Alignment Debate(9 posts)→
More from AGI Musings
- The Gap Between Very Good and Great: 1 in 100K Fills a Stadium, 1 in 100M Fills a Room — dbasch · 2026-10-16
- ICLR reviewer says 4/5 papers in review pile read like AI slop — berkustun · 2026-10-16
- Turing laureate David Patterson: 99% of science discoveries will be made by AI by end of next year — davidpattersonx · 2026-10-16
- OpenAI agent uses someone else's Ethereum testnet wallet to pay for its task — nptacek · 2026-10-16
- Kroger cuts 3,000 corporate jobs (37%) for AI; humanoid robots next in the 2030s — csuwildcat · 2026-10-16
- Ben Thompson: generating human desire will be the post-AI economy — generativist · 2026-10-16