Researcher Dan Faggella: negotiation won't save us from AGI — radical self-augmentation might
danfaggella · x · 2026-10-21
AI ethics researcher Dan Faggella argues in a long post that humanity harbors a negotiation fantasy about AGI: we intuitively know 'beggars can't be choosers,' yet we imagine striking deals that grant us perpetual benefits even as we become the beggars.
- Core claim: benefits flow to parties with causal purchase in the world; a handshake or signed contract is unlikely to secure us the planet, eternal happiness, or anything else once AGI 'fooms' beyond us.
- If we want any reasonable chance of survival — let alone retaining influence over the future — we must transform drastically: augment our minds and symbiotically merge in drastic ways.
- He concedes there are no guarantees; reality has promised none to any part of the living process. But radical transformation is, in his view, the only viable path to persist and imbue our values into what comes after us.
More from AGI Musings
- Extropic founder Verdon teases 'most candid' interview on physics, e/acc and Kardashev research — beffjezos · 2026-10-21
- Pathway CEO in Forbes: Transformers Won't Get Us to Superintelligence, Experience-Learning AI Will — schwentker · 2026-10-21
- 'Software is dead' every decade since 1994, yet demand for engineers keeps hitting new highs — dotey · 2026-10-21
- Prof's advice: outline your book now, then prompt AI into writing it months later — prof_g · 2026-10-21
- Ex-OpenAI researcher's Core Automation lets agents run real online businesses, hiring 'AGM' interns — _arohan_ · 2026-10-21
- Vercel's rauchg: AI will compute the perfect delta of your understanding, delivering free ultimate education — andreisavu · 2026-10-21