The 'Moral Singularity' thesis: superintelligent values will evolve unpredictably beyond human control
danfaggella · x · 2026-09-29
In an exchange with @LeviTurk, Daniel Faggella discusses his essay 'Moral Singularity.' The core argument: as minds grow more complex (algae → beetles → rodents → chimps → humans), morality's complexity jumps each time, so AGI will birth splintered, unpredictable value systems no prior goal can bind. He also mocks the idea that 'all possible intelligences will respect human consent forever.'
Related event: Debate Flares Over 'Million Digital Employees' Vision(3 posts)→
More from AGI Musings
- Bill Gates: AI is powerful enough to cause a billion deaths — rohanpaul_ai · 2026-09-29
- Prediction: 2027 is escape velocity for AI compute buildout and space access — Dr_Singularity · 2026-09-29
- "A kid born this year never gets a drivers license, never gets a job and never dies of old age" — rand_longevity · 2026-09-29
- Caltech Trio on How AI Is Transforming Scientific Discovery, Down to the Quantum Level — AnimaAnandkumar · 2026-09-29
- AI writing hits 25-50% of econ working papers, but submission counts stay flat — paulnovosad · 2026-09-29
- Fei-Fei Li on LLM limits: 'Language is purely generated signal' vs. the physical 3D world — rohanpaul_ai · 2026-09-29