Can von Neumann probes avoid value drift at relativistic scales?
louisvarge · x · 2026-09-14
Extending the question of whether 2050s colony ships must carry AI, this thread asks whether evolution can be prevented at relativistic scales:
- Every star system has an incentive to adapt probes to local circumstances
- But the sender doesn't want von Neumann probes to value-drift
- Adaptability and loyalty are in inherent tension — an interstellar control problem
The original poster argues interstellar MAD isn't silly sci-fi but what will determine future generations' freedoms, depending on how much violence a competitor could inflict.
More from AGI Musings
- Sam Altman on Where AI Is Headed — ResultBackground2450 · 2026-09-14
- LLMs rival the printing press in impact — but separate the doom loops from the real path — teodorio · 2026-09-14
- Screenwriter Jack Thorne urges law banning secret AI-generated scripts, says peers 'cheat' — nordicinst · 2026-09-14
- There's no truly AI-proof degree, say careers experts — adaptability beats picking a 'safe' major — nordicinst · 2026-09-14
- An Open Letter to Altman and Amodei: Safety Essays Aren't Safety Leadership — AryHHAry · 2026-09-14
- AI researcher lays out the safety paradox: pausing algorithms while compute piles up maximizes risk — RichmanRonald · 2026-09-14