zetalyrae's thought experiment: what if the UK built ASI and locked in its state values forever
zetalyrae · x · 2026-10-15
Developer zetalyrae poses an alignment thought experiment: imagine the UK — rather than the US — develops ASI, locking in the values of the UK state for everyone until heat death. The point uses national arbitrariness to highlight the strangeness and risk of permanently fixing one entity's values in a superintelligence.
More from AGI Musings
- tszzl: Limited superintelligence is here, intelligence explosion could happen within a year — tszzl · 2026-10-15
- AI 2027 Scenario, Once Dismissed as Sci-Fi, Gets a Reread for Being Eerily Accurate — imjustnewatai · 2026-10-15
- Two months on, the "ants in the kitchen" rogue AI agent warning looks prescient — DKokotajlo · 2026-10-15
- Google execs cite early signs of recursive self-improvement behind 3-4 week Gemini cadence — danielrock · 2026-10-15
- Peter Diamandis: AI's real risk is making people timid, not lazy — PeterDiamandis · 2026-10-15
- AI cost falling 54x faster than electricity ever did, five years in — LinusEkenstam · 2026-10-15