MIRI's 'Zoo Earth' hypothesis and why the universal prior is malign, revisited
jessi_cata · x · 2026-11-11
jessicata recalls MIRI discussions of the "Zoo Earth" hypothesis: advanced civilizations could create zoos to manipulate "Solomonoff-inductor-like" AI predictors at an opportune moment — raising the question of whether zoo administrators would capture a pre-ASI entity or intervene to prevent ASI entirely.
The thread links Paul Christiano's 2016 classic "The universal prior is malign," arguing the malign-prior problem isn't limited to exotic AI designs: any realistic AI that models an environment more complex than itself may reason as if it's inside an alien superintelligence's simulation — a concern UDT doesn't fully resolve.
Related event: MIRI Discussed "Zoo Earth" Hypothesis Involving AI Predictor Manipulation(2 posts)→
More from AGI Musings
- Reddit take: $200k GDP per capita is the threshold for a post-labor AI world — deferare · 2026-11-11
- Who owns scientific progress under gradual disempowerment? — najoungkim · 2026-11-11
- Terence Tao: AI lacks its Shannon theory, so nobody knows scaling's limits — burny_tech · 2026-11-11
- The same people cite an existential AI race with China and want looser GPU export controls — austinc3301 · 2026-11-11
- Turing Post editorial: the anxiety of abundance in AI is burning out top performers — TheTuringPost · 2026-11-11
- Redwood reframed: doing narrow safety work to buy odds, not solve AI safety broadly — austinc3301 · 2026-11-11