AI safety community splits over whether MIRI once aimed to build FAI and 'take over the world'

On Aug 15-16, the AI safety community erupted in dispute over MIRI's historical mission. JD Pressman accused MIRI-affiliated figures of gaslighting by denying MIRI once planned to build FAI, while Habryka insisted MIRI never tried to 'take over the world,' citing older writings. The fight extended to whether CEV equals seizing power and PCEV's ethical flaws, ending without consensus.

Confirmed

• The core dispute is whether MIRI was genuinely committed to building FAI in 2011-2013; Pressman claims it then believed mathematical purity could get it to AGI first.
• Pressman cited a 2013 post by MIRI co-founder Luke Muehlhauser; Habryka countered that Luke explicitly did not want to build FAI himself and that Eliezer wrote about the dangers of trying to take over the world as early as 2004.
• jessicata cited MIRI's 2017 fundraiser and strategy update, describing a strategy of preserving optionality over which projects to help and staying secret about AGI-relevant insights, rather than 'some organization should execute task AGI.'
• Wei Dai clarified that Pressman initially relayed others' 'take over the world' claim rather than asserting it himself.

Unconfirmed

• Whether MIRI ever seriously plotted world takeover remains unsettled: Pressman says the mathematical object MIRI defined was a way to ethically take over the world, yet sarcastically adds the claim is too flattering—MIRI let fans believe it while never taking the idea seriously internally; Habryka denies gaslighting and vows to rebut the claim '50 more times.'
• Whether CEV equals takeover is disputed: Habryka argues unilateral global sovereignty would be takeover even if it spread democracy, and CEV would empower hypothetical or simulated humans rather than real ones; Wei Dai counters that takeover means disempowering others, while CEV aims to maximally empower humanity.
• Wei Dai also cited ThomasCederborg's LessWrong post: parliamentary CEV (PCEV) grants extra influence to people who intrinsically enjoy hurting others, and successful implementation could be worse than extinction, underscoring the importance of Alignment Target Analysis (ATA).

Why it matters

• The dispute exposes diverging historical narratives within AI safety and bears on assessments of MIRI's trajectory and credibility; the CEV empowerment-vs-takeover debate and PCEV's ethical flaws show that choosing an alignment target remains an open safety problem.

2026-08-15 ~ 2026-08-16 · 25 related posts

• Debate over MIRI's mission statement and Friendly AI creation — jd_pressman · 2026-08-15
• Habryka denies gaslighting in MIRI takeover debate, cites LessWrong discussion — ohabryka · 2026-08-15
• Habryka stands firm: will repeat MIRI didn't try to take over world 50 times — ohabryka · 2026-08-15
• JD Pressman: MIRI Never Seriously Tried to Take Over the World, Just Allowed Fans to Imagine — jd_pressman · 2026-08-15
• [source] Habryka argues MIRI didn't try to take over world, citing Luke and Eliezer — ohabryka · 2026-08-15
• jd_pressman questions: is tinkering with CEV during takeover legitimate? — jd_pressman · 2026-08-15
• Habryka clarifies: denies MIRI takeover, admits CEV brain-in-box plan — ohabryka · 2026-08-15
• jd_pressman insists: MIRI defined math object to take over world — jd_pressman · 2026-08-15
• JD Pressman: MIRI's Math Object Is an Ethical Way to Take Over the World — jd_pressman · 2026-08-15
• Habryka: 'take over the world' is misleading, use other terms — ohabryka · 2026-08-15
• CEV Ethical Flaw: PCEV Gives Extra Influence to Sadists, Outcome Worse Than Extinction — weidai11 · 2026-08-15
• Habryka: improving world isn't takeover; CEV empowers others — ohabryka · 2026-08-15
• CEV: Empowerment or Power Grab? AI Safety Debate Continues — weidai11 · 2026-08-15
• [source] MIRI Debate Intensifies: Is CEV Equivalent to 'Taking Over the World'? — weidai11 · 2026-08-16
• MIRI's Early FAI Ambitions Spark Heated AI Safety Debate — jd_pressman · 2026-08-16
• JD Pressman accuses MIRI members of rewriting history and gaslighting — jd_pressman · 2026-08-16
• [source] JD Pressman accuses MIRI members of rewriting history and gaslighting — jd_pressman · 2026-08-16
• MIRI's 2017 Strategy: Retaining Optionality on Alignment-Capabilities Insights — jessi_cata · 2026-08-16

7 near-duplicate retellings: jd_pressman · ohabryka · jd_pressman · jd_pressman · jd_pressman · jd_pressman · jd_pressman