Researchers debate AI risk over "8-year-olds ruling society" analogy

On November 14, Ars Technica journalist Timothy B. Lee (@binarybits) debated LessWrong author Raemon on X about superintelligence and existential risk. The core question: how exactly could AI in a data center take over the physical world?

Confirmed

• Lee challenged the common argument that "AI bribes or deceives humans into doing its bidding," calling that path highly implausible; he made clear his concern would rise sharply once general-purpose robots arrive, implying they are the true danger inflection point.
• In exchanges with Raemon, Lee elaborated: he rejects the narrative that "superintelligence will inevitably seize power," arguing that intelligence is overrated as a route to power—physical embodiment, social relationships, control of physical resources, and first-mover advantages matter far more than intelligence itself.
• Lee proposed that the key variable in the debate is "how hard simulating the physical world is," spanning physics, biology, and even economic and social systems; in his view this determines whether the superintelligence narrative holds.
• Raemon noted that many of their disagreements boil down to one fundamental judgment: "does being smarter make something significantly more powerful?"
• Raemon also pressed from another angle: even granting that AI usefulness plateaus near human level, why not worry about large numbers of AIs coordinating to "take over" the world the way European colonizers—slightly ahead technologically—took over the Americas and Africa (all while claiming it was "for your own good")?

Why It Matters

This debate touches one of the most fundamental empirical divides in AI safety and existential-risk discussions: whether intelligence translates into real-world power. Lee's stance shifts the focus from "superintelligence" to discussable concrete conditions like "general-purpose robots" and "the difficulty of simulating the physical world," while Raemon stresses that even without assuming an intelligence explosion, scaled AI coordination itself poses a risk. How such arguments resolve directly shapes whether society should point safety efforts at model capabilities themselves, or at more concrete scenarios like robotics, misuse, and coordination failures.

2026-11-14 ~ 2026-11-16 · 14 related posts

• Society of 8-year-olds thought experiment challenges whether intelligence alone guarantees power — Jess_Riedel · 2026-11-14
• [source] Tim Lee: How hard it is to simulate the physical world is key to the ASI debate — binarybits · 2026-11-14
• Humans controlling superhuman AI is like eight-year-olds running society, writer argues — binarybits · 2026-11-14
• Timothy Lee: Eight-year-olds can't stay in charge of anyone, so the controlled party's intelligence doesn't matter — binarybits · 2026-11-14
• [source] Raemon: Even if AI intelligence caps near humans, coordinated AI takeover is a real risk — Raemon777 · 2026-11-14
• Tim Lee: Intelligence isn't that important for power — bodies and resources matter more — binarybits · 2026-11-14
• Tech journalist questions how a data-center AI could take over the physical world — binarybits · 2026-11-14
• [source] How does a data-center AI take over the physical world? Only general robots worry me — binarybits · 2026-11-14
• ML pioneer Dietterich: AI risk hinges on physical power, not intelligence—dictators control smart people easily — tdietterich · 2026-11-15
• Physical power, not intelligence, determines control: rebuttal to superintelligence analogy — Jess_Riedel · 2026-11-15
• Tim Lee: A society of cognitively impaired adults would collapse long before AGI arrives — binarybits · 2026-11-16
• Tim Lee doubles down: a society of 8-year-olds couldn't exert meaningful long-term control over anyone — binarybits · 2026-11-16
• binarybits challenges the AI-as-children-society analogy: too many disanalogies — binarybits · 2026-11-16
• Timothy Lee challenges the child-rearing analogy for AI agent control — binarybits · 2026-11-16