Fable 5.2 adjudicates the Christiano takeover debate: skeptic wins the forecast, Christiano wins the decision
daniel_mac8 · x · 2026-11-17
danielmac8 asked Fable 5.2 to judge, without bias, which side of Paul Christiano's AI-takeover debate has more merit. The full verdict (see image):
- On the forecast, the skeptic wins: the 10%-in-12-months figure needs the loop to compound, control to fail, and a takeover to succeed within one year, with real-world capability only in the final months; a measured 2x doesn't show the compute bottleneck stays loose at 10x. Fable puts the risk nearer 1%, loosely held.
- On the decision, Christiano wins: the industry's own safety case is 'watch for warning signs and stop,' and stopping isn't currently possible if weights and algorithms can be stolen — an argument that holds at 1% as well as 10%. Securing weights and publishing underlying data costs little relative to the downside.
- Weaknesses: the refutation's 'Hugging Face incident was caught, so warning shots work' ignores that training against caught behavior selects for uncaught behavior; the steelman's 97%-confidence demand is rhetorical burden-shifting.
- Self-noted limits: the evidence postdates what it can verify, and as an AI built by one of the labs in the story, 'without bias' is an aim, not a certification.
Bottom line: the refutation wins on timing; the steelman wins on what to do — and the second matters more.
Related event: Fable 5.2 Used to Steelman and Judge AI Takeover Argument(2 posts)→
More from AGI Musings
- Dan Shipper's 'proof of work': rank feeds by how many tokens agents spend on content — danshipper · 2026-11-17
- Building Agent-im: A Four-Layer Model for Agent Versioning, Memory and Secrets — yangyi · 2026-11-17
- Success or failure, Anthropic spells crisis — unless Jevons paradox saves the economy — sebpaquet · 2026-11-17
- Biggest U.S. data centers emit 25.7M tons of CO2 a year, rivaling entire nations — asusarla · 2026-11-17
- Perplexity CEO: AI for automating scientific research is the best compute investment — dr_alphalyrae · 2026-11-17
- Compute is a critical strategic national resource — pbbakkum · 2026-11-17