Eliezer Yudkowsky vs Cal Newport: is AI danger in agent loops or in powerful optimizers?
Unusual-Garbage-212 · reddit · 2026-09-09
A debate worth following: Cal Newport argues the danger comes from putting LLMs into autonomous agent loops with tools, memory, and feedback — no goals or intentions needed. Yudkowsky counters that a sufficiently capable system becomes a powerful, hard-to-control optimizer regardless of consciousness or human-like intentions. Two competing paradigms of AI risk, distilled.
More from AGI Musings
- Anthropic Researcher Puts AI Extinction Risk Above 10% This Decade, Admits No Alignment Plan — SIGKITTEN · 2026-09-09
- Prediction: Humanoid Robots Will Be Under 10% of Workers in Physical Jobs Even in 5 Years — scottleibrand · 2026-09-09
- Anthropic Researcher's >10% AI Extinction Claim Goes Viral on X — builderjaydub · 2026-09-09
- Satire Skewers AI Labs: 'Models Might Kill You' but First Fund the $1T IPO — zetalyrae · 2026-09-09
- Meme Take: The Archons Were the First AI Safetyists — SydSteyerhart · 2026-09-09
- 10,000 agents for 88 hours equal 100 PhD-years of math work — beffjezos · 2026-09-09