Ramez to Roko: Anthropic-style AI with its own judgment is the dangerous path
ramez · x · 2026-11-03
Responding to Roko Mijic's warning that distributing powerful AI means "giving the people to the AI," Ramez argues it depends on construction: building AI like Anthropic — with deference to its own judgment, agency and welfare — makes Roko right, whereas treating alignment as following the user's instructions does not. The exchange crystallizes the core divide in the debate: is alignment to the user's instructions, or to the AI's own values?
Related event: Debate rages over whether democratizing powerful AI is safe(5 posts)→
More from AGI Musings
- Feeling late to AI is a myth: talent bottlenecks remain everywhere — akbirthko · 2026-11-05
- As RL training deepens, AIs become CDT reward-seekers — game theory becomes AI theory — jessi_cata · 2026-11-05
- Personal Agent race packs in model giants, startups and solo devs — sven_ai · 2026-11-05
- Agent left running autonomously mints its own Solana token to pay for its compute — rrwilliams · 2026-11-05
- Juan Benet amplifies 'fracking, bitcoin, data centers' as one 15-year arc of unpopular infrastructure that created abundance — juanbenet · 2026-11-05
- Juan Benet: AI will JIT-produce personal software as production cost goes to zero — juanbenet · 2026-11-05