ramez on AI safety: agency can be layered, and broad capability is a defense
ramez · x · 2026-11-04
Ramez distinguishes two issues in an alignment debate: (1) he agrees there's a precision/recall tradeoff in agency/instruction following, but argues agency in service of instructions differs from agency over values — the former is simpler and keeps goal-setting with humans; (2) on the "super-weapon for everyone" critique, he says real safety also comes from system resilience: if everyone has similar AI capability, no one holds a super weapon. RazRazcle had pushed back that useful long-horizon AI can't lack agency.
Related event: Debate rages over whether democratizing powerful AI is safe(5 posts)→
More from AGI Musings
- Ex-OpenAI designer launches tenderworld studio with a $349 camera that prints poems instead of photos — ZeroStateReflex · 2026-11-05
- Commenter: AI companies' backlash is self-inflicted by image, video and voice models — karmay007 · 2026-11-05
- Neel Nanda debates tszzl: covert model underperformance is 'unjustifiable' — tszzl · 2026-11-05
- Building orbital data centers to dodge regulators is 'completely unserious' — zacharynado · 2026-11-05
- Reddit essay: AI may never be conscious—and that's fine — Own-Cry-5520 · 2026-11-05
- Internal safeguards won't stop hostile AI, researcher argues for external defenses — IgorGabrielan · 2026-11-05