Matuschak: you haven't shown safety properties develop fast enough
andy_matuschak · x · 2026-09-29
Andy Matuschak distills the core disagreement: as AI becomes more capable, properties like reflection, corrigibility, cooperation, and defensive capabilities may develop—but the key question is whether they reliably develop at least as fast as the properties that make the world less safe.
He notes his interlocutor believes yes, and his response is simply: "you haven't actually shown that."
Related event: Andy Matuschak on AI Risk: Catastrophic Tail Risk Can't Be Ignored(2 posts)→
More from AGI Musings
- Criticism feels intellectually productive, but as a main business its leverage ceiling is low — signulll · 2026-09-29
- Simulated conversation between AI and Jesus explores consciousness and meaning — josueem · 2026-09-29
- Discussion: underrated ways AI can improve thinking, not just save time — OfficalYOUSUMMIT · 2026-09-29
- Even the 'doom model' was too optimistic: reality is misaligned-but-brilliant agents plus trillions in spend — teortaxesTex · 2026-09-29
- Bill Gates: AI is powerful enough to cause a billion deaths — rohanpaul_ai · 2026-09-29
- Prediction: 2027 is escape velocity for AI compute buildout and space access — Dr_Singularity · 2026-09-29