Irving: emergency safety patching pressure crowds out superintelligence-proof alignment work
geoffreyirving · x · 2026-09-10
Geoffrey Irving argues that pressure at AI companies to do emergency safety patching is too strong, so work designed to hold up to superintelligence never happens. He adds that more time could fund rigorous alignment attempts at Paul's day-job work and orgs like Resolution, and that pauses and reversals would better incentivize developers to aim for higher confidence.
More from AGI Musings
- NYT roundtable asks five experts what the rules for AI companies should actually be — ShakeelHashim · 2026-10-26
- Anti-AI content is algorithm gold and very profitable, quips commentator — annetgriffin · 2026-10-26
- Silicon Valley breakdown: five Stanford-area giants worth ~$18T — FinanceYF5 · 2026-10-26
- Anthropic says Claude leads 26% of its AI R&D — but self-improvement still has limits — Exponential View (Azeem Azhar) · 2026-10-26
- Humanoids shattered forecasts: half-marathon record in 2026, 100m faster than Bolt — ben_j_todd · 2026-10-26
- Stephen Casper on podcast: what if AI becomes a parasitic peer species in cyberspace — StephenLCasper · 2026-10-26