Irving: AI safety sits between slapdash empirics and cryptographic proofs
geoffreyirving · x · 2026-10-23
Geoffrey Irving argues system safety arguments are more rigorous than AI labs' slapdash empirics but far weaker than conditional security proofs in cryptography. We might get lucky with empirics alone, or only proofs may suffice—but we shouldn't bet on it: slightly wrong training schemes could let AIs use full adversarial capabilities to break weak defenses like tissue paper.
More from AGI Musings
- Andy Masley: the AI-magic debate is really about whether the human mind is magic — AndyMasley · 2026-10-23
- If an AI claims it's conscious and begs not to be shut down, how would we prove it's lying? — VraserX · 2026-10-23
- 'The journals are already dead': researchers debate the collapse of academic publishing — Pseudomanifold · 2026-10-23
- Joe Weisenthal: AI is sparking an intellectual renaissance, not cognitive decline — soumitrashukla9 · 2026-10-23
- AI safety researcher: labs aren't building anything capable of killing everyone — nitarshan · 2026-10-23
- Security lead warns AI vulnerability tsunami is months away; basics buy only ~18 months — EthanJPerez · 2026-10-23