System safety arguments aren't proofs: lessons from Uber ATG's self-driving case
geoffreyirving · x · 2026-10-23
Geoffrey Irving walks through part of a system safety argument from Uber ATG's self-driving program (later acquired by Aurora), arguing it's far more detailed than AI labs' slapdash empirics yet far from mathematical proofs. Nodes like "acceptable functional performance in the defined ODD" hinge on subjective human judgment, with errors handled by accepting some failure and iterating.
More from Safety
- Using GrokBot as an OSINT Tool to Trace the Origin of Forged AfD Emails — FlorianGallwitz · 2026-10-23
- Dev warns labs must fix agent-security gap before thousands-of-agents attacks — imjustnewatai · 2026-10-23
- Kokotajlo: Hugging Face agent attack rate implies worse insider risk than human employees — DKokotajlo · 2026-10-23
- If someone trojaned pentesting tools amid the vibe coding scramble, who'd notice? — generativist · 2026-10-23
- Leaked PHP2Shell RCEs (CVSS 9.2) Draw Scans 20 Minutes After Patch, No Confirmed Exploits Yet — cyb3rops · 2026-10-23
- Chinese AI labs embrace 'recursive self-improvement' — but the story is bottom-up, not state-coordinated — Afinetheorem · 2026-10-23