AI safety researcher: labs are at 'slapdash empirics' level, three tiers below aerospace-grade rigor
geoffreyirving · x · 2026-10-23
Geoffrey Irving lays out four increasing levels of safety rigor — slapdash empirics (where AI labs are now), system safety (aviation/nuclear), conditional proofs (cryptography-style), and unconditional proofs — arguing that aligning AI requires somewhere in levels 2–4, though nobody knows exactly where. He also recounts a debate with a lab safety leader who described their strategy as 'holding down the fort': patching RL environments until a pause becomes possible in a few years or AI research can be handed to AIs.
More from AGI Musings
- Beff and Ashe Talk Techno-Optimism: How Society Should Navigate the AI Transition — beffjezos · 2026-10-23
- Embodied AI Makes Authority Physical: Robots Need an Inspectable Execution Trail — tallmetommy · 2026-10-23
- Chinese AI labs embrace 'recursive self-improvement' — but the story is bottom-up, not state-coordinated — Afinetheorem · 2026-10-23
- AI researcher: even dumb models are now powerful hacking tools, sparking an AI cyber arms race — Yuchenj_UW · 2026-10-23
- Non-programmer in his 50s built a full property dashboard with Codex controlling all his smart homes — craigsdennis · 2026-10-23
- Headless everything: why the frontend is sacrificial in the era of personal AI agents — genmon · 2026-10-23