Skeptic: OpenAI can't formalize 'this AI won't murder you' in Lean
zetalyrae · x · 2026-11-17
User @RatOrthodox argues OpenAI will not be able to formalize a guarantee like "this AI will not murder you" in Lean. Any attempt, they claim, would run into the same problems GOFAI-era alignment researchers discussed in the late 1990s: behavioral and moral guarantees resist formalization as checkable mathematical proofs.
A concise entry in the ongoing debate over whether formal methods can ever verify frontier model safety.
Related event: Skeptics Question OpenAI's Formal AI Safety Plans(3 posts)→
More from AGI Musings
- Andrew Trask: Model Stability Is a Safety Feature, Not a Risk — iamtrask · 2026-11-17
- State of AI Report 2026: frontier is now a three-lab race, plus agents ran 12 expert interviews in 2.5 days — nathanbenaich · 2026-11-17
- Are top AI conferences failing? High-quality authors may be dropping out — analisereal · 2026-11-17
- 10,000 collaborative agents crack a near-unsolvable math problem via parallel compute — emmanuelvivier · 2026-11-17
- “Stop aiming for 20% productivity — AI should end work-for-money altogether” — VraserX · 2026-11-17
- iamtrask: agent time horizon measures stability, not intelligence — don't conflate ASI signals — iamtrask · 2026-11-17