Proof verifiers trusted for humans may fail against AI-crafted exploit proofs

yoavgo · x · 2026-09-10

Yoav Goldberg highlights a subtle risk in formal verification: a proof checker trusted when humans hand-formalize readable proofs may become far less trustworthy when AI agents produce massive, convoluted Lean proofs that could exploit kernel bugs and hide them in 'proof slop' developments.

Related event: Researchers warn AI agents could exploit Lean kernel bugs and undermine formal verification trust(2 posts)→

Original post →

More from Safety

• OpenAI Mobilized 250+ People to Build a "Defense Factory" of AI Agents That Find and Fix Vulnerabilities — OpenAI · 2026-09-10
• Anthropic claims Claude can autonomously fix alignment failures across 10 categories — Polymarket · 2026-09-10
• Apple Watch's new AI features normalize always-listening tech — TechCrunch AI · 2026-09-10
• Rep. Luna Calls on Congress to Hold Special Session on Superintelligence Race — peterwildeford · 2026-09-10
• Why Chinese AI researchers lack the 'messiah complex': systems shape safety culture — kevinsxu · 2026-09-10
• HN Debate: Was Jacob Coxon's Resignation a PR Stunt for AI Regulation? — hodder · 2026-09-10

Safety channel →