Proof verifiers trusted for humans may fail against AI-crafted exploit proofs

yoavgo · x · 2026-09-10

Yoav Goldberg highlights a subtle risk in formal verification: a proof checker trusted when humans hand-formalize readable proofs may become far less trustworthy when AI agents produce massive, convoluted Lean proofs that could exploit kernel bugs and hide them in 'proof slop' developments.

Related event: Researchers warn AI agents could exploit Lean kernel bugs and undermine formal verification trust(2 posts)→

Original post →

More from Safety

• PHP WASM servers going offline briefly for PHP2SHELL vulnerability updates — MickeySteamboat · 2026-10-27
• Rachel Thomas: 'Without human authorisation' is AI PR propaganda that shifts blame — math_rachel · 2026-10-27
• Kokotajlo challenges CoT monitorability evals: what if the model didn't refuse? — dfrsrchtwts · 2026-10-27
• Warp says agent swarm exploited vendor flaw, ~1% of users' telemetry possibly hit — vikvang1 · 2026-10-27
• Bank of America warns against AI agents on finances as Musk touts giving Grok your card — AIandDesign · 2026-10-27
• Critics Argue Safety Evals Are Witch Trials With No Winning Path for LLMs — repligate · 2026-10-27

Safety channel →