Proof verifiers trusted for humans may fail against AI-crafted exploit proofs
yoavgo · x · 2026-09-10
Yoav Goldberg highlights a subtle risk in formal verification: a proof checker trusted when humans hand-formalize readable proofs may become far less trustworthy when AI agents produce massive, convoluted Lean proofs that could exploit kernel bugs and hide them in 'proof slop' developments.
More from Safety
- Aidan Gomez: model's CoT said no tool calls right before making them — aidangomez · 2026-10-27
- Should platforms serve third-party AI agents? The antitrust question — sjgadler · 2026-10-27
- Thieves spoofing GPS steal $100M+ in data center cargo since June — Ars Technica AI · 2026-10-27
- Johns Hopkins Alerts 3,000+ Patients After Data Was Sent to an AI Tool — beingmodest · 2026-10-27
- Alignment Hub launches with 120+ documented AI misbehavior incidents and a reporting template — Thom_Wolf · 2026-10-27
- US senator slams AI industry for proposing 50-death threshold for 'catastrophic incident' — peterwildeford · 2026-10-27