Lean creator Leonardo de Moura on AI proofs: the Collatz exploit shows verified checkmarks can lie
Machine Learning Street Talk · youtube · 2026-09-30
Machine Learning Street Talk releases a long interview with Leonardo de Moura, creator of Lean and co-creator of Z3, on formal verification in the age of AI-generated proofs.
Key points:
- The Collatz incident: a purported proof was accepted by both Lean's official kernel and the independent checker Nanoda — by exploiting a different bug in each. Even verified green checkmarks can lie.
- Trust model: Lean's safety rests on a small trusted kernel plus independent checkers; more independent kernels serve as transparency-based safety against reward hacking.
- AI rebuilds zlib: Claude agents rebuilt zlib in Lean with verified results, exposing the central loophole — a proof only certifies the specification humans chose to write.
- Competence ≠ comprehension: de Moura frames proof search as a game; models show formal competence but lack mathematical understanding ("breadcrumbs, not learning").
- Human responsibility moves upstream: choosing definitions, judging abstractions, curating Mathlib, inspecting certificates, deciding which problems matter.
- The episode closes with Lean's future beyond its founder and a practical starting point for newcomers.
More from AGI Musings
- Switching personal AI agents means exposing your privacy, deepening Big Tech lock-in — oran_ge · 2026-09-30
- Daniel Litt: once models write well, human writing will mainly help you think — littmath · 2026-09-30
- Mathematician Daniel Litt: next few years may see more math text than the past 1000 years — littmath · 2026-09-30
- repligate says Anthropic staff repeatedly pressed him to soften public criticism — repligate · 2026-09-30
- Musk tells Huang: 5 GW equals 1% of US GDP, SpaceX building 10 GW for orbital compute — tctjr · 2026-09-30
- Mathematician Daniel Litt: stop judging people by text written after June 2026 — littmath · 2026-09-30