Lean4 proofs are not a silver bullet: consistency gaps, soundness bugs, and flawed benchmarks
elie · x · 2026-10-11
A critical look at why Lean formal proofs can't (yet) be blindly trusted, even though having a proof is valuable:
- Foundational: Lean 4's underlying theory lacks a complete consistency proof, so it may in principle accept proofs that are simply wrong.
- Practical: the Lean codebase likely contains soundness bugs that let invalid proofs pass — the author's team found some as recently as August.
- Proving the wrong thing: a recent ICML study found hundreds of verified defects in Lean theorem-proving benchmarks.
Bottom line: a Lean proof is not a substitute for validating what was proved and trusting the system that checked it.
More from Research
- Bittensor subnet Metanova uses competition-driven AI drug discovery with robot labs — const_reborn · 2026-10-11
- Training inside the harness lifts Qwen3-14B from 22.2% to 54.8% on Spider 2.0-SQLite — omarsar0 · 2026-10-11
- REMAG brings GPU-first SIESTA to magnetic materials with up to 111x speedups — CatAstro_Piyush · 2026-10-11
- Yoav Goldberg blasts COLM mood: RL loss tweaks aren't 'science again' — yoavgo · 2026-10-11
- Stanford paper: fix looping agents via harness edits, bad plans need weight training — rohanpaul_ai · 2026-10-11
- Harvard Med workshop teaches building AI co-scientists with ToolUniverse's 2,700+ tools — marinkazitnik · 2026-10-11