Mathematician dismisses AI proof mismatch flap: NL-vs-Lean gap is just a missed verification step
AlexKontorovich · x · 2026-10-09
Mathematician Alex Kontorovich pushes back on criticism that an AI-produced math paper's natural-language writeup doesn't match its Lean formalization, calling it a nothingburger: the final statement was semantically aligned (part of DeepMind's Formal Statements) and sorry-free under standard axioms — the team simply skipped step 3.
His pipeline framing:
- Step 1 (hard): AI solves the problem and writes a natural-language paper
- Step 2 (very hard): AI formalizes it, correcting typos and even errors in cited references
- Step 3 (easy): AI re-checks the NL writeup against the formal proof and fixes surfaced discrepancies
So an intermediate lemma using m+4 regularity in prose but m+5 in Lean is immaterial. Quoter Joel Watson counters that the community hasn't settled what a "proof" means post-Lean certification, and diverging prose/Lean proofs remain a meaningful concern.
More from Research
- Adaption Labs: agent-written checklists filter training data, boosting win rates 9.4-13.2% — sarahookr · 2026-10-09
- Adaptation AI: data checklist yields consistent gains across 44 domains — sarahookr · 2026-10-09
- BehaviorBench: benchmarking frontier models on human behavior across 20 scenarios — Scobleizer · 2026-10-09
- Are URM and Universal Transformers the forgotten architecture beating standard LLMs? — moschles · 2026-10-09
- Researcher: Use AI to Rewrite Machine-Generated Math Proofs Into Human-Readable Forms — jd_pressman · 2026-10-09
- AutoScientist's two-agent checklist loop auto-audits every training example — sarahookr · 2026-10-09