Paper: Lean verification of AI autoformalisation doesn't guarantee correct natural language proofs
natanielruizg · x · 2026-10-08
A new arXiv paper by Bastounis, Circelli and Hansen challenges the reliability of autoformalisation — the pipeline behind OpenAI's announced Navier-Stokes blow-up proof — where AI translates natural-language math into Lean for mechanical verification.
Key claims:
- Passing Lean verification does not guarantee the original natural-language argument is correct, because translation may be semantically unfaithful.
- Resolving ambiguities in mathematical natural-language text (needed for faithful translation) sits arbitrarily high in the Solvability Complexity Index hierarchy (SCI = ∞), making it harder than any computational problem, including the Halting problem (SCI = 1).
- The authors provide concrete examples of AI mistranslations of NL statements and proofs into Lean, producing mismatches between the informal and formal versions.
More from Research
- Are URM and Universal Transformers the forgotten architecture beating standard LLMs? — moschles · 2026-10-09
- Researcher: Use AI to Rewrite Machine-Generated Math Proofs Into Human-Readable Forms — jd_pressman · 2026-10-09
- AutoScientist's two-agent checklist loop auto-audits every training example — sarahookr · 2026-10-09
- NeurIPS GenAI4Health oral: retrieval-based medical fact-checking fails in ways bigger models can't fix — mdredze · 2026-10-09
- Bigger models, more reasoning, better sources won't fix medical fact-checking, researchers say — mdredze · 2026-10-09
- DEX best abstract: top LLMs catch many physician diagnostic errors, but big gaps remain — mdredze · 2026-10-09