LeanPRM: Lean-grounded step labels train a 4B verifier lifting accuracy 48.2→74.7
Pavel_Izmailov · x · 2026-11-07
Pavel Izmailov's team released LeanPRM, a framework that uses the Lean proof assistant to generate step-level correctness labels for natural-language math proofs with zero human annotation.
- Method: a model writes each natural-language proof alongside a paired formal Lean proof; Lean labels every step, and a second model filters out failures caused purely by formal syntax so negative labels mark genuine math errors.
- Scale: 10,634 proofs with 152K labeled steps, plus LeanPRM-Bench, a 200-proof held-out set.
- Results: a Qwen3-4B verifier trained on LeanPRM raises exact step-label accuracy on LeanPRM-Bench from 48.2 to 74.7, versus 60.3 when trained on PRM800K; it also wins best-of-N selection on IMO-level problems against the PRM800K verifier in three of four settings.
- Motivation: the authors hope Lean plus formal-to-informal generalization can help supervise superhuman models, addressing the bottleneck of checking long proofs step by step.
Related event: LeanPRM Trains Math Proof Verifiers on Lean-Generated Synthetic Data(3 posts)→
More from Research
- Unverified claims: frontier labs allegedly gaming ICLR reviews with malicious reviews — examachine · 2026-11-07
- SSF teaches robots to reason about likely hiding spots when searching for hidden objects — _krishna_murthy · 2026-11-07
- 'Die getting Schmidhubered or live long enough to become one': scaling law priority drama — teortaxesTex · 2026-11-07
- Stanford and Together AI propose Next-Concept Modeling to replace next-token prediction — bendee983 · 2026-11-07
- New benchmark shows some LLMs refuse over 50% of legitimate biology queries — nlarusstone · 2026-11-07
- From AI-designed phage DNA to patient care: the ecosystem behind the model — exnx · 2026-11-07