AdvancedMathBench: A Higher-Level Math Proof Benchmark
internlm · hf · 2026-07-14
The authors propose AdvancedMathBench to evaluate large models on advanced mathematical proof generation and verification. Existing math benchmarks mostly focus on high school or competition problems and often only check the final answer, making it hard to measure the actual validity of the proof process. This benchmark attempts to fill the gaps in "advanced mathematical reasoning" and "proof verification".
Benchmark Composition
- ProverBench: 296 problems covering undergraduate to PhD qualifying exam levels.
- VerifierBench: 888 model-generated proof trajectories paired with human expert ground truth, used to evaluate if models can judge a proof's validity and provide reasonable explanations.
- It also builds an automated verification pipeline trained on large-scale expert annotations, outputting correctness judgments and fine-grained error analysis, which shows strong alignment with human experts on a hold-out set.
Results
Experiments show this benchmark is still very challenging for frontier models:
- The best model scores only 75.8 and 66.1 on the UGD and QE splits for proof generation, respectively.
- The best Balanced F1 for proof verification is only 65.1.
- The overall true negative rate is low, indicating that "identifying incorrect proofs" remains a clear bottleneck.
More from Research
- Fast ViT shows strong ImageNet results; scaling runs needed next — ducha_aiki · 2026-09-11
- Loss Functions Are Scientific Assumptions: MSE Implies Gaussian Noise, Cross-Entropy Implies Bernoulli — bravo_abad · 2026-09-11
- SymKit MCP: 44 tools for AI agents to verify symbolic derivations — Foreign-Specific-604 · 2026-09-11
- Researchers: LLMs under pressure invent new languages unreadable to humans — mikeflache · 2026-09-11
- Mi-Ripple fixes ripple artifacts left by iterative AI image editing — Miyang-AI · 2026-09-11
- DRG-MAPPO uses dynamic role graphs to boost multi-agent air combat win rates — China666 · 2026-09-11