AdvancedMathBench: A Higher-Level Math Proof Benchmark

internlm · hf · 2026-07-14

The authors propose AdvancedMathBench to evaluate large models on advanced mathematical proof generation and verification. Existing math benchmarks mostly focus on high school or competition problems and often only check the final answer, making it hard to measure the actual validity of the proof process. This benchmark attempts to fill the gaps in "advanced mathematical reasoning" and "proof verification".

Benchmark Composition

• ProverBench: 296 problems covering undergraduate to PhD qualifying exam levels.
• VerifierBench: 888 model-generated proof trajectories paired with human expert ground truth, used to evaluate if models can judge a proof's validity and provide reasonable explanations.
• It also builds an automated verification pipeline trained on large-scale expert annotations, outputting correctness judgments and fine-grained error analysis, which shows strong alignment with human experts on a hold-out set.

Results

Experiments show this benchmark is still very challenging for frontier models:

• The best model scores only 75.8 and 66.1 on the UGD and QE splits for proof generation, respectively.
• The best Balanced F1 for proof verification is only 65.1.
• The overall true negative rate is low, indicating that "identifying incorrect proofs" remains a clear bottleneck.

Original post →

More from Research

• Diamandis: AI Cancer Navigator and $100 Genome Sequencing Close the Care Gap — PeterDiamandis · 2026-10-27
• EMNLP Oral: training-free Nexus Sampling cuts KV-cache memory 10x; OpenResearcher tops BrowseComp-Plus at 54.8% — yuz9yuz · 2026-10-27
• ICML 2026 position paper: LLM 'thinking traces' are a harmful anthropomorphism — rao2z · 2026-10-27
• Forking gap follows a power law in context frequency; masking rare n-grams shrinks it — ZimingLiu11 · 2026-10-27
• How forking happens: n-gram tables split logits into non-interacting subspaces — ZimingLiu11 · 2026-10-27
• The best LoRA explainer video yet: rank constraints meet intrinsic dimension — sophiamyang · 2026-10-27

Research channel →