Paper says a claimed “100% wrong” MATH set missed many valid answers

burny_tech · x · 2026-08-04

The paper argues that a prior claim about a "100% wrong" MATH dataset was flawed because the dataset construction missed many valid equivalent answers.
The authors show that MATH problems can have multiple correct answers and multiple equivalent expressions, so labeling an answer wrong is not always correct.
The attached figure is from "Spurious Rewards: Rethinking Training Signals in RLVR", which studies how weak or even spurious training signals affect MATH-500 accuracy across models such as Qwen2.5-Math-7B, Qwen2.5-7B, Llama3.1-8B-Instruct, and OLMo2-7B.
The main takeaway: some reward signals can boost certain Qwen models, but the same signals do not transfer uniformly to other models with different reasoning priors.

More from Research