Researcher says RLVR gains on noisy labels are spurious, not real model improvement

StellaLisy · x · 2026-08-04

The author argues that the paper’s interpretation of RLVR is overstated.

Prior work on Spurious Rewards was not claiming that incorrect or random labels truly improve models; it showed that such labels can produce spurious learning that does not reflect real model improvement.
The paper’s setup is also questioned: it appears to use fully on-policy training with async turned off, which matches the author’s own ablation showing that once clipping is removed or on-policy training is used, the spurious gains disappear.
The takeaway: spurious rewards should not be read as a fundamental way to improve models, but as a warning that researchers need careful benchmarks and more rigorous evaluation before making strong claims.

More from Research