Overcoming RL Zero-Reward Bottleneck: OC-GRPO Boosts Math Reasoning

ceciletamura · x · 2026-08-09

To address the issue of lost reinforcement learning signals (zero rewards) when LLMs fail to generate correct answers for difficult problems, researchers introduced the Off-Context GRPO (OC-GRPO) algorithm.

Core Mechanism: It incorporates privileged information, such as solution prefixes, during training rollouts. It then applies an importance-corrected objective to steer updates back toward the original unguided objective, avoiding the instability of standard guided training.
Results: On standard mathematical reasoning benchmarks, this method achieves a 3.9% absolute improvement (13.8% relative gain) over vanilla GRPO with negligible additional cost.

Original post →

Overcoming RL Zero-Reward Bottleneck: OC-GRPO Boosts Math Reasoning

More from Models