OC-GRPO paper: privileged guidance unlocks RLVR learning on hard problems
ceciletamura · x · 2026-11-05
A new arXiv paper introduces Off-Context GRPO (OC-GRPO), tackling the zero learning-signal problem in RLVR when models can't produce any correct solution on hard problems.
- Adds privileged guidance (e.g. solution prefixes) during training to produce non-zero-reward "off-context" rollouts
- An importance-corrected objective steers updates back to the original unguided objective, avoiding training mismatch
- Results: 3.8% absolute (13.7% relative) average gain over vanilla GRPO on standard math reasoning benchmarks at negligible extra cost
Related event: Meta AI Proposes OC-GRPO to Tackle Zero-Reward Learning Cliff(2 posts)→
More from Research
- RoboColosseum founder: robotics lags AI because it lacks real-world benchmarks — DJiafei · 2026-11-05
- ByteDance study finds on-policy distillation leaves models 2.45-16.25% below base on out-of-distribution tasks — ByteDance · 2026-11-05
- FrontierPhysics: 56 expert tasks benchmark AI agents on end-to-end physics research — Bingran You · 2026-11-05
- VideoMemBench: new benchmark exposes major gaps in MLLMs' in-context video memory — Zhengdong Hu · 2026-11-05
- Muon-RT swaps momentum for teacher trajectory, cutting memory and beating AdamW by 2.6 points — IITK · 2026-11-05
- Self-Harness: Shanghai AI Lab's Method Lets Agents Fix Themselves, 30.2%→84.6% Without Retraining — Promptmethus · 2026-11-05