Scoring Higher, Answering Worse: ByteDance's ProRubric Fixes Reward Hacking in Rubric-RL
ByteDance · hf · 2026-10-14
Researchers from BUPT and ByteDance show that additive aggregation is the weak point of rubric-based RL: under a weighted sum, criteria compensate for each other, so a policy that misses the one decision that matters can buy points back with unsolicited advice. On clinical consultation, such a policy scores higher yet answers worse—rubric coverage rises while appropriateness on held-out physician criteria falls below the untrained model.
Grouping the same medical criteria so they must hold together recovers a third of the loss; shorter answers recover almost none.
They propose ProRubric, which groups checklists into a few protocol-level dimensions that count only when all criteria hold and no failure clause fires. Grouping is done once offline, leaving the optimizer unchanged. ProRubric raises appropriateness by 10.8 points without losing coverage and achieves the best seven-benchmark average at both scales. Code is open-sourced.
More from Research
- Nvidia's TacitVLA cuts reasoning VLA latency up to 3.4x with no performance loss — nvidia · 2026-10-14
- JevGPT: dual-system autonomous research agent matches rivals faster and cheaper — Yasheng Sun · 2026-10-14
- Neuro for AI & AI for Neuro workshop at AAAI 2027 opens call for papers in Montreal — naturecomputes · 2026-10-14
- BVB Benchmark Tests Agent Video Understanding by Rebuilding 288 Real Videos in Blender — SonglinYang4 · 2026-10-14
- NSF awards $20M to AI Institute in Dynamic Systems for robotic manufacturing — eigensteve · 2026-10-14
- Should ICLR slash its acceptance rate to ~10% as submissions surge? — boknilev · 2026-10-14