CriPO: Self-Distillation RL Method Halves Optimization Steps

Mingxuan Xia · hf · 2026-08-03

Current rubric-based reinforcement learning (RL) for LLMs suffers from limited exploration. The research highlights a failure mode termed "Suppressed Criteria"—criteria that are met but whose learning signals are lost during optimization. Data shows over 57% of samples exhibit this issue.

To address this, the authors propose CriPO (Criterion-Distilled Policy Optimization). CriPO uses on-policy self-distillation to tackle both unexplored and suppressed criteria without introducing train-inference mismatch.

Experiments on medicine and science benchmarks demonstrate that CriPO outperforms existing rubric-based RL methods, achieving stronger final performance with approximately 2x fewer optimization steps.

Original post →

More from Research

• Don't Trust Benchmarks Blindly: Expert Warns Harness Discrepancies Skew LLM Scores — cedric_chee · 2026-08-03
• Human Bindome: Open-Sourcing Protein Binder Candidates for Every Human Protein — jajoosam · 2026-08-03
• Transition-Factorized LAM Paper Updated with New Experiments — ceciletamura · 2026-08-03
• Discussing Hallucination Risks and Limitations of LLMs in Bibliography Retrieval — deliprao · 2026-08-03
• AI Independently Derives General Relativity and Solves Decades-Old Math Problems — elonmusk · 2026-08-03
• Microsoft Exec Laments Top AI Conference Crisis: NeurIPS Paper Quality Drops — MParakhin · 2026-08-03

Research channel →