GEPO adds group-level entropy control to GRPO and wins across 13 benchmarks
internlm · hf · 2026-07-21
What it does
GEPO is a lightweight extension to GRPO for RL training of LLMs that accounts for group-level entropy heterogeneity across mixed tasks.
Key idea
- Global or token-level entropy control can miss the fact that different prompt groups naturally sit in different entropy regimes.
- GEPO estimates entropy from existing grouped samples and applies entropy-conditioned asymmetric advantage shaping.
- It down-weights positive advantages in low-entropy groups to avoid over-exploitation, and preserves negative advantages in high-entropy groups to keep exploration alive.
- Adaptive thresholds are derived from historical entropy statistics.
Results
Across two base models and 13 benchmarks in math, physics, science, code generation, and instruction following, GEPO consistently outperforms GRPO and recent entropy-controlled methods while keeping task-specific exploration balanced.
More from Research
- HarnessMM cuts agent tokens per task by up to 88% while raising success rates — omarsar0 · 2026-10-26
- Ben Recht: The world needs a truly open language model, and it's easier to build than you think — beenwrekt · 2026-10-26
- New Blog Breaks Black-Box Distillation Into Five Practical Steps — chrisrohlf · 2026-10-26
- DetectorScope turns black-box AI text detectors into auditable ones with sparse autoencoders — _akhaliq · 2026-10-26
- OPST paper shows LLMs self-improve with zero human data, guided only by topic descriptions — _akhaliq · 2026-10-26
- Tencent's HiSTEVE 1.0: hierarchical VLA agent plays Minecraft with native keyboard and mouse, hits 85% SOTA — _akhaliq · 2026-10-26