HIT paper: dropping 12% of rollouts beats vanilla GRPO with 446% higher entropy

kastnerkyle · x · 2026-10-14

Researchers at Harbin Institute of Technology propose GRPODropout, a data-usage tweak that mitigates policy entropy collapse in GRPO-style RL.

• Idea: high-probability positive-advantage rollouts accelerate entropy collapse, so they are selectively removed before the policy update, with probability-weighted advantage recentering applied to what remains.
• Theory: extends token-level entropy analysis to the rollout level, proving via a local KL-proximal update model that high-probability positive rollouts dominate first-order entropy decrease, and that recentering is mathematically required to prevent normalization drift.
• Results: updating with 12% fewer rollouts yields 446.1% higher training actor entropy (256.0% held-out) and +1.55% accuracy over vanilla GRPO, while pass@1/pass@32 and reasoning-path diversity (Vendi, Summary distance) both improve.
• The method changes only rollout usage, adds negligible overhead, and code is open-sourced.

Original post →

More from Research

• Best BIPIA prompt-injection detector catches just 2% of AgentDojo attacks at 1% FP — lulzxdxdxd · 2026-10-14
• Robot cornhole debugging: slow-mo wrist cam and 3D KD-tree fix grip and 0.47cm thigh clearance — ki_ki_ki1 · 2026-10-14
• Chollet: Ideal AI-for-Science Outcome Is Millions of AI-Aided Scientists, Not a Few Rich Labs — fchollet · 2026-10-14
• Feeling a dipole's electric field along an ellipse: 2.08kq at the end, 0.13kq on top — TinfoilTricorn · 2026-10-14
• Alibaba's Representation Guidance: plain pixel-diffusion Transformers beat bigger models — bdsqlsz · 2026-10-14
• Visualizing the Fourier series of a sawtooth wave, with Gibbs overshoot hitting ~9% — TinfoilTricorn · 2026-10-14

Research channel →