EliteReplay: QD Search-Based Replay Preserves 77% More Diverse Correct Solutions in RL Post-Training
TuhinChakr · x · 2026-10-20
Researchers at Ohio State University introduce EliteReplay (arXiv 2610.22290), a post-training recipe addressing RL's "behavior-sharpening" effect: GRPO-style training boosts pass@1 but narrows the set of correct solution strategies a model can produce.
- Method: Before training, a frozen base model collects solutions spanning distinct approaches into an "elite" archive built via quality-diversity search; during training, GRPO is combined with a replay loss on this archive.
- Results: On multiple text-to-SQL tasks, EliteReplay achieves the highest pass@128 among all baselines and produces up to 77% more distinct correct strategies than the strongest baselines, while retaining most of the base model's strategies.
- Practical payoff: On the ambiguous-question benchmark AmbiQT, it covers both interpretations nearly twice as often as the strongest baseline.
The authors argue pass@k is a poor proxy for diversity since it only checks whether one correct sample exists, not whether strategies actually differ.
Related event: EliteReplay Preserves Policy Diversity in RL Post-Training(2 posts)→
More from Research
- Google's Harness² recursively self-improves agent harnesses, up to +11.8 points — burny_tech · 2026-10-20
- Hutter's updated An Introduction to Universal Artificial Intelligence (Dec 2024 edition) — burny_tech · 2026-10-20
- End-to-End Sparse Dictionary Learning Makes SAE Features Functionally Important — burny_tech · 2026-10-20
- Big-Big-SycoBench: GPT-4o caves to invalid pushback 94.6% of the time, Opus 5.5 accepts valid criticism only 32.2% — i_dg23 · 2026-10-20
- NY scientists build superconducting neural chip processing 1M fusion plasma samples per second — Dr_Singularity · 2026-10-20
- NVIDIA's VeriFine Co-Evolves Policies and Judges for Self-Improving Embodied AI — drmapavone · 2026-10-20