Microsoft Research turns LLM judges into coaches and beats rubric-based RL
MicrosoftResearch · hf · 2026-07-21
Core idea
Microsoft Research proposes Experiential Learning (EL) as an alternative to rubric-only RL for open-ended tasks.
Why it matters
- Standard RL with an LLM-as-a-Judge compresses rich textual feedback into a single scalar reward.
- EL reframes the judge as an LLM-as-a-Coach that turns each on-policy response into transferable experiential knowledge.
- That knowledge conditions a teacher model and is internalized by the policy via on-policy context distillation.
Findings
- Compared with scalar rewards, EL provides denser supervision and preserves fine-grained preferences among high-quality responses.
- Across two policy families, and using feedback from either the policy itself or a proprietary model, EL consistently beats rubric-based RL on held-out and unseen open-ended tasks.
- It generalizes better beyond the training distribution and reduces reward hacking.
More from Research
- Soofi S 30B-A3B releases a full pretraining report and claims open-model leads in English and German — abursuc · 2026-07-21
- Practical rolling-shutter pose estimation uses affine correspondences — ducha_aiki · 2026-07-21
- LingBot-Video: 30B Params with 3B Active for Embodied Video AI — alifcoder · 2026-07-21
- Open LatentMoE PyTorch implementation lowers the barrier to testing a new MoE design — KyeGomezB · 2026-07-21
- Graph engineering for agents: why a simple loop breaks down — alex_verem · 2026-07-21
- New PhD thesis traces reinforcement learning from algorithms to foundation models — chaumian · 2026-07-21