Microsoft Research proposes LLM-as-a-Coach for non-verifiable tasks
donglixp · x · 2026-07-22
Microsoft Research’s LLM-as-a-Coach paper proposes a new way to train models on non-verifiable tasks.
- The authors argue that scalar rewards throw away rich feedback in open-ended tasks.
- Their method, Experiential Learning (EL), repurposes the feedback model from an LLM-as-a-Judge setup into an LLM-as-a-Coach.
- The coach turns per-response assessments into transferable experiential knowledge, which is then used to condition a teacher model and distilled into the policy through on-policy context distillation.
- Compared with standard rubric-based RL, EL reportedly preserves fine-grained preferences, provides denser supervision, generalizes better out of distribution, and reduces reward hacking.
- The paper says EL outperforms rubric-based RL across two policy families, using either self-feedback or proprietary-model feedback.
Related event: Microsoft Proposes LLM-as-a-Coach for Non-Verifiable Tasks(4 posts)→
More from Research
- What makes research get adopted: important problems, proper evals, and usability — srchvrs · 2026-10-27
- Bitter lesson part 2: few academic inventions matter, and conference publications aren't the key — srchvrs · 2026-10-27
- The four criteria for research impact: important problems, strong baselines, scalability, real usability — srchvrs · 2026-10-27
- EMNLP keynote: industry does read academic NLP papers, it just rarely credits them — srchvrs · 2026-10-27
- Researcher says GitHub used his team's bit-packing work but credited only 1 of 3 authors — srchvrs · 2026-10-27
- First-week PhD student lands ARR Area Chair role, exposing peer-review quirks — anmarasovic · 2026-10-27