RL-XAR recipe: replace LLM judges with expert-aligned rubrics for RL
ivan_bezdomny · x · 2026-09-28
jaseweston shares the RL-XAR (RL with eXpert Aligned Rubrics) recipe:
- Collect high-quality human-written texts
- Learn LLM judgments via rubrics that score expert texts higher than model generations
- RL on the learned rubrics
Iterate until meta-optimization of the rubrics finds no discernible gap. Optimized rubrics shift from preferring current 'slop' models to preferring humans.
Quoter ivanbezdomny adds he took a similar route: instead of LLM judges (whose flaws he's long flagged), learn better intermediate values via a process and have the LLM optimize for those.
Related event: Meta's RL-XAR Uses Expert-Aligned Rubrics to Fix AI Slop(4 posts)→
More from Research
- Continual-ARC: lifelong ARC-AGI stream finds agent harness persistence rarely beats simple caching — unifyai · 2026-11-16
- DySENSE: MIT's GenAI framework synthesizes satellite imagery with dynamic city energy maps — MIT · 2026-11-16
- WebArena-Pro: a 300-task multimodal benchmark across 20 self-hosted apps for web agents — McGill-NLP · 2026-11-16
- Training-free page-sparse decode in llama.cpp: 1.72× faster 128k decoding on an RTX 5090 — topppggg · 2026-11-16
- Training Qwen3-8B to drive a simulated hand spawns a dedicated internal 'pressure' direction — repligate · 2026-11-16
- Google Research open-sources PaperOrchestra, a multi-agent framework for writing AI papers — google-research · 2026-11-16