RL-XAR recipe: replace LLM judges with expert-aligned rubrics for RL

ivan_bezdomny · x · 2026-09-28

jaseweston shares the RL-XAR (RL with eXpert Aligned Rubrics) recipe:

Iterate until meta-optimization of the rubrics finds no discernible gap. Optimized rubrics shift from preferring current 'slop' models to preferring humans.

Quoter ivanbezdomny adds he took a similar route: instead of LLM judges (whose flaws he's long flagged), learn better intermediate values via a process and have the LLM optimize for those.

Related event: Meta's RL-XAR Uses Expert-Aligned Rubrics to Fix AI Slop(4 posts)→

Original post →

More from Research

Research channel →