Meta proposes reward transformation exploiting GRPO rollouts to equalize subgroups
nagpalchirag · x · 2026-10-20
Method details from Meta's "Aligning Language Models towards Counterfactual Invariance" thread: the team proposes a reward transformation technique that exploits the multi-rollout structure of GRPO to quantify reward discrepancy between counterfactual subgroups, plus a custom leave-one-out reward designed to match reward expectations across subgroups in L2 distance.
Related event: Meta study finds RL post-training amplifies counterfactual bias in LLMs(3 posts)→
More from Research
- TerraVis at NeurIPS 2026: New Benchmark Flags Physically Wrong AI Images That Score High — jiqizhixin · 2026-10-20
- New MULTIFINGROUND Benchmark Shows LLMs Recognize Financial Concepts but Fail at Full Evidence Extraction — TheFinAI · 2026-10-20
- EliteReplay: QD Search-Based Replay Preserves 77% More Diverse Correct Solutions in RL Post-Training — TuhinChakr · 2026-10-20
- EliteReplay Preprint: Replay Buffer of Diverse Strategies Boosts RL Post-Training Diversity — TuhinChakr · 2026-10-20
- Video Geometry Corrector fixes 3D distortions in AI-generated videos, NeurIPS paper — kwangmoo_yi · 2026-10-20
- Ex-GitHub CEO Sid Sijbrandij open-sources 125,091 radiology images and multi-omics from his cancer journey — OgundipeOre · 2026-10-20