RL alignment amplifies counterfactual bias, group-level reward fix shows new paper
nagpalchirag · x · 2026-10-20
A new arXiv paper finds that RL post-training boosts reasoning and instruction following but widens quality gaps across demographics and prompt categories, amplifying counterfactual bias. The authors propose a policy-gradient method with a group-level reward over matched counterfactual prompt pairs, plus a controlled dataset and evaluation separating causally relevant demographic info. Fairness benchmarks show reduced group disparities without hurting overall response quality.
Related event: Meta paper: GRPO post-training unevenly boosts subgroups, proposes fix(3 posts)→
More from Safety
- Researchers pull real leaked API keys from LLMs using only API output — nordicinst · 2026-10-20
- Microsoft says AI agents should be treated like insider threats as 'frontier models can't be trusted' — TechCrunch AI · 2026-10-20
- Studios Want AI to Mint New Franchises, but Pure AI Content Can't Be Copyrighted — WhatTheLJW · 2026-10-20
- French AI-generated novel axed from Goncourt longlist; writers demand AI text watermarking — zetalyrae · 2026-10-20
- NYC to take 15% of QFab IP proceeds and three board seats in landmark startup deal — MJBiercuk · 2026-10-20
- Fake Anthropic employee accounts run a novel X follow-based phishing scheme — edgarpavlovsky · 2026-10-20