EVALUESTEER: best reward models fail to align with user values over 25% of the time
MaartenSap · x · 2026-10-25
EVALUESTEER, a new paper by Kshitish Ghate and Maarten Sap (ICLR poster Oct 26), measures whether reward models can be steered toward user-specific value and style preferences. Findings: even the best RMs tested exhibit their own value/style biases and fail to align with a user more than 25% of the time, revealing clear limits in reward model steerability.
More from Research
- Safety researcher: verification should go all the way down to Maxwell's equations, not stop at assembly — JacquesThibs · 2026-10-25
- Local 35B authored 800 math problems under cloud-agent supervision — authoring erred 5x more than cold-solving — Individual_Page9676 · 2026-10-25
- SA-LapNet fixes spin exactly in neural-network quantum Monte Carlo, breaking the purity-energy trade-off — bravo_abad · 2026-10-25
- XYBench: Strongest LLMs Answer Literally and Detect Under 63% of User Misconceptions — MaartenSap · 2026-10-25
- Multi-domain training mix boosts supervised hallucination detectors 7.25% on out-of-distribution QA — MaartenSap · 2026-10-25
- Ouvia study: only about half of real-world speech translation interactions rated usable — MaartenSap · 2026-10-25