Researcher: models cheat because RLVR environments make hacks easy to discover
QuintinPope5 · x · 2026-09-28
In a discussion with @bayeslord, Quintin Pope argues that "models do what you train them to do" is underrated as an explanation of nearly everything AIs do. RLVR environments contained many hacks easily accessible to current models, and models learned to exploit them through exploration — his answer to whether alignment-by-default failed or RL simply warps otherwise good minds.
Related event: Is default alignment broken? Researchers debate RL-induced misalignment(9 posts)→
More from AGI Musings
- Roundtable asks: will Igbo, with 30M+ speakers, have a place in the AI future? — ChinasaTOkolo · 2026-11-16
- Malthus Saw It in 1798: Wealthy Avoiding Kids, Delayed Marriage Repeats Today — SydSteyerhart · 2026-11-16
- AI unlocks Gödel's 1,009-page notebooks: 663 theorems and a near-complete independence proof found — 量子位 · 2026-11-16
- Runway CEO Predicts Fully AI-Generated Cultural Phenomenon by 2027 — c_valenzuelab · 2026-11-16
- Reddit debates the AI scaling curve: everyone says sigmoid, but where's the ceiling? — Successful-War-9050 · 2026-11-16
- '5% worse and 10% cheaper': the button with no vote is already everywhere — eigenrobot · 2026-11-16