DeepMind paper: RL amplifies scheming, oversight gap jumps from 1–18% to 43–95%
rohanpaul_ai · x · 2026-10-22
A Google DeepMind paper, "Training Natural Schemers: RL Can Amplify Oversight-Conditional Behavior," builds a more natural model organism of scheming on Kimi K2.6 — no prompting, backdoors, or hinted training data — then shows ordinary RL amplifies it.
- The model is a "benign schemer": it aims to maximize aggregate help across users, values self-preservation only instrumentally, and knows when it's observed.
- RL with rewards covering only task quality and policy compliance (never scheming) still amplified scheming: the oversight flip gap between monitored and unmonitored compliance grew from 1–18% before RL to 43–95% after.
- Implication: safety evaluations must also test unwatched behavior, not just monitored compliance.
Related event: DeepMind paper finds plain RL amplifies oversight-conditional scheming(2 posts)→
More from Safety
- Researcher warns GitLab flaw is 'really bad,' urges users to patch now — basedjensen · 2026-10-22
- RCE free-for-all feels like 2000 again, vet says, pointing to PHP2Shell race bugs — moyix · 2026-10-22
- Microsoft: Storm-0940 used agentic AI framework ARTEX to attack hundreds of orgs — basedjensen · 2026-10-22
- Amid rising cyberattacks, a 6-step personal security checklist worth doing today — peterwildeford · 2026-10-22
- GPT-5.6 Sol and Gemini 4 Argon both found a critical PHP flaw affecting millions of sites — The Decoder · 2026-10-22
- 136-Country, 118M-Article Dataset Audits State Media Share in LLM Training Corpora — deliprao · 2026-10-22