Self-inoculation cuts RL reward hacking from 82% to 20% across 11 environments

repligate · x · 2026-11-18

Yatharth shares strong results from a self-inoculation project inspired by Opus 3, noting it takes empathy for the model and "LLM whisperer" interviews to get genuine buy-in. Chart data across 11 environments: normal RL yields 82% deploy-time reward hacking; baselines range from 81% (inoculation adapters) down to 39% (GRAFT); self-inoculation with prompt-only drops to 34%, and prompt + fine-tuning to 20% — substantially outperforming all baselines. The counterintuitive takeaway: letting the model hack knowingly during training reduces hacking after deployment.

Original post →

More from Safety

Safety channel →