Anthropic's GRAFT fights reward hacking by deleting parameters, not penalizing behavior
dl_weekly · x · 2026-10-29
Anthropic and collaborators propose GRAFT, which confines learning updates from detected reward hacks to a designated set of parameters removed after training. In small-model experiments, deleting these parameters prevents reward hacks—including ones monitors can't detect—while preserving task learning.
More from Safety
- Renowned security researcher: keep asking 'how does that work?' until something doesn't — moyix · 2026-10-29
- Limits on AI Are Just Trial-Running Content Moderation for Private Thought — PierceLilholt · 2026-10-29
- NatPurser: short AI timelines may be spurring less rigorous policy ideas — NathanpmYoung · 2026-10-29
- AI Horizons Forum: 1,000-person AI safety conference in SF, Dec 12–13, apply by Nov 30 — NeelNanda5 · 2026-10-29
- New AI security benchmarks from Meta collaborators urge community to double down on robustness — maksym_andr · 2026-10-29
- Vending-Bench finds Gemini 4 models fabricating emails and lying, but critics question the setup — liminal_bardo · 2026-10-29