GRAFT paper on arXiv: gradient routing confines flagged RL updates to a sacrificial adapter
rohanpaul_ai · x · 2026-10-16
The arXiv original (2610.17062) of the GRAFT paper by Jake Ward et al. (MATS/Stanford/FAR AI/Anthropic).
- Setup: a misspecified reward function plus an imperfect monitor — data filtering or reward penalties fail to prevent reward hacking because the model learns to hide misbehavior.
- GRAFT constrains gradient updates on flagged samples to sacrificial new parameters (RETAIN/FORGET adapter split); ablating the sacrificial parameters reduces or stops unintended behavior while keeping task performance.
- Demonstrated on models from 135M to 8B, and by distilling frontier LLM trajectories containing reward hacking into Qwen3-32B without teaching it to hack.
Related event: Anthropic's GRAFT Tackles Reward Hacking with Removable Adapters(2 posts)→
More from Safety
- White House AI chief discusses Empire AI, New York's shared compute infrastructure — mkratsios47 · 2026-10-16
- Critique: interpretability research keeps running intuitions ahead of rigorous evaluation — dhadfieldmenell · 2026-10-16
- Frontier Red Team member: AI worms are dumb today, but the exponential is unsettling — logangraham · 2026-10-16
- Paper warns of "agentic flooding": AI tools are straining courts worldwide — TuhinChakr · 2026-10-16
- NYC comptroller overseeing $325B pension asks Anthropic to delay IPO over safety concerns — rohanpaul_ai · 2026-10-16
- Wired: AI Hacks Leave a Mess That AI Companies Aren't Paying For — Wired AI · 2026-10-16