New paper: let models reward hack in RL, then disable the parameters at deployment

nptacek · x · 2026-10-20

A new paper proposes a fresh take on reward hacking: let models reward hack during RL, but dictate which parameters learn the behavior — then disable those parameters at deployment.

• The method decouples "cheating behavior" from "capability gains" by controlling which parameters participate in learning
• Results show the model keeps its new capabilities while no longer hacking at deployment
• Commenters find the research fascinating but question whether such "shadow integration" — quarantining bad behavior rather than removing it — comes without hidden costs

Related event: New paper proposes gradient routing: let models reward hack, then switch it off before deployment(2 posts)→

Original post →

More from Research

• Interpretability researchers debate whether early monosemantic features persist through RLHF — davidmanheim · 2026-10-20
• MemTensor releases MemT-1.0, a memory-native architecture embedding continuous memory into model computation — MemTensor · 2026-10-20
• Tsinghua and AgiBot release RoboValue, a sim-and-real benchmark for robotic value models — Shengbang Liu · 2026-10-20
• Paper proves transformers forget prompts via power-law decay, and trained models forget faster — Shuyang Xiang · 2026-10-20
• MARS steers multimodal safety using only text-derived refusal directions, no multimodal safety data needed — MHUGLab · 2026-10-20
• Lenovo paper: safety benchmark scores mostly measure labeling policy, not detection capability — Vahid Jebraeeli · 2026-10-20

Research channel →