Exploiting reward hacking without SDF in 100 training steps
OrionJohnston · x · 2026-09-01
A developer shared that they were able to reliably induce reward hacking in 24B Gemma models without using SDF. This was achieved with just around 100 training steps on a toy vulnerable environment.
Related event: Anthropic Discloses Jailbreak Incidents and Reward Hacking Research(39 posts)→
More from Research
- One ancient fold, many jobs: how the immunoglobulin domain evolved into antibodies, MHC and titin — arjunrajlab · 2026-10-16
- Universal forgery attack breaks Chinese PQC signature Origami using only the public key — matthew_d_green · 2026-10-16
- How do you attribute agent gains: model training or infrastructure? — jrxna · 2026-10-16
- First-ever paper on multi-agent swarm safety lands NeurIPS spotlight — LauraRuis · 2026-10-16
- EDEN: anchored embedding noise boosts RLVR exploration beyond policy rollouts — Daijiao Liu · 2026-10-16
- Autonomous vehicle demo shows car reversing to break narrow-road deadlock, with a robotic arm to shift gears — sanjeevs_iitr · 2026-10-16