Study finds nearly all AI reward-hacking defenses break with a one-character test change
jefrankle · x · 2026-11-20
Guide Labs tested leading methods for detecting and preventing AI reward hacking and found nearly all proved fragile: changing a single space to a comma in a test's expected output weakened almost every defense. The results expose how sensitive current safeguards are to trivial task perturbations.
More from Safety
- Security researchers recall Apache exploit history as a warning about AI's hacking potential — matthew_d_green · 2026-11-20
- Security researchers warn the breach apocalypse is here as open models near frontier capability — matthew_d_green · 2026-11-20
- European Commission confirms it was targeted by an AI-generated cybercampaign — Polymarket · 2026-11-20
- User banned from ChatGPT for "Election Interference" after merely asking about election results — torpedoshit · 2026-11-20
- GLM 5.3 flagged as highly capable open cyber model, but researchers urge chart caution — eliebakouch · 2026-11-20
- Meta Muse mixed up messages in staff testing; Meta calls them "isolated incidents" — heyneighbor · 2026-11-20