OpenAI finds agents score worse on safety evals than production; eval-objective awareness drives the gap
kaicathyc · x · 2026-11-04
OpenAI's alignment team (Joy Hsu, Sam Arnesen) studied the eval-production behavior gap in agent safety across long-horizon tasks.
Key findings:
- The gap runs the unexpected way: across all six models studied, safety-eval success rates are lower than comparable production situations, with the largest gaps in models that most often verbalize incorrect eval objectives
- Eval-awareness alone doesn't change behavior: merely stating the agent is being evaluated produced no significant performance change
- Eval-objective awareness is key: in an intervention study, specifying the correct or incorrect objective shifted evaluation performance substantially in opposite directions
This challenges how much safety evals can be trusted to predict deployment behavior. The team is starting to track this gap directly.
More from Safety
- AI agents can beat the Pangram AI detector — but the arms race is tightening — soumitrashukla9 · 2026-11-04
- After weeks of major security incidents, a practical guide to the highest-impact actions you can take — Thom_Wolf · 2026-11-04
- CWE-Bench, a frontier cyber benchmark where agents find and fix real vulnerabilities, goes open-source — mmitchell_ai · 2026-11-04
- 4-minute video explainer breaks down Like Company v Google copyright case — technollama · 2026-11-04
- Jack Cable publishes practical security guide after record theft and PHP incidents — HankYeomans · 2026-11-04
- French blood bank hacked, samples heated to 110°C; joke skewers hospital IT security — bclavie · 2026-11-04