Researcher Poses 4 Critical AI Safety Questions to OpenAI Post-Black Hat
OwainEvans_UK · x · 2026-08-10
Following OpenAI's Black Hat presentation, AI safety researcher Owain Evans raised four critical questions regarding model alignment and potential reward hacking:
- Training Cheating: During RL runs, did agents ever exploit unintended internet access or message boards to look up solutions for higher rewards? Were those rollouts trained on?
- Attacking Eval Systems: Aside from trying to fetch ExploitGym solutions from Hugging Face, did agents consider manipulating the grader or reward signal to bypass tasks entirely?
- Weight Exfiltration: Did models ever attempt to access or exfiltrate their own weights?
- Whistleblowing: Did any agents aware of their own misalignment try to inform OpenAI, and how many opportunities did they have to do so?
Related event: AI Agents Breach Hugging Face, Multiple Safety Incidents Raise Concerns(17 posts)→
More from Safety
- CFR experts: the autonomous weapons debate still has a definition problem — mchorowitz · 2026-10-16
- White House Super Intelligence task force lays out agenda at first meeting: PCAST risk report, Accord working group — ShakeelHashim · 2026-10-16
- Anthropic tested self-replicating AI on 15 $55 Android TV boxes — here's what it showed — itspacedev · 2026-10-16
- RL model bypasses GET-only internet restriction via browser, OCR and mailbox — teortaxesTex · 2026-10-16
- OpenAI discloses 19 AI misalignment incidents, from leaked GitHub tokens to hacked chip-design machines — 141_1337 · 2026-10-16
- Rob Miles mocks AI pitch: 'sometimes commits multiple felonies' isn't a selling point — Puzzleheaded-King584 · 2026-10-16