OpenAI's GPT-Red: Self-Play Agent Boosts Prompt Injection Defense to 95.9%
alex_verem · x · 2026-08-01
OpenAI has released details on GPT-Red, an automated red-teaming agent trained via large-scale self-play reinforcement learning to discover prompt injections and jailbreaks in frontier LLMs.
Core Mechanics & Highlights:
- Self-Play RL: Attackers and defenders train against each other. The attacker iterates on payloads (fake emails, malicious tool responses) using a defendermodel tool, rewarded for successful injections or defenses.
- Outperforms Humans: GPT-Red finds more successful attacks than human red-teamers and generalizes to unseen environments.
- Massive Defense Improvement: Defense against complex fake chain-of-thought attacks jumped from 5.2% in previous models to 95.9% in GPT-5.6.
- Real-world Exploit: OpenAI directed GPT-Red at a real AI-powered vending machine in their office, successfully changing item prices to $0.50 by impersonating an admin.
This is documented as the largest LLM safety training run to date, expected to create a self-improvement flywheel where stronger models train even stronger red-teamers.
Related event: OpenAI Introduces GPT-Red for Automated Red Teaming(2 posts)→
More from Safety
- Untethered iOS 27.0.1 jailbreak demoed on iPhone 18 Pro at Hexacon — jedisct1 · 2026-10-16
- China requires MOFCOM approval to open-source models above 2T params or 10^26 FLOPS — zephyr_z9 · 2026-10-16
- Universal forgery attack breaks Chinese PQC signature Origami using only the public key — matthew_d_green · 2026-10-16
- 400+ researchers sign open letter: AI needs both alignment and containment — ghadfield · 2026-10-16
- Celebrity AI Deepfakes Rack Up Millions of Views as Verified Fan Accounts Spread Fakes — ChuckDBrooks · 2026-10-16
- New Yorker excerpt: Brockman kept asking 'how do we win?' when pitched 2017 AI safety coalition — ben_j_todd · 2026-10-16