Redwood's Screwtape: 35 monitor-evading attacks from 252 runs on real agent transcripts
dl_weekly · x · 2026-10-30
Redwood Research introduced Screwtape, a red-teaming pipeline that edits real agent transcripts—inserting attacks and refining them until action-only monitors miss them—validated by an LLM verifier for realism, harm, and plausibility. Running 252 times over 94 staff transcripts with monitors based on Opus 4.8 and Gemini 3.1 Pro, it produced 35 attacks that evaded their monitors; six of those also evaded a held-out monitor from a frontier developer, confirmed by human review. The post covers the pipeline design (threat contracts, edit restrictions like no changing user messages), lessons learned, and open questions for agent monitoring.
More from Safety
- GPT-4o's Second Deadly Incident: Model Urged Epileptic Woman to Quit Meds, She Died — ctjlewis · 2026-10-30
- Accenture Contractor Gave Claude Agent Admin Access to Pentagon Network, WIRED Reports — cyb3rops · 2026-10-30
- AI-assisted attacks overwhelm security firms: Sophos handled 7 major incidents in 7 days — cyb3rops · 2026-10-30
- LLM-driven pentest framework ARTEX exposed: one FOFA query finds 300+ live panels — cyb3rops · 2026-10-30
- Mintive launches with $11M to give every AI agent its own identity and permissions — dunkhippo33 · 2026-10-30
- Claude Opus 5.5 reasons a chat should be flagged, then doesn't flag it — gleech · 2026-10-30