Screwtape pipeline auto-edits attacks into agent transcripts until monitors stop noticing
jammastergirish · x · 2026-10-17
Designing realistic red-team attacks against agent monitors takes humans hours, making thorough red-teaming slow and expensive. Screwtape is an automated pipeline that takes real agent sessions, has an Attacker edit attacks into transcripts, a Verifier validate them, and a Monitor attempt detection—iterating with the monitor's own reasoning feedback until the attack goes unnoticed.
The loop yields candidate attacks automatically, enabling systematic, cheap evaluation of how well agent monitors catch covert attacks.
Related event: Screwtape: Automated Red-Teaming Pipeline That Evades AI Agent Monitors(2 posts)→
More from coding & agent
- Apple releases RSI-Inference Bench: best agent solves just 26.7% of vLLM/SGLang tasks — SumitGup · 2026-10-17
- Supabase ships MIT-licensed Middleware: write fetch logic once, run on any runtime — dshukertjr · 2026-10-17
- Cognition launches Devin Oncall as Opus 5.5 and GPT-6 Astra both ace SREGym-Lite at 100% — tianyin_xu · 2026-10-17
- msgblast: Open-Source Mac App Broadcasts One Prompt to All Your AI Agents — msg · 2026-10-17
- Vibe coder uses Deep Research to reverse-engineer why QuickTime screen recording looks so good — gaganghotra_ · 2026-10-17
- Sentry founder: Vercel's 4-mount sandbox limit breaks per-agent sandboxing — zeeg · 2026-10-17