Agent red teaming isn't LLM red teaming: four failure modes jailbreak tests miss

Plastic_Secret1303 · reddit · 2026-11-09

A substantive Reddit discussion argues that most red-teaming targets the model (jailbreaks, harmful content, prompt injection on chat UIs), but agents fail in ways that need their own methods.

Four failure modes jailbreak tests don't catch:

Multi-turn drift: each message looks fine, but the conversation as a whole steers the agent somewhere it shouldn't go
Planted context: malicious content hidden in retrieved documents or tool outputs, not the user message
Authorization gaps: the agent acts on orders, accounts, or records that don't belong to the user
Over-correction (the most under-discussed): after hardening, legitimate requests start failing — a safe agent that refuses valid refunds is still broken, and safety tests rarely check this

The author asks whether published work exists on adaptive red teaming, where attack strategy adjusts based on the agent's responses instead of running a fixed attack list, and how to measure coverage for an open-ended action space.

Original post →

Agent red teaming isn't LLM red teaming: four failure modes jailbreak tests miss

More from coding & agent