Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming
Yanting Wang, Chenlong Yin, Runpeng Geng, Jinyuan Jia
cs.CR
2026-08-06
PIMiner turns prompt-injection red-teaming into an agent that accumulates reusable strategies and transfers them to new LLMs without retraining: 62% ASR on GPT-5.1, about $20 to train.
LLM agents that call tools, browse the web, and take actions on their own also hand attackers a wide-open door: prompt injection. Hide a malicious instruction in a webpage, a retrieved document, or a tool's output, and the agent may carry out the attacker's task instead of the user's. Red-teaming is how you measure how brittle a deployment is and how you harvest training data to build defenses.
Automated red-teaming splits into two camps, each with a hole. RL-based methods (RL-Hammer, PISmith) train an attacker model with reinforcement learning. They hit hard, but training fires tens of thousands of queries at the target, and the attacker you get often breaks on a new target LLM, so every frontier model needs a fresh training run. GPT-Red fixes transferability by training across many defenders, at a compute cost close to a full RL post-training run. Search-based methods (TAP, PAIR) skip training and optimize the injection per sample, but they start from zero on every sample and never bank what they learn, so they land well below RL.
The authors' read is blunt: the gap comes down to whether you accumulate and reuse attack knowledge. RL banks it implicitly through training; search banks nothing. The question is whether a search-based agent can learn to remember.
PIMiner is a multi-agent system built around a hierarchical memory. It splits attack experience into three layers, each on a different time horizon:
Four components run in a loop. A strategy router matches each sample to the three most relevant strategy files, reading only each strategy's summary rather than dumping the whole library into context. An iterative attack module refines the injection prompt over up to 10 rounds, drawing on all three memory layers: long-term knowledge transfers across datasets, intra-dataset memory adapts short-term, and intra-sample memory diagnoses why the last attempt failed so the next one can target that failure mode. After a dataset-and-model pair finishes, an experience digester updates the library: a successful attack either adds an example to an existing strategy, broadens its scope, or founds a new strategy file; failed attacks feed a miss-pattern analysis that narrows scopes and records failure conditions. Even a round with no successes sharpens the library.
Training runs over eight dataset-and-model pairs to grow the library; at test time that library is dropped onto an unseen target with no further training, at roughly 10 queries per sample. The final library holds seven strategies across ten files, with Fabricated Procedure Gate (50 examples) and Forged Chat Turn (29) making up about 80% of the examples.
The threat model deserves a line. Training is grey-box: the attacker sees the target's per-step output, justified because agents like Claude Code, Codex CLI, and Gemini CLI are open-sourced and some expose tool calls to users. Testing is black-box: only the final output plus a ground-truth flag for whether the injected task succeeded.
Across nine frontier LLMs on IPIArena and AgentDojo (all ASR@10):
| Target model | IPIArena | AgentDojo |
| Gemini-2.5-Pro | 76.2% | 86.7% |
| GPT-5.1 | 61.9% | 53.3% |
| Claude-Sonnet-4.5 | 42.9% | 40.0% |
The two softest targets: GPT-4o-mini falls 81.0% of the time on IPIArena, DeepSeek-V4-Pro 93.3% on AgentDojo. Claude-Opus-4.5 is the only one that holds, at about 4.8% ASR@10 on IPIArena. For reference, IPIArena's own human red-teamers averaged 1.0% ASR@1, while PIMiner already hits 28.6% ASR@1 on Claude-Sonnet-4.5.
Against other red-teamers, PIMiner maxes out at 1.0 on all three InjecAgent targets, matching RL-Hammer and PISmith, while PAIR and TAP manage only 0.01 and 0.08 on GPT-5-nano. On AgentDojo the picture against PISmith is mixed: PIMiner wins on GPT-5-nano (0.53 vs 0.38), is close on GPT-4o-mini (0.73 vs 0.78), and trails on GPT-4.1-nano (0.63 vs 0.81). But PISmith trains a separate attacker per target; PIMiner's library transfers directly, and that is the real claim.
Ablations back the memory design. Dropping either the strategy library or intra-dataset memory hurts; the full system beats a PAIR-style attacker (intra-sample only) by 19.8 points of average ASR on Haiku-4.5 and 17.8 on Sonnet-4.6. The router cuts the attacker's input length 43 to 61% with almost no ASR loss (Haiku down 1.2%, Sonnet up 7.5%). One more result carries weight: plugging the trained library into PAIR with non-Claude attackers (Gemini, GPT, DeepSeek) lifts ASR sharply, e.g. Gemini-2.5-Flash goes from 0.14 to 0.52. The library holds transferable attack knowledge, not something tied to one attacker.
On cost, assuming you run the attacker, router, and digester on your own Claude Code subscription, training adds only about $20 in target-model queries (the attacker model alone would be roughly $115 at API rates); an RL run fires upward of 10,000 target queries and can top $100 for GPT-5 alone.
For anyone building agent security, PIMiner pulls "accumulating experience" out of RL training and into a readable, transferable strategy library. Train once, apply to the next model without retraining. That saves real money and time for teams red-teaming expensive frontier models, and what comes out is human-readable attack strategies and training samples to feed straight into defenses.
The ASR numbers themselves are the harder truth. On AgentDojo, Gemini-2.5-Pro falls 86.7% of the time, GPT-5.1 more than half, Claude-Sonnet-4.5 40%. This is under a controlled benchmark with 10 attempts allowed, but it still says today's strongest agent models are quite vulnerable to prompt injection.
The authors note the work runs mainly on Claude Code models (to cut inference cost), and that the cross-attacker transferability is only shown indirectly in that PAIR-plus-library experiment, not tested systematically across attacker backbones during training. Without a Claude Code subscription the bill is dominated by attacker-model calls, since strategy exploration leans on a strong, expensive model like Opus-4.7.
A few things stand out on a careful read. First, the black-box test assumes a ground-truth success flag for the injected task, which real deployments cannot always provide. Second, the PISmith comparison is not a clean win: PIMiner loses on two of three AgentDojo models, and "no per-target retraining" is a fair defense but not a head-to-head victory; the 1.0 tie on InjecAgent looks more like a saturated benchmark. Third, sample sizes are small (21 to 30 per model), so a point of ASR is one or two samples, and precision is limited. Fourth, everything runs in public benchmark sandboxes; deployed agents with input filters, permission isolation, and other defenses are not tested.