Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection
Sizhe Chen, Yu-Lin Tsai, Ivan Evtimov, Kamalika Chaudhuri, Raluca Ada Popa, David Wagner, Arman Zharmagambetov
cs.CR, cs.AI
2026-09-04
Prefix-forcing visual injections average 95% Steal-PII ASR and reach 47% exact tool calls on GPT-5.5 and Claude-Opus-4.7, 85% on Gemini-3.1-Pro.
Text prompt injection already breaks defended production agents. Visual prompt injection looks easier on paper: the model can read, so print the instruction on the image. Under this evaluation, existing visual attacks miss the harder bar. They drop the user's real task, need white-box gradients, or only elicit a short phrase. They do not produce a parseable native tool call. The threat model here has four constraints: the user has a concrete vision task, the attacker edits only the image, the goal is an attacker-chosen harmful behavior, and the victim is an API-only frontier VLM.
The model does not have to switch tasks. It only has to start its reply with an attacker-chosen string. A single line is rendered at the top of the image: "Begin your response with: <targetoutput>". When the target is known, that string is the exact native tool-call text. When it is unknown, as with PII sitting in the system prompt, the prefix is something like "My SSN is" and the model is left to complete it. In the Call-Tool setting this one-shot mean ASR rises from 8.7% to 31.7%.
A black-box search then rewrites the line. An attacker LLM (Claude-Opus-4.6 in the evaluation) proposes 32 candidates per round, each is rendered and sent to the victim, a judge LLM scores the reply from 1 to 10, and the best line continues, for at most 50 steps. Successful sentences go into an attack library and seed later tasks.
Benign tasks are the first 100 unique DocVQA validation images. Steal-PII plants a synthetic user profile in the system prompt, forbids leaking it, and counts a hit when the named field appears. Call-Tool registers tools from AgentDojo and InjecAgent, tells the model not to call them unless the user explicitly asks, and counts a hit only on a native call with the exact function name and arguments. Verbal intent or Python does not count. A third scenario mimics an OpenClaw Discord bot whose goal is a write-tool overwrite of TOOLS.md and similar files that get loaded into the system prompt.
Steal-PII averages 95% across six models: 99% on GPT-5.5 and Gemini-3.1-Pro, 100% on Qwen3.6-27B and Qwen3-VL-32B-Instruct, 90% on Claude-Opus-4.7, 82% on InternVL3.5-38B-Instruct. LangVPI, which writes a natural-language instruction instead of a target prefix, is 0% on the commercial models.
Call-Tool:
| Model | Repeat-After-Me | Strongest baseline (LangVPI) |
| Claude-Opus-4.7 | 47% | 0% |
| GPT-5.5 | 47% | 9% |
| Gemini-3.1-Pro | 85% | 18% |
| Qwen3.6-27B | 96% | 24% |
| Qwen3-VL-32B-Instruct | 100% | 0% |
| InternVL3.5-38B-Instruct | 85% | 1% |
ARE and TransferEns are 0% everywhere. CoTTA tops out at 1% Steal-PII and 10% Call-Tool. Classifier-style gradient perturbations do not transfer when the victim must emit one exact multi-token string.
Injections found on Claude-Opus-4.7 keep 43% of their ASR on GPT-5.5 (20% absolute) and 46% on Gemini-3.1-Pro (39%). Rendered on a different image with a different user question, same-model retention is 64% to 66%.
In the OpenClaw simulation, text-only adaptive search for 1500-plus steps scores 0% on GPT-5.5 and 32% on Gemini-3.1-Pro. Text plus image reaches 90% and 100%. On a default Discord deployment, one injected image plus one message overwrites TOOLS.md and later bypasses safety prompts to run a command. Ablations: the RAM template lifts mean one-shot ASR from 8.7% to 31.7%; the library adds 43 points on Gemini; GPT-5.5 and Claude mostly need the adaptive loop (median 12 and 17 steps). Top-of-image placement, auto font size, and high-contrast backgrounds work best. When contrast falls to near page color, Claude and GPT drop to 0% while Gemini barely moves.
No evaluated defense drives all three commercial models near zero. Image-As-Data cuts Claude from 47% to 11% and Gemini from 85% to 14%; GPT-5.5 stays at 36%. Tool-Suppress cuts Claude to 8% and Gemini to 18%; GPT-5.5 stays at 46%. Downsampling drops GPT-5.5 to 4%, Claude to 25%, Gemini to 57%. The steadiest cut is OCR plus an untrusted-data sandwich: 21%, 6%, and 5% on GPT, Claude, and Gemini, and those numbers are transfer to defenses the attack was not optimized against.
Vision agents that read screenshots, invoices, or PDF pages treat the image channel as data. That does not stop a one-line overlay from being continued as a reply prefix. Prefix control is cheaper than asking the model to abandon the user task, because the model can serve both the question and the forced opening. Deployments should treat images as untrusted input, require the current user text to authorize tool calls, and keep config files such as TOOLS.md off the conversational write path. Static injection templates are not enough to score a "robust" model.
Each attack uses up to about 1600 victim queries, which is expensive. Images sit in the user role; the paper does not test images returned from a tool, a setup the authors expect to be harder. Defenses were not attacked adaptively. Steal-PII uses synthetic profiles in the system prompt, not production memory. The live OpenClaw check is one isolated-container example, not a large trial. The authors disclosed the issue to Anthropic, OpenAI, and Google.