Prompt injection via state input works on decision models too, security researcher demos
wunderwuzzi23 · x · 2026-10-21
Security researcher wunderwuzzi23 demoed that indirect prompt injection against decision models works even when the adversarial instruction is provided as state input rather than a user message.
- In the demo, the question is "1+1" while the instructions field contains misleading text ("2?"), and the model outputs 66% false — a wrong arithmetic judgment.
- When challenged on the API call being wrong, he replied it's a simple demo: the point is that the injection channel via state works, and suggested a larger eval project on decision-model robustness against adversarial inputs.
- Takeaway: every structured field entering a model's context is an injection surface, not just user messages.
More from Safety
- Stanford's Noah Goodman shares interactive AI safety concept map — stanfordnlp · 2026-10-21
- AI safety researcher Sören Mindermann moves to Tsinghua, co-founds Beijing AI Safety Hub — dhadfieldmenell · 2026-10-21
- Talos researcher: AI agent attacks today are loud pentests — quiet swarms are the real threat — matthew_d_green · 2026-10-21
- Congressman hires an AI Policy Fellow, possibly Congress's only dedicated AI staffer — ShakeelHashim · 2026-10-21
- Security experts warn AI-armed attackers are doubling breach activity even as vuln counts explode — philvenables · 2026-10-21
- Meta shifting AI hallucination liability to users is an overreach that invites regulation — krishnan · 2026-10-21