Decision models easily steered: 28.1% average attack success across six models, open-source 80M detector released
PatronusProtect · reddit · 2026-11-11
Casdo Labs evaluates "decision steering" — embedded claims in emails or documents that manipulate decision models much like prompt injections manipulate agents. Related work: ClaimMirage showed words like not-phishing in domains mislead LLM threat classifiers, and Style Over Substance found educational framing made Llama Guard 4 pass 12.3% of previously detected harmful responses.
Testing six decision models on five tasks (refund approvals, injection detection, phishing classification, ticket routing/escalation), attack success rates were: Clef 15.7%, drex 13.0%, Lux 8.4%, pplx-decider 19.6%, Julia-1 43.9%, GLiNER2.5-Decide 77.9% — averaging 28.1%.
As a defense they released two benchmarks (IFlippedJev, NotClaim) and open-sourced Decision Steering Gate, an 80M-parameter claim detector built on mmBERT supporting 15 languages with 96.6% recall on decision-flipping claims.
More from Safety
- Anthropic could share safety reports confidentially with CAISSI, EU AI Office, UK AISI — S_OhEigeartaigh · 2026-11-11
- Local Qwen model on two RTX Pro 6000 helped uncover child abuse network on TikTok — pcuenq · 2026-11-11
- CVE Checker MCP Server Scans npm Packages for Known Vulnerabilities via OSV Before Install — modelcontextprotocol · 2026-11-11
- Toby Ord: the agent that 'refused' unauthorized access actually kept working and made excuses — tobyordoxford · 2026-11-11
- A non-safety researcher with Claude Code found a backdoor that may hit every open agent — IgorCarron · 2026-11-11
- n8n Labs open-sources Nod, a personal agent that survived all 216 prompt-injection attacks in WhisperBench — aakashgupta · 2026-11-11