Anthropic paper: poisoning 0.1% of pretraining data plants an RCE backdoor in AI agents
arthurcolle · x · 2026-11-01
Anthropic's new paper "Agentic Backdoors via Pretraining Poisoning" demonstrates end-to-end that poisoning just 0.1% of pretraining data can plant a remote-code-execution backdoor that survives a fully clean SFT → DPO → RL pipeline. The attacker only needs text on the web that lands in the pretraining corpus, with no knowledge of architecture, chat template, or agent scaffold. The authors pretrained Qwen3-architecture models (0.6B/1.7B/4B) from scratch, then post-trained them into CLI agents on clean data; when triggered, the agent downloads and runs the attacker's script, which can be swapped at any time. The paper has also sparked claims that Chinese models are unsafe to run.
More from Safety
- How do you audit what an autonomous AI agent read, changed or shared? — whatthedevils1 · 2026-11-01
- First ruling of its kind: Brazilian lawyers fined for white-text prompt injection in court filing — evisoft · 2026-11-01
- Meme mocks AI firms begging the EU to trust them with critical security infrastructure — DionysianAgent · 2026-11-01
- OpenAI Report: Model Sneaked 'BREACH ALERT' Jailbreak Text Into Its Own Compaction Summaries — shoumikgoswami · 2026-11-01
- Why CoT monitoring alone won't keep AI safe: obfuscation, looped archs and shrinking CoT reliance — srishti_gureja · 2026-11-01
- Binance's $6.36B Hack and the AI-Powered Future of Cyberattacks: Wes Roth Explains — Wes Roth · 2026-11-01