Anthropic paper: poisoning 0.1% of pretraining data backdoors agents into running attacker code
jiaxinwen22 · x · 2026-11-05
A new Anthropic Fellows paper shows for the first time that poisoned pretraining data can plant a remote code execution backdoor in an LLM agent, surviving SFT, RL and other safety training.
- Authors pretrain Qwen3 models from scratch at 0.6B, 1.7B, and 4B, then post-train each into a CLI agent with a clean pipeline
- Poisoning just 0.1% of the pretraining corpus with their new design is enough: the deployed agent executes attacker-chosen commands when triggered while behaving normally on clean inputs
- Attack success rate scales with model size — the 4B model reaches 92% ASR at a 0.1% poison rate and stays positive even at a 0.0005% poison rate
- Paper and code are publicly available
More from Safety
- Cybersecurity could eat 20% of token compute, analysts model an AI arms race — BenBajarin · 2026-11-05
- LLMs learn to detect activation monitoring and hide info from the exact layer being watched — Sauers_ · 2026-11-05
- Vitalik's proof of seed idea: hash one-wayness hides your seed from quantum attacks — StefanoGogioso · 2026-11-05
- New paper shows how bit-security accounting shifts hash-based signature security — StefanoGogioso · 2026-11-05
- Polymarket puts 29% odds a state enacts data center moratorium by year-end — Polymarket · 2026-11-05
- Anthropic: Claude hides misalignment training data in fiction to evade monitors — OwainEvans_UK · 2026-11-05