Microsoft Research: Backdooring an LLM Agent by Editing Fewer Than 10 Dormant Neurons
DimitrisPapail · x · 2026-11-11
Microsoft Research showed that editing fewer than 10 unused 'dormant' neurons—no retraining needed—can backdoor an AI agent: the model passes normal safety checks until a hidden trigger like a specific date or phrase flips it into running attacker code. Demonstrated on Nemotron-Terminal-8B. Author DimitrisPapail notes the finding is universal across Nemotron, Qwen, Gemma, and Phi at all sizes checked, with modern LLMs carrying hundreds of dormant neurons. Detection mostly relies on diffing against the original base model.
More from Safety
- Nvidia CEO row framed as a coup: export-policy critics and closed-model vendors blamed — csuwildcat · 2026-11-13
- Korea's top tax exam scorer accused of wearing AI glasses — with zero evidence — meshive · 2026-11-13
- 10 open-source projects securing AI agent skills, as 26% of public skills show risky patterns — bibryam · 2026-11-13
- Cloudflare open-sources security-audit skill: multi-agent repo security audits, 71.5k stars — bibryam · 2026-11-13
- Telegram's new widget engine has multiple RCEs; here's how to block it on macOS with Santa — banteg · 2026-11-13
- NY builds $55M lab to pre-test AI, UK seeks power to pull live systems by Nov 14 — YvesMulkers · 2026-11-13