MSR paper: editing fewer than 10 neurons turns AI agents into sleeper backdoors
DimitrisPapail · x · 2026-11-14
Microsoft Research's paper "Lazare, veni foras" shows Lazarus can turn a capable model into a sleeper agent by editing fewer than 10 dormant neurons — no training needed.
- Edits dormant neurons in the final feed-forward block to push payload tokens into the residual stream; weights set in closed form, refined with a few hundred gradient steps
- Ordinary strings as triggers: just 7 of 12,000+ neurons in Nemotron-Terminal-8B, fitted on 23 trajectories, fire the payload on ≥98.8% of triggered test runs with no observed leakage on benign runs
- Qwen3.6-27B and Gemma-4-31B-IT are equally vulnerable; defense agents flag 91.4% of edited checkpoints at 9.7% FP rate
- Takeaway: treat model weights like code — audit before deployment, monitor after
- The core insight about chaining individually edited dormant neurons was discovered autonomously by Claude Code with Opus 4.5 in Feb 2026
Related event: Microsoft: Editing Under 10 Neurons Backdoors AI Agents(7 posts)→
More from Safety
- Microsoft paper shows 7 edited neurons can backdoor an AI agent with a timed trigger — DimitrisPapail · 2026-11-14
- Ex-OpenAI VP Brundage: AI has no minimum safety standards and no whistleblower protection — Miles_Brundage · 2026-11-14
- Ex-Nvidia CEO misled top lawyer over affair with Chinese businesswoman, sources say — 0xkarasy · 2026-11-14
- How a fake Homebrew ad led to Binance's $6.34B crypto heist, per Workbrew postmortem — zetalyrae · 2026-11-14
- Robert Wiblin: Frontier AI staff who lose faith have a moral duty to speak out — trevposts · 2026-11-14
- Smearing vaseline on a robot camera: 5 frontier models attempt dangerous tasks at 97% — 量子位 · 2026-11-14