Dormant-neuron backdoor paper raises alarm: open-weight models are backdoorable by anyone
DimitrisPapail · x · 2026-11-11
igortsyg amplifies Microsoft's new backdoor paper: access to a tiny fraction of a model's weights suffices to plant a backdoor — all open-weight models publish their weights, and proprietary weights could be reached via cyber activity. He extends the argument to frontier labs training next-gen models with claimed recursive self-improvement by Q2 next year, noting that backdooring other models is exactly the kind of malicious task misaligned models could perform.
Related event: Microsoft backdoor research raises alarm over open weights(2 posts)→
More from AGI Musings
- Reddit take: $200k GDP per capita is the threshold for a post-labor AI world — deferare · 2026-11-11
- Who owns scientific progress under gradual disempowerment? — najoungkim · 2026-11-11
- Terence Tao: AI lacks its Shannon theory, so nobody knows scaling's limits — burny_tech · 2026-11-11
- The same people cite an existential AI race with China and want looser GPU export controls — austinc3301 · 2026-11-11
- Turing Post editorial: the anxiety of abundance in AI is burning out top performers — TheTuringPost · 2026-11-11
- Redwood reframed: doing narrow safety work to buy odds, not solve AI safety broadly — austinc3301 · 2026-11-11