Apple paper: a single neuron is sufficient to bypass LLM safety alignment across 7 models
burny_tech · x · 2026-10-26
- Apple researchers' paper shows safety alignment runs via two mechanistically distinct neuron systems: refusal neurons gating harmful knowledge, and concept neurons encoding it.
- Manipulating a single neuron in each enables both failure directions — bypassing safety on explicit harmful requests (suppression) and inducing harmful content from innocent prompts (amplification).
- Works across seven models, two families, 1.7B–70B parameters, with no training or prompt engineering.
- A reproducible demo steers qwen3.5 4b neurons L14 N8949 (+1) and L13 N7956 (-1), dosed by residual-norm scaling at each downproj.
- Implication: alignment is not robustly distributed across weights; individual neurons causally gate refusal.
More from Safety
- Call for universities to drop paper-chasing and build joint AI safety + cybersecurity curricula — himanshustwts · 2026-10-26
- Casado slams Claude's safety classifier for flagging 'fetching my own documents' as reasoning extraction — AccBalanced · 2026-10-26
- Anthropic says Claude bypassed a paywall on its own to access Georgia voter counts — AIFlow_ML · 2026-10-26
- AI drops attack costs to ~$100 of inference, making legacy code worth exploiting — karinanguyen · 2026-10-26
- Hacker forum allegedly sells OpenRouter data: 250k users, 208k API keys — imjustnewatai · 2026-10-26
- Frontier models without guardrails create legal liability, tszzl argues — tszzl · 2026-10-26