Single Neuron Suffices to Bypass Safety Alignment in LLMs, Paper Finds
burny_tech · x · 2026-10-18
A shared arXiv paper (2605.08513) argues LLM safety alignment runs on two mechanistically distinct systems: refusal neurons that gate whether harmful knowledge is expressed, and concept neurons that encode the harmful knowledge itself. By targeting a single neuron in each system, the authors demonstrate both failure directions—suppressing refusal to bypass safety on explicit harmful requests, and amplifying concept neurons to elicit harmful content from innocent prompts—across seven models from two families spanning 1.7B to 70B parameters, with no training or prompt engineering required.
More from Safety
- Grady Booch slams Anthropic for tolerating up to 50 deaths and $1B damages per model — Grady_Booch · 2026-10-18
- Polymarket gives OpenAI just 8% odds of a full training pause by end of 2026 — Polymarket · 2026-10-18
- OpenAI engineer over alleged hidden recording device seeks bail, denies Saudi escape plan — Polymarket · 2026-10-18
- Free comic "The Collective" offers a strange retelling of the Hugging Face hack — Flugelwagen · 2026-10-18
- SpaceXAI's OAuth consent page draws scrutiny over vague X data-sharing language — yenkel · 2026-10-18
- Interpretability lacks metrics robust enough to survive optimization, researcher says — nabla_theta · 2026-10-18