Matryoshka Attribution: reverting 1% of weights removes most refusal in Llama 3.1 8B
aryaman2020 · x · 2026-09-29
The paper "Matryoshka Attribution" introduces a method that traces model behaviors back to the smallest set of internal components responsible for them.
- It learns a single ranked mask over internal components across many sparsity levels at once, efficiently isolating compact causal circuits.
- It achieves SoTA circuit localization and can attribute behaviors to weight changes, not just activations.
- Key result: in Llama 3.1 8B, reverting just 1% of weights toward the base model removes most refusal behavior while largely preserving capabilities.
This suggests safety alignment may be concentrated in a tiny fraction of weights, with direct implications for interpretability and safety research.
More from Research
- Why chirality matters for AI for Science: enantiomers behave differently in biology — madeofAjala · 2026-11-16
- Ropedia Xperience 2 embodied AI dataset lands on Hugging Face under CC BY-NC license — AdinaYakup · 2026-11-16
- RSS 2027 submissions open with FAQ clarifying two-stage review and negative-result bar — GeorgiaChal · 2026-11-16
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16
- Berkeley paper: LLMs answer with stale values even when the update is in context — ombre-ai · 2026-11-16
- LatamGPT releases 178.4M-document open Latin American corpus on Hugging Face — OmarUFlorez · 2026-11-16