MAttr traces Llama 3.1 refusals to 1% of parameter delta, resetting them disables refusal
aryaman2020 · x · 2026-09-24
MAttr works beyond representations: it can trace qualitative LLM behaviors to a subset of parameters updated during training, by (1) training attribution scores with RL using a judge score (StrongREJECT) as reward, and (2) applying the mask to model weights across two checkpoints instead of representations across prompts. Result: Llama 3.1 8B refusals map to 1% of the parameter delta between Base and Instruct; resetting those weights in the Instruct model keeps its capabilities while largely turning off refusal.
More from Research
- Autonomous agent edits just 1,024 weights to lift Qwen3.5 0.8B from 7.4% to 37.4% on MATH-500 — VictorSanh · 2026-11-12
- Gait dataset labeled by encounter, spanning pre-op to 3 months post-TKA — zawthet · 2026-11-12
- Walking alignment measured 2.5-3.1° more neutral than X-ray, iPad gait study finds — zawthet · 2026-11-12
- MIT's Tegmark coins 'formal-washing': Signal protocol proof is far from complete — tegmark · 2026-11-12
- Signal formal verification hits 80%, Tegmark says Wired overstated completion — tegmark · 2026-11-12
- Category-theoretic framework asks: when does a physical system actually compute? — neurovium · 2026-11-12