Centered Ablation: a closed-form fix that boosts refusal-direction ablation by up to 46 points

knoveleng · hf · 2026-11-17

Directional ablation is the standard way to switch off behaviors like refusal, but it is erratic—under-firing on some models and collapsing text into repetition on others. The paper shows the last-layer ablation is exactly a logit bias whose gate is the cosine with the direction, but last-layer states share a large common mode far from the origin, so directions overlapping it fire on benign text and under-fire on targets. Centered Ablation (CA) removes the common mode in closed form: across 8 models it raises attack success by 29–46 points where the standard recipe under-fires, cuts Gemma-2-9B collapse from 50% to 1%, and preserves 12–40 more benign answers (39–46 above prior affine offsets). Two pre-generation numbers predict which models gain faithfulness vs. attack success, and two standard safety judges score collapsed text in opposite directions, which can fabricate a 46-point effect.

Original post →

More from Safety

Safety channel →