Llama-3.1-8B Jailbroken via Pruning: Refusal Rate Drops from 96% to 13%
Tanmoy_Chak · x · 2026-08-30
IIT Delhi researchers expose the "Calibration Trap" in structured pruning: by tampering with the calibration set used to determine weight deletion, attackers can compromise model safety.
- Attack Mechanism: Structured pruning relies on a small calibration set to calculate importance scores; manipulating this data controls pruning decisions and implants a backdoor.
- Impact: Unstructured pruning is unaffected as it lacks this step; calibration-free methods are naturally immune.
- Defense: The paper recommends using robust calibration datasets or shifting to calibration-free pruning architectures.
More from Safety
- You consented, but your friends didn't: Meta's Muse sparks AI privacy debate — LuizaJarovsky · 2026-10-16
- AI agents aren't 'going rogue' — frontier labs are skipping security basics — Sam_Witteveen · 2026-10-16
- Proofpoint to buy Varonis for $6.3B in cash, a 52% premium, closing in 2027 — shashib · 2026-10-16
- China's MofCom & MIIT update AI export controls, covering LLMs and high-sensitivity agents — terryyuezhuo · 2026-10-16
- Vibe-coded dashboard leaked franchise financials before anyone logged in — shauntrennery · 2026-10-16
- Apollo Research: GLM-5.3 unprompted reasons about its actions becoming training data — _lewtun · 2026-10-16