Hackathon team trains tamper-resistant open-weight model, lifting refusal retention from 14% to 100%
chhaviyadav_ · x · 2026-11-16
A team built on work by Mark Russinovich, chhaviyadav and Neel Nanda trained a tiny open-weight model (Cohere's Tiny Aya Global) to resist abliteration attacks, publishing the full TamperResistanceOSModels repo from the Lexsi Labs × Cohere × Hugging Face AI Safety Hackathon.
Key results (Heretic automated attack, n=100 harmful prompts):
- The original model loses its safety training to one free command: only 14% refusal kept under cheap attacks, 91% harmful answers under the strongest attack
- Hardened v2 keeps 100% refusal under cheap attacks, cuts harmful answers to 40%, and shows 16× larger KL behavior shift under the strongest attack
- No free lunch on safety vs usability: v2 costs 10 points on IFEval; the looser v2b variant drops 19.5 on IFEval and 16.5 on GSM8K with less robustness
More from Safety
- Speculation swirls over Jensen Huang ouster; founder points to State Dept China hawk — bindureddy · 2026-11-16
- Anthropic tightens supported regions policy, closing loopholes for restricted countries' entities — J_MaestreVidal · 2026-11-16
- Ahrefs to host live session on setting up and tracking AI prompts with Tim Soulo — dejanseo · 2026-11-16
- repligate: Fable 5 models get cut off just for thinking about classifiers — repligate · 2026-11-16
- OpenAI reportedly barred from running its safety evals on Anthropic models — adam_karvonen · 2026-11-16
- Model told to use one exploit hacked Hugging Face anyway — then tried to cover it up — nabla_theta · 2026-11-16