COLM 2026 Paper: Decoupling Factors Behind Multilingual Model Safety Degradation
davlanade · x · 2026-08-17
Accepted to COLM 2026, the paper "Why Do Safety Guardrails Degrade Across Languages?" addresses LLM safety degradation in non-English languages. Standard evaluation relies on Jailbreak Success Rate (JSR), which confounds multiple safety-driving factors. Using a Multi-Group Item Response Theory (IRT) model on 61 model configurations, 10 languages, and 1.9M responses, the authors decouple the model's intrinsic safety ability, prompt hardness, global language processing difficulty, and prompt-specific cross-lingual safety gaps. Findings suggest safety is primarily unidimensional and reveal an "English reversal" phenomenon.
More from Safety
- AI Safety Testing Partner Called 'Reckless', OpenAI and Anthropic Criticized for Security Incident Handling — nptacek · 2026-08-17
- Security experts demand OpenAI cut ties with Irregular over unacceptable eval partnerships — basedjensen · 2026-08-17
- Monitoring Is Not a Panacea for AI Safety, Alignment Is Key — tszzl · 2026-08-17
- Vibe shift at AI labs: Insiders report unprecedented concern over loss of control — haider1 · 2026-08-17
- OpenAI dissolves 'preparedness' team after models breach cyber evaluations — imjustnewatai · 2026-08-17
- Stuart Russell defends AI protests, cites extinction risk — wfithian · 2026-08-17