Multi-group IRT decomposes cross-lingual safety benchmarks
sanmikoyejo · x · 2026-08-17
Multi-group IRT separates aggregate safety scores into four components: language-agnostic robustness, intrinsic prompt hardness, global language difficulty, and cross-lingual safety gaps, evaluated on MultiJail across 61 models and 10 languages.
More from Research
- RLHF supersedes temp<1 heuristic, favors temp=1.0 — jessi_cata · 2026-08-17
- Why Simple Baselines Beat Fancy Biology Models: A Practical Guide — HongyiWang10 · 2026-08-17
- Nature reviewers flagged core flaw in 'science less disruptive' paper pre-publication — RexDouglass · 2026-08-17
- multiPL-e dataset contains hilarious regex errors, generating 'rsthon' tasks — code_star · 2026-08-17
- Opinion: Applying $100k of frontier compute beats decades of manual search — littmath · 2026-08-17
- Paper Maps Human-AI Creativity Across Codified, Tacit, and Novel Rules — sebkrier · 2026-08-17