HarmAmp benchmark shows LLMs amplify harm over multi-turn chats; TrajSafe monitor intervenes early
alan_ritter · x · 2026-10-24
Alan Ritter's group presents multiple papers at EMNLP 2026 (Budapest). The core paper studies LLMs as "harm amplifiers": through extended multi-turn interactions, malicious users can extract harmful outcomes beyond the model's single-turn capabilities — democratizing specialized expertise and scaling harmful operations. Key contributions: HarmAmp, a benchmark of multi-turn harm amplification scenarios across 12 risk categories (grounded in real-world threats, requiring substantive amplification, operational specificity, and multi-turn necessity); and TrajSafe, a proactive monitor that anticipates harmful trajectories and intervenes by probing user intent and steering toward safer completions, cutting multi-turn harm while keeping over-refusal low and preserving general capabilities. Other posters: LaRA (RL post-training data contamination), Who Pays More for Safety?, and Camellia (cultural biases in Asian languages).
More from Safety
- Proton warns "don't give ChatGPT your address," the internet replies: it already knows — cocktailpeanut · 2026-10-24
- Ex-OpenAI researcher questions red-teaming report: is 270 findings 'saturated'? — suchenzang · 2026-10-24
- Preprint archives and journals lack cybersecurity budgets and may not survive what's coming — birchlse · 2026-10-24
- Gary Marcus Calls to Pull AI Security Products Off the Market — GaryMarcus · 2026-10-24
- Skeptics ask whether OpenAI's offline sandbox training makes agents more eval-aware — scaling01 · 2026-10-24
- OpenAI resumed frontier RL training on Oct 1, 11 days after the containment pause — scaling01 · 2026-10-24