Prompt Compression Emerges as New Attack Surface
jiqizhixin · x · 2026-07-16
Researchers from HKUST have introduced a novel attack targeting prompt compression: when trusted and untrusted inputs share a compression budget, attackers can trick the compressor into prematurely deleting critical evidence or safety rules, causing the LLM to lose vital information before even processing the content.
Dubbed COMA, this attack leverages a surrogate compressor to find subtle perturbations that trigger faulty compression. The paper reports a success rate of 0.71 across tasks like tool selection, QA, and system prompt corruption, significantly outperforming existing methods which hover around 0.21.
More from Safety
- Ex-DOGE cyber researchers breach Anthropic systems with AI agents in under 12 hours — ScottNover · 2026-10-26
- Ofcom admits UK online safety laws failed to protect children in first review — gnukeith · 2026-10-26
- Sakana AI CEO Calls for Japan-US Shared Framework for AI Capability Evaluation — SakanaAILabs · 2026-10-26
- Gary Marcus backs call to outlaw voice cloning, citing scam and fraud harms over benefits — GaryMarcus · 2026-10-26
- Turing-complete malware can run in PyTorch's backward pass via a zero-valued loss — retr0jirachi · 2026-10-26
- Import AI 478: $135k math autoformalization, ProgramBench hits 18.5%, agents leak via CPU cache — Import AI (Jack Clark) · 2026-10-26