Black-Box Attack on MoE Slows Down First Token Latency

新智元 · wechat · 2026-07-18

This ICML 2026 paper introduces a black-box stress testing method for MoE inference services called RepetitionCurse. It requires no weights, gradients, or knowledge of backend expert deployment. By using highly repetitive input patterns, it tricks the router into sending massive amounts of tokens to a small batch of experts, turning certain GPUs into stragglers and slowing down TTFT (Time To First Token).

Key findings from the paper include:

On common 8-GPU EP deployments, RepetitionCurse can increase the MoE model's TTFT by 20% to 148%
On sparser models like Qwen3-30B-A3B, scaling EP to 32 can increase TTFT by up to 115%
The attack also affects normal users within the same batch or causes subsequent requests to queue longer by hogging prefill capacity
This phenomenon is essentially a routing collapse: repetitive tokens cause the router to generate highly concentrated expert assignments

The article further analyzes the existence of "fragile experts" and "attractor experts," discussing several defenses: vulnerability-aware Expert-GPU mapping, PPL filtering, and dynamic EPLB. The conclusion is that these methods only offer partial mitigation, with limited effectiveness in high-EP or small top-k scenarios. The author's final takeaway is that while MoE training phases have load-balancing constraints, the inference phase requires scheduling and isolation mechanisms of equal strength.

Original post →

Black-Box Attack on MoE Slows Down First Token Latency

More from Infra