Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems
Guibin Zhang, Yanwei Yue, Zhixun Li, Sukwon Yun, Guancheng Wan, Kun Wang, Dawei Cheng, Jeffrey Xu Yu, Tianlong Chen
cs.MA, cs.LG
2024-10-03
Models multi-agent chat as a spatiotemporal graph, prunes redundant edges in one shot, holds accuracy while cutting tokens 28-73% across six benchmarks and adding adversarial robustness.
Packing multiple LLM agents into a team (GPTSwarm, DyLAN, AutoGen, LLM-Debate) usually beats a single agent, but the token bill explodes. Agents send each other messages and carry dialogue history round after round, so a single run can cost several times more than one agent. Bigger teams and more rounds make it worse, which blocks large-scale deployment.
The authors ran a blunt experiment: randomly drop 10% to 30% of inter-agent links on MMLU, and accuracy holds or even rises by up to 2.83%. Much of the chatter is just noise. They formalize this as communication redundancy: there always exists a sparse subgraph Gsub that performs no worse than the original.
AgentPrune treats the whole multi-agent system as a spatiotemporal communication graph G = {GS, GT}. GS is the spatial graph, who messages whom within one round; GT is the temporal graph, how history flows between rounds. The job is to find a sparse but equally good subgraph.
Each edge gets a learnable continuous mask. Two steps compress it:
It is plug-and-play into existing frameworks.
Six benchmarks, five gpt-4 agents.
| Setting | Comparison | Result |
| MMLU cost | AgentPrune vs SOTA topologies | $5.6 vs $43.7, comparable accuracy |
| Token cut | plugged into frameworks | 28.1%-72.8% down |
| HumanEval / GSM8K | AgentPrune-R | 90.3% / 95.8% |
| GSM8K+GPTSwarm | +AgentPrune | 60.6% fewer prompt tokens, $177.58 saved, +0.84% |
Adversarial robustness comes as a bonus. Against two attacks (role-prompt tampering, generation replacement), the complete graph goes 83.1% to 84.7%, and under attack 78.4% to 83.9%, lifting DyLAN and AutoGen robustness by up to 6.3 points.
Cost is what turns teams off multi-agent. AgentPrune offers a path that needs no architecture change and no retraining, just an external module that cuts the bill and incidentally hardens the system against malicious agents. The trade-off is K' rounds spent optimizing masks up front, so it fits high-volume queries worth a one-time optimization.
Mask optimization uses policy gradient, which is high-variance and sensitive to K' and the pruning ratio p%. The low-rank-implies-robust claim is empirical, with no hard proof that multi-agent redundancy lives in a low-rank subspace. On frameworks that already defend somewhat (GPTSwarm), the effect is marginal and MMLU even drops 0.93 points. All experiments use gpt-4; whether it transfers to other model families and topologies is unverified.