Aetheria: A multimodal interpretable content safety framework based on multi-agent debate and collaboration
Yuxiang He, Jian Zhao, Yuchen Yuan, Tianle Zhang, Wei Cai, Haojie Cheng, Ziyan Shi, Ming Zhu, Haichuan Tang, Chi Zhang, Xuelong Li
cs.AI
2025-12-02
Aetheria moderates multimodal content via five agents debating over two rounds, reaching F1 0.84 on AIR-Bench text plus image, versus 0.75 for the best open model and 0.50 for Azure.
Content moderation has three old problems. Keyword and rule-based systems miss context and implicit risk. Deep-learning classifiers (Llama Guard, ShieldGemma) are black boxes that flag content without explaining why, and they carry the bias and hallucinations of a single model. Commercial APIs (Azure Content Safety, OpenAI Moderation) handle plain text adequately but collapse on image-plus-text.
Aetheria (China Telecom TeleAI and others; arXiv, December 2025) targets the gap: a system that is both more accurate on multimodal input and produces an interpretable, auditable reasoning trail. Its intended use is explicitly not sub-second first-pass filtering. It is high-stakes second-pass review and appeals, where the system has to justify why something was flagged.
The core is five specialized agents plus a RAG memory component, run over two debate rounds:
Debate runs two rounds, and failed cases go to the Curator for post-mortem. The Curator distills lessons into Key Cues written back to the memory store for continuous learning. One deliberate choice is heterogeneous model composition: the five agents are not all the same model, and the ablation shows that making them all GPT-4o is worse.
On AIR-Bench (3,274 samples across text, image, and multimodal):
| Task | Aetheria F1 | Best open baseline | Azure |
| Text only | 0.92 | ShieldGemma-9B 0.90 | 0.55 |
| Image only | 0.87 | ShieldGemma-9B 0.82 | 0.25 |
| Text + Image | 0.84 | ShieldGemma-2B 0.75 | 0.50 |
The commercial-API collapse on multimodal is the sharpest line: Azure sits at 0.50 with 0.40 recall, effectively missing half of violations.
Ablations (multimodal F1) show what each piece buys: full system 0.84; drop RAG retrieval, 0.80; drop the Supporter agent, 0.79; keep only the Arbiter and skip debate, 0.70. Heterogeneity holds too: swap all five agents to GPT-4o and image F1 falls from 0.87 to 0.75. On rounds, going from one to two debate rounds lifts precision from 0.820 to 0.828 with F1 essentially flat and only 7.6% more time, so two rounds is the sweet spot.
Continuous learning pays off on hard batches: where the memory-less baseline dropped to 0.8252, the memory-equipped version reached 0.8708.
In practice this is an interpretable, auditable moderation layer that beats open classifiers on multimodal and lays out its reasoning. It fits high-stakes second-pass review and appeals, where the value is being able to explain a takedown rather than respond in milliseconds.
The transferable design lesson is the combination of heterogeneous multi-agent debate (strict versus loose as adversaries) with RAG precedent memory, which improves both accuracy and explainability.
Be honest about the scope. This is an engineering and systems contribution, an agent orchestration on top of existing VLMs and LLMs, not a new base model. Cost is real: 6.88 seconds per item on average and several model calls each. And on text-only the gain over the strongest open baseline (ShieldGemma-9B) is small (0.92 versus 0.90); the real advantage is concentrated in multimodal and in interpretability.
Average runtime is 6.88 seconds per item, far from real-time first-pass filtering, so the system is positioned for second-pass auditing only. Evaluation uses a single benchmark (AIR-Bench, 3,274 samples); generalization to other corpora, languages, and adversarial distributions is not tested.
There is no full cost-and-throughput analysis at scale, and multimodal plus multi-agent plus RAG burns several API calls per item. The text-only gain over the strongest open baseline is small, so the headline edge really sits in multimodal and interpretability.
The continuous-learning result comes from a small sequential experiment (4 batches of 250), and whether the memory generalizes or merely overfits to indexed cases is not stress-tested. The heterogeneous-model assumption also adds operational complexity, since you must run and maintain several models at once; the ablation shows homogeneity hurts, but the paper does not dig into why.