AMRD: Adaptive Multi-Teacher Distillation for Lightweight Speech Emotion Recognition
Yuqi Li · hf · 2026-07-31
While large self-supervised models excel at Speech Emotion Recognition (SER), their high compute cost makes them unsuitable for edge devices. This paper proposes Adaptive Multi-teacher Relational Distillation (AMRD) to compress these models into lightweight students.
AMRD tackles two key challenges: a one-class SVM dynamically evaluates teacher reliability per batch to assign weights, and a relational distillation loss aligns the inter-sample similarity matrices between teacher and student, preserving structural information missed by standard logit matching. On the IEMOCAP and CREMA-D datasets, AMRD outperforms single-teacher baselines in most settings, with ablations confirming the complementary gains of both components.
More from Research
- Review Paper on Reasoning Shortcuts in Neuro-Symbolic AI Published in JAIR — tetraduzione · 2026-07-31
- OpenMLE: Open-Source AI4AI System Hits 71% on MLE-Bench Lite Using a Single RTX 4090 — TianbaoX · 2026-07-31
- ShadowDancer: Teaching Video World Models Any Action via Shadow Pairs — AlayaLab · 2026-07-31
- Study Shows Late Interaction Models Generalize Better in Multilingual Tasks — lateinteraction · 2026-07-31
- Developer Open-Sources 'Unbiased' LLM Benchmark — cephaloform · 2026-07-31
- 3D Chest CT Vision-Language Models for Automated Report Generation Accepted at RSNA 2026 — maier_ak · 2026-07-31