AMRD: Adaptive Multi-Teacher Relational Distillation for Lightweight Speech Emotion Recognition

2026-07-28

AI Summary

This work addresses the challenges of on-device speech emotion recognition, where large self-supervised models incur prohibitive computational costs, and existing knowledge distillation approaches suffer from unreliable teacher predictions and neglect of inter-sample relational structures. To overcome these limitations, the authors propose an adaptive multi-teacher relational distillation framework. This framework incorporates a one-class SVM–based mechanism to assess teacher reliability and dynamically weight their predictions, while also introducing a relational distillation loss that preserves cross-sample semantic structure by aligning similarity matrices between teachers and students. Experiments on IEMOCAP and CREMA-D demonstrate that four lightweight student models consistently outperform single-teacher baselines, and ablation studies confirm the complementary benefits of the two core components.

Abstract

On-device speech emotion recognition (SER) is critical for real-time applications, yet large self-supervised models that excel at SER are too costly for edge devices. Multi-teacher knowledge distillation can compress them into a lightweight student, but two challenges remain: teacher reliability varies across batches, and logit-level distillation ignores inter-sample relational structure. We propose Adaptive Multi-teacher Relational Distillation (AMRD) to address both. A one-class SVM on each teacher's logit similarity matrix assigns per-batch weights favoring more coherent teachers. A relational distillation loss aligns teacher and student similarity matrices, capturing structure that logit matching misses. On IEMOCAP and CREMA-D datasets across four student architectures, AMRD outperforms single-teacher distillation baselines in most settings, and ablations confirm both components yield complementary gains.

Problem

• speech emotion recognition
• knowledge distillation
• multi-teacher
• edge devices
• relational structure

Innovation

• Adaptive Multi-teacher Distillation
• Relational Knowledge Distillation
• On-device Speech Emotion Recognition
• Logit Similarity Matrix
• One-class SVM