Explainable Lightweight Compact Deep Models for Speech Emotion Recognition

2026-07-18

AI Summary

This study addresses the limitations of existing speech emotion recognition approaches, which often rely on complex deep models suffering from poor interpretability and high computational costs, thereby hindering their deployment in high-transparency domains such as healthcare. To overcome these challenges, this work proposes a lightweight and compact convolutional neural network framework that integrates log-Mel spectrograms, attention-based statistical pooling, and Grad-CAM visualization to enhance both decision transparency and classification accuracy while maintaining a low parameter count. Experimental results demonstrate that the proposed method achieves competitive performance on the SAVEE dataset, significantly reducing model size compared to mainstream approaches and effectively balancing efficiency, accuracy, and interpretability.

Abstract

Speech Emotion Recognition (SER) is an important component in a wide range of human-centered applications, including healthcare, customer service, and human-omputer interaction. In medical and decision-support settings, there is increasing interest in models that not only achieve accurate emotion recognition but also support transparent predictions and efficient deployment. However, many existing SER approaches rely on complex deep learning architectures that limit interpretability and increase computational cost. This paper presents an explainable and lightweight speech emotion recognition framework based on a compact convolutional neural network architecture. The proposed approach utilizes log-Mel spectrogram representations to capture spectro-temporal speech characteristics and employs attentive statistics pooling to emphasize emotionally salient temporal segments. To improve model transparency, gradient-based class activation mapping (Grad-CAM) is incorporated to visualize the time-frequency regions that influence the model's predictions. Experimental evaluation on the SAVEE emotional speech dataset demonstrates that the proposed framework achieves competitive recognition performance while maintaining a compact architecture with significantly fewer parameters than many existing SER models. The results indicate that efficient convolutional architectures combined with interpretable analysis can provide a practical balance between recognition accuracy, computational efficiency, and model transparency.

Problem

• Speech Emotion Recognition
• Explainability
• Lightweight Models
• Model Interpretability
• Computational Efficiency

Innovation

• lightweight
• explainable AI
• convolutional neural network
• Grad-CAM
• attentive statistics pooling