Efficient and Interpretable Body-Based Emotion Recognition with Lightweight Temporal Convolutional Networks

2026-07-22

AI Summary

This study addresses the high computational cost, deployment challenges, and limited interpretability of graph neural networks in body-based emotion recognition by proposing lightweight temporal convolutional networks (TCNs) as an efficient alternative. We systematically evaluate multiple TCN variants on the DIEM-A dataset and compare them against the graph-based temporal semantic graph model (G-TSG). Experimental results show that TCN-Base achieves only a marginal drop in performance—1.58% in accuracy and 1.25% in macro-F1—while reducing parameter count by 79.18% and accelerating inference by approximately 12.5×. Through region-specific modeling, zero-value masking, and gradient saliency analysis, we further reveal for the first time that upper-body motion serves as the strongest independent cue for emotion recognition, thereby demonstrating both the effectiveness and interpretability of lightweight TCNs in this task.

Abstract

Body-based emotion recognition is important for real-time affective systems, but graph-based skeleton models can be computationally expensive. This paper studies whether lightweight temporal convolutional networks (TCNs) can provide an efficient and interpretable alternative for body-based emotion classification. We evaluate a family of TCN models on DIEM-A and compare them with a graph-based time-series graph (G-TSG) baseline using accuracy, macro-F1, parameter count, and inference latency. Although G-TSG achieves the highest mean performance, TCN-Base remains within $1.58$ accuracy points and $1.25$ macro-F1 points while using $79.18\%$ fewer parameters and reducing classifier latency by approximately $12.5\times$. We also analyze body-region contributions using region-specific TCN models, zero-based occlusion, and G-TSG gradient saliency. The results show that upper-body motion provides the strongest standalone regional cue, that the usefulness of body regions varies across emotions, and that different interpretability methods capture distinct aspects of model behavior. These findings suggest that lightweight TCNs can support efficient body-based emotion recognition while also providing practical insight into how motion cues contribute to classification.

Problem

• body-based emotion recognition
• lightweight models
• temporal convolutional networks
• interpretability
• computational efficiency

Innovation

• Lightweight Temporal Convolutional Networks
• Body-based Emotion Recognition
• Model Interpretability
• Efficient Inference
• Region-specific Analysis