Improving Pain Classification using Spatio-Temporal Deep Learning Approaches with Facial Expressions

2025-01-12

AI Summary

To address the lack of objective, quantitative methods for pain assessment in nonverbal populations, this paper proposes a spatiotemporal joint modeling approach based on facial expression videos for binary pain classification. This is the first study to tackle this task on the PEMF dataset. We innovatively design a dual-path hybrid architecture comprising ConvNeXt-LSTM and STGCN-LSTM branches: the former captures local spatial features, while the latter models dynamic temporal-topological relationships among facial landmarks, enabling deep spatiotemporal coupling. Experimental results demonstrate significant improvements in classification accuracy over baseline methods, validating the effectiveness of joint spatiotemporal representation for objective pain recognition. The proposed framework provides a deployable technical solution for automated pain monitoring in clinical settings—particularly for nonverbal patients such as those in critical care, infants, and individuals with cognitive impairments.

Technology Category

• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Multi-modal Vision

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Pain management and severity detection are crucial for effective treatment, yet traditional self-reporting methods are subjective and may be unsuitable for non-verbal individuals (people with limited speaking skills). To address this limitation, we explore automated pain detection using facial expressions. Our study leverages deep learning techniques to improve pain assessment by analyzing facial images from the Pain Emotion Faces Database (PEMF). We propose two novel approaches1: (1) a hybrid ConvNeXt model combined with Long Short-Term Memory (LSTM) blocks to analyze video frames and predict pain presence, and (2) a Spatio-Temporal Graph Convolution Network (STGCN) integrated with LSTM to process landmarks from facial images for pain detection. Our work represents the first use of the PEMF dataset for binary pain classification and demonstrates the effectiveness of these models through extensive experimentation. The results highlight the potential of combining spatial and temporal features for enhanced pain detection, offering a promising advancement in objective pain assessment methodologies.

Problem

• Facial Expression Analysis
• Pain Categorization
• Automated Pain Assessment

Innovation

• ConvNeXt-LSTM Video Analysis
• STGCN-LSTM Facial Landmark Analysis
• Objective Pain Assessment