High Semantic Features for the Continual Learning of Complex Emotions: a Lightweight Solution

2025-10-15

AI Summary

To address catastrophic forgetting in continual learning for sustained emotion recognition, this paper proposes an Action Unit (AU)-based progressive continual learning framework. Unlike end-to-end image inputs, the method leverages high-level, non-transient facial AUs as stable semantic features—first acquiring fundamental emotions and then incrementally expanding to complex ones—to mitigate inter-task feature distribution shifts. The model employs a lightweight neural architecture, achieving 75% accuracy on the CFEE dataset—comparable to state-of-the-art (SOTA) methods—while significantly reducing parameter count and memory footprint. This design enhances cross-task transferability and deployment efficiency, overcoming two key limitations of conventional CNNs in continual learning: representational instability and computational redundancy.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Computer Vision: Representation Learning for Vision
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Incremental learning is a complex process due to potential catastrophic forgetting of old tasks when learning new ones. This is mainly due to transient features that do not fit from task to task. In this paper, we focus on complex emotion recognition. First, we learn basic emotions and then, incrementally, like humans, complex emotions. We show that Action Units, describing facial muscle movements, are non-transient, highly semantical features that outperform those extracted by both shallow and deep convolutional neural networks. Thanks to this ability, our approach achieves interesting results when learning incrementally complex, compound emotions with an accuracy of 0.75 on the CFEE dataset and can be favorably compared to state-of-the-art results. Moreover, it results in a lightweight model with a small memory footprint.

Problem

• Addresses catastrophic forgetting in incremental emotion recognition learning
• Identifies Action Units as stable features for complex emotion classification
• Develops lightweight model for sequential learning of compound emotions

Innovation

• Uses Action Units for facial emotion features
• Implements incremental learning for complex emotions
• Creates lightweight model with small memory