Disentangled Source-Free Personalization for Facial Expression Recognition with Neutral Target Data

2025-03-26

AI Summary

In medical facial expression recognition (FER), domain adaptation is hindered by the absence of labeled non-neutral expressions in the target domain. Method: This paper proposes a Decoupled Source-Free Domain Adaptation (DSFDA) framework that requires only a single neutral-expression video from the target subject—without access to source-domain data, target-domain labels, or any non-neutral target samples. DSFDA implicitly synthesizes non-neutral expression features via adversarial generation and self-supervised reconstruction, while simultaneously disentangling identity and expression representations. It incorporates identity preservation constraints and source-domain expression transfer to eliminate reliance on full-category target data—unlike conventional source-free domain adaptation (SFDA). Contribution/Results: Evaluated on multiple FER benchmarks under ultra-low-resource settings (one neutral video per subject), DSFDA achieves a 12.7% absolute accuracy improvement over baselines in cross-subject evaluation, demonstrating strong feasibility for clinical deployment.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Application Domains: Humanities & Computational Social Science

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Decentralized Web and Fediverse systems

Abstract

Facial Expression Recognition (FER) from videos is a crucial task in various application areas, such as human-computer interaction and health monitoring (e.g., pain, depression, fatigue, and stress). Beyond the challenges of recognizing subtle emotional or health states, the effectiveness of deep FER models is often hindered by the considerable variability of expressions among subjects. Source-free domain adaptation (SFDA) methods are employed to adapt a pre-trained source model using only unlabeled target domain data, thereby avoiding data privacy and storage issues. Typically, SFDA methods adapt to a target domain dataset corresponding to an entire population and assume it includes data from all recognition classes. However, collecting such comprehensive target data can be difficult or even impossible for FER in healthcare applications. In many real-world scenarios, it may be feasible to collect a short neutral control video (displaying only neutral expressions) for target subjects before deployment. These videos can be used to adapt a model to better handle the variability of expressions among subjects. This paper introduces the Disentangled Source-Free Domain Adaptation (DSFDA) method to address the SFDA challenge posed by missing target expression data. DSFDA leverages data from a neutral target control video for end-to-end generation and adaptation of target data with missing non-neutral data. Our method learns to disentangle features related to expressions and identity while generating the missing non-neutral target data, thereby enhancing model accuracy. Additionally, our self-supervision strategy improves model adaptation by reconstructing target images that maintain the same identity and source expression.

Problem

• Adapts FER models using neutral target data only
• Generates missing non-neutral expression data
• Disentangles identity and expression features

Innovation

• Disentangled SFDA for missing target expression data
• Neutral target video for end-to-end data generation
• Self-supervision enhances identity and expression reconstruction