Advancing Generalizable Remote Physiological Measurement through the Integration of Explicit and Implicit Prior Knowledge

2024-03-11

arXiv.org

AI Summary

Weak cross-dataset generalization of remote photoplethysmography (rPPG) methods—due to sensitivity to camera characteristics, illumination conditions, skin tone, and motion—remains a critical challenge. To address this, we propose a dual-branch network that jointly incorporates explicit and implicit physiological priors: the explicit branch models multi-source noise mechanisms (e.g., optical imaging physics and physiological coupling), while the implicit branch enforces physiological feature–interference disentanglement via label correlation constraints. Our method enables zero-shot cross-modal transfer between RGB and near-infrared (NIR) modalities. In RGB cross-dataset evaluation, it significantly outperforms state-of-the-art methods. Notably, it achieves the first zero-shot RGB→NIR generalization, empirically validating the pivotal role of physiological prior modeling in domain adaptation. The source code is publicly available.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Remote photoplethysmography (rPPG) is a promising technology that captures physiological signals from face videos, with potential applications in medical health, emotional computing, and biosecurity recognition. The demand for rPPG tasks has expanded from demonstrating good performance on intra-dataset testing to cross-dataset testing (i.e., domain generalization). However, most existing methods have overlooked the prior knowledge of rPPG, resulting in poor generalization ability. In this paper, we propose a novel framework that simultaneously utilizes explicit and implicit prior knowledge in the rPPG task. Specifically, we systematically analyze the causes of noise sources (e.g., different camera, lighting, skin types, and movement) across different domains and incorporate these prior knowledge into the network. Additionally, we leverage a two-branch network to disentangle the physiological feature distribution from noises through implicit label correlation. Our extensive experiments demonstrate that the proposed method not only outperforms state-of-the-art methods on RGB cross-dataset evaluation but also generalizes well from RGB datasets to NIR datasets. The code is available at https://github.com/keke-nice/Greip.

Problem

• rPPG
• generalizability
• interference sources

Innovation

• rPPG Enhancement
• Dual-Stream Network
• Cross-Modal Adaptation