Time-Series U-Net with Recurrence for Noise-Robust Imaging Photoplethysmography

2025-03-21

AI Summary

This study addresses the reduced robustness of contactless, low-cost video-based remote heart rate and photoplethysmographic (PPG) waveform estimation under facial occlusion and extreme head poses. We propose a three-stage interpretable pipeline: (1) robust face and facial landmark detection; (2) motion-robust spatiotemporal signal extraction; and (3) PPG signal reconstruction and heart rate estimation using TURNIP—a novel temporal network integrating U-Net architecture with recurrent mechanisms. To enhance reliability under extreme poses, we introduce the first explicit self-occlusion region detection module. The framework supports multimodal RGB and near-infrared (NIR) inputs, enabling real-world deployment. Evaluated on public RGB and NIR datasets, it achieves state-of-the-art performance, with significantly lower mean absolute error in heart rate estimation compared to existing iPPG methods. Crucially, it requires no specialized hardware or skin contact, delivering high-accuracy, motion-robust remote physiological monitoring.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Large-scale security measurements

Abstract

Remote estimation of vital signs enables health monitoring for situations in which contact-based devices are either not available, too intrusive, or too expensive. In this paper, we present a modular, interpretable pipeline for pulse signal estimation from video of the face that achieves state-of-the-art results on publicly available datasets.Our imaging photoplethysmography (iPPG) system consists of three modules: face and landmark detection, time-series extraction, and pulse signal/pulse rate estimation. Unlike many deep learning methods that make use of a single black-box model that maps directly from input video to output signal or heart rate, our modular approach enables each of the three parts of the pipeline to be interpreted individually. The pulse signal estimation module, which we call TURNIP (Time-Series U-Net with Recurrence for Noise-Robust Imaging Photoplethysmography), allows the system to faithfully reconstruct the underlying pulse signal waveform and uses it to measure heart rate and pulse rate variability metrics, even in the presence of motion. When parts of the face are occluded due to extreme head poses, our system explicitly detects such"self-occluded"regions and maintains estimation robustness despite the missing information. Our algorithm provides reliable heart rate estimates without the need for specialized sensors or contact with the skin, outperforming previous iPPG methods on both color (RGB) and near-infrared (NIR) datasets.

Problem

• Remote vital sign estimation without contact devices
• Modular pipeline for noise-robust pulse signal extraction
• Handling face occlusion and motion in heart rate monitoring

Innovation

• Modular pipeline for pulse signal estimation
• Time-Series U-Net with Recurrence (TURNIP)
• Self-occlusion detection for robust estimation