Gaze into the Heart: A Multi-View Video Dataset for rPPG and Health Biomarkers Estimation

2025-08-25

AI Summary

Existing public remote photoplethysmography (rPPG) datasets suffer from limited scale, high privacy risks, narrow acquisition scenarios, and insufficient physiological ground-truth annotations—hindering model generalization and clinical deployment. To address these limitations, we introduce VitaFace, the first large-scale, multi-view rPPG dataset, comprising 3,600 synchronized video clips from 600 subjects, captured using off-the-shelf cameras at multiple angles alongside 100-Hz reference PPG signals. VitaFace provides expert-annotated measurements of 12 health biomarkers, including heart rate variability, blood pressure, and blood oxygen saturation. The dataset enables multimodal physiological modeling and cross-device validation. Leveraging VitaFace, we train a lightweight rPPG model that achieves state-of-the-art performance in cross-dataset evaluation, reducing mean absolute error by 23.6% over prior methods. Both the dataset and source code are publicly released, establishing a foundational infrastructure for contactless AI-driven health monitoring.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Humans and AI: Interaction Techniques and Devices
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Progress in remote PhotoPlethysmoGraphy (rPPG) is limited by the critical issues of existing publicly available datasets: small size, privacy concerns with facial videos, and lack of diversity in conditions. The paper introduces a novel comprehensive large-scale multi-view video dataset for rPPG and health biomarkers estimation. Our dataset comprises 3600 synchronized video recordings from 600 subjects, captured under varied conditions (resting and post-exercise) using multiple consumer-grade cameras at different angles. To enable multimodal analysis of physiological states, each recording is paired with a 100 Hz PPG signal and extended health metrics, such as electrocardiogram, arterial blood pressure, biomarkers, temperature, oxygen saturation, respiratory rate, and stress level. Using this data, we train an efficient rPPG model and compare its quality with existing approaches in cross-dataset scenarios. The public release of our dataset and model should significantly speed up the progress in the development of AI medical assistants.

Problem

• Addresses limitations of existing rPPG datasets with small size and privacy concerns
• Introduces multi-view video dataset for remote physiological signal estimation
• Enables health biomarker analysis through synchronized multimodal sensor data

Innovation

• Multi-view video dataset for rPPG
• Synchronized multimodal health metrics collection
• Cross-dataset evaluation of efficient model