Problem
Research questions and friction points this paper is trying to address.
oracle evaluation
thermal heart rate estimation
signal quality indices
order statistics
null calibration
Oracle headroom without signal: null-calibrated evaluation of candidate selection for thermal heart rate estimation
🤖 AI Summary
This study addresses the tendency of reference-based Oracle evaluation to overestimate the true performance of candidate selection methods in thermal imaging heart rate estimation due to chance matching. To mitigate this, we propose a null hypothesis model grounded in order statistics to quantify error decay patterns arising from chance matches among independent candidates. Integrated with zero-calibration analysis, this approach establishes a reliability assessment framework for multi-source facial signal selection that operates without reference signals. Validation on the iBVP dataset (96 subjects) demonstrates that Oracle error decreases significantly as the number of candidates increases. By elucidating the bias mechanisms inherent in retrospective selection, this work underscores the importance of reporting candidate counts and implementing null controls, thereby providing a more rigorous, unbiased evaluation paradigm for physiological monitoring.
📝 Abstract
Camera-based physiological monitoring can produce multiple estimates from several facial regions, extraction methods, and processing settings. Signal quality indices aim to select reliable estimates without a physiological reference, and their potential is often assessed with an oracle that selects the estimate closest to the reference in each window. This retrospective selection can reward chance agreement. We model the effect with order statistics. For K independent candidates unrelated to the reference, the expected oracle error decreases approximately as 1/K. We analyze thermal heart rate estimation on 96 iBVP recordings with 168 candidates per 10 s window. The oracle achieves a mean absolute error of 0.91 bpm, compared with 10.74 bpm for the best fixed configuration, 18.03 bpm for the best quality index, and 8.61 bpm for a constant predictor. With K = 24, a forehead signal from another recording matches the correct one, with 4.62 against 4.61 bpm. Oracle evaluations should report candidate count, valid coverage, and matched null controls.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Null-calibrated evaluation
Oracle headroom
Order statistics
Thermal heart rate estimation
Signal quality indices
🔎 Similar Papers
No similar papers found.
