🤖 AI Summary
This study addresses the lack of reproducibility and comparability in existing photoplethysmography (PPG)-based non-invasive blood glucose estimation research, which stems from inconsistent datasets, data leakage, and ambiguous evaluation metrics. To resolve these issues, the authors introduce the first reproducible and extensible evaluation framework and systematically reassess five representative algorithms under three rigorous data-splitting protocols: random window-level, participant-aware, and leave-some-participants-out. Comprehensive evaluation combines the coefficient of determination (R²) with the Clarke Error Grid. The findings reveal that random splitting induces sample-level data leakage, substantially inflating generalization performance estimates. Under strict protocols, all models exhibit R² values near or below zero—performing no better than a mean baseline—yet over 90% of predictions fall within clinically acceptable zones, exposing a critical flaw in current clinical error metrics that mask fundamental model failure.
📝 Abstract
Non-invasive blood glucose level (BGL) estimation from photoplethysmography (PPG) holds great promise for wearable health monitoring, but results across studies are hard to compare due to inconsistent datasets, data leakage, and non-standardized evaluation metrics. We present the first reproducible, extensible evaluation pipeline and use it to reassess five representative PPG-based BGL methods on published datasets under three increasingly strict data-split protocols: random window-level, participant-aware, and leave-some-participants-out (LSPO). Models appeared competitive under random splitting but collapsed under participant-aware and LSPO evaluation, with nearly all yielding near-zero or negative R$^2$ values comparable to a mean-prediction baseline. Critically, across every model and split, over 90% of predictions fell within clinically acceptable zones (Clarke Error Grid A+B), including the baseline. This reveals a fundamental disconnect: clinical zone metrics systematically conceal model failure in this domain. Our findings demonstrate that random train-test splits substantially overestimate the generalization of PPG-based BGL models due to sample-level data leakage, and that robust ML evaluation must precede clinical validation to meaningfully assess real-world utility.