Functional principal component analysis for functional data with detection limits

2025-10-19

AI Summary

Non-random missingness due to detection limits—common in longitudinal biomarkers and other functional data—constitutes a missing-not-at-random (MNAR) mechanism. Conventional functional principal component analysis (FPCA) imputes values at detection limits, ignoring the MNAR structure and inducing bias in eigenfunction and score estimation. Method: We propose the first FPCA framework specifically designed for MNAR functional data, systematically integrating detection-limit–adjusted mean and covariance function estimation directly into the FPCA pipeline—thereby circumventing imputation-induced bias. Contribution/Results: Building on the estimation theory of Liu & Houwing-Duistermaat (2022, 2023), our method achieves consistency and robustness under both sparse and dense sampling regimes, as confirmed by asymptotic analysis and extensive simulations. Empirical evaluation—including simulation studies and real-data applications—demonstrates substantially improved accuracy in eigenfunction and score estimation, enhancing the reliability and applicability of functional data analysis under detection limits.

Technology Category

Data Mining & Knowledge Management: Anomaly/Outlier DetectionMachine Learning: Kernel MethodsIntelligent Robots: State Estimation

Application Category

Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphsUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and rankingWeb Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

When measurements fall below or above a detection threshold, the resulting data are missing not at random (MNAR), posing challenges for statistical analysis. For example, in longitudinal biomarker studies, observations may be subject to detection limits. Functional principal component analysis (FPCA) is commonly used method for dimension reduction of dense and sparse data measured along a continuum, but standard approaches typically ignore MNAR mechanisms by imputing detection limit values, leading to biased estimates of principal components and scores. Building on recent work by Liu and Houwing-Duistermaat (2022, 2023), who proposed estimators for the mean and covariance functions under detection limits, we extend FPCA to accommodate functional data affected by such limits. We derive the asymptotic properties of the resulting estimators and assess their performance through simulations, comparing them to standard methods. Finally, we illustrate our approach using longitudinal biomarker data subject to detection limits. Our method yields more accurate estimates of functional principal components and scores, enhancing the reliability of functional data analysis in the presence of detection limits.

Problem

• Extending FPCA for functional data with detection limits
• Addressing MNAR mechanisms in functional principal component analysis
• Improving estimation accuracy under detection limit constraints

Innovation

• Extends FPCA for functional data with detection limits
• Derives asymptotic properties of the proposed estimators
• Improves accuracy of principal components and scores