New M-estimator of the leading principal component

2025-10-03

AI Summary

This paper addresses robust estimation of the first principal component direction under elliptical symmetry, focusing on high-dimensional settings ($p o infty$) where the objective function is nonconvex and nonsmooth. We propose a novel class of M-estimators that achieve consistent and asymptotically normal directional estimation under only weak moment conditions and distributional symmetry—bypassing the conventional convexity and differentiability requirements of standard PCA and classical M-estimation. To solve the resulting nonsmooth optimization problem, we design a modified Weiszfeld-type iterative algorithm and establish its finite-step convergence rigorously. Leveraging empirical process theory, we further develop a foundation for statistical inference. Numerical simulations demonstrate that the proposed method exhibits strong finite-sample stability and robustness against heavy-tailed and contaminated distributions.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Learning with Manifolds
• Intelligent Robots: State Estimation

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

We study the minimization of the non-convex and non-differentiable objective function $v mapsto mathrm{E} ( | X - v | | X + v | - | X |^2 )$ in $mathbb{R}^p$. In particular, we show that its minimizers recover the first principal component direction of elliptically symmetric $X$ under specific conditions. The stringency of these conditions is studied in various scenarios, including a diverging number of variables $p$. We establish the consistency and asymptotic normality of the sample minimizer. We propose a Weiszfeld-type algorithm for optimizing the objective and show that it is guaranteed to converge in a finite number of steps. The results are illustrated with two simulations.

Problem

• Develops robust M-estimator for principal component analysis
• Studies non-convex optimization for elliptical symmetric distributions
• Establishes consistency and asymptotic normality properties

Innovation

• M-estimator minimizes non-convex robust objective
• Weiszfeld-type algorithm ensures finite-step convergence
• Recovers principal component under elliptical symmetry conditions