Theoretical Guarantees for the Subspace-Constrained Tyler's Estimator

2024-03-27

AI Summary

Robust low-dimensional subspace recovery under highly contaminated data—where inliers constitute an extremely small fraction—is challenging for existing methods. Method: We propose the Subspace-Constrained Tyler Estimator (STE), which embeds the Tyler M-estimator into a subspace-constrained optimization framework and solves it via iterative reweighted least squares. Contributions: First, we establish the first convergence guarantee for STE. Second, under a weak inlier–outlier model, we prove its exact subspace recovery property. Third, we significantly lower the feasible inlier rate bound—enabling exact recovery even when inliers drop far below the failure threshold of classical Tyler estimation, i.e., in extreme sparsity regimes. Experiments on the generalized haystack model demonstrate that STE, initialized with the Tyler Mean Estimator (TME), successfully recovers subspaces with as little as 0.1% inlier fraction—substantially expanding the practical applicability of robust subspace learning.

Technology Category

Intelligent Robots: State EstimationMachine Learning: Learning with ManifoldsSearch and Optimization: Non-convex Optimization

Application Category

Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphsWeb Mining and Content Analysis: Robustness and generalizability of Web mining methodsSecurity and Privacy: Large-scale security measurements

Abstract

This work analyzes the subspace-constrained Tyler's estimator (STE) designed for recovering a low-dimensional subspace within a dataset that may be highly corrupted with outliers. It assumes a weak inlier-outlier model and allows the fraction of inliers to be smaller than a fraction that leads to computational hardness of the robust subspace recovery problem. It shows that in this setting, if the initialization of STE, which is an iterative algorithm, satisfies a certain condition, then STE can effectively recover the underlying subspace. It further shows that under the generalized haystack model, STE initialized by the Tyler's M-estimator (TME), can recover the subspace when the fraction of iniliers is too small for TME to handle.

Problem

• Recover low-dimensional subspace from outlier-corrupted data
• Establish recovery guarantees for subspace-constrained Tyler's estimator
• Handle inlier fractions below typical robust recovery thresholds

Innovation

• Subspace-constrained Tyler's estimator for robust subspace recovery
• Computationally efficient method with weak inlier-outlier model
• Initialization with Tyler's M-estimator enhances recovery guarantees