🤖 AI Summary
This work addresses the global optimization challenge of overparameterized nonconvex low-rank matrix recovery under noise, where existing methods often get trapped at non-strict saddle points and lack rigorous theoretical guarantees. We propose a unified analytical framework for escape directions and nonexistence proofs of counterexamples, establishing—for the first time under the Restricted Isometry Property (RIP)—a sharp min-max optimal recovery bound for nearly second-order stationary points in the overparameterized regime. By integrating symmetric and asymmetric parameterizations, balanced regularization design, and higher-order optimization theory, we prove that such stationary points achieve recovery accuracy within a constant factor of convex methods. Moreover, the resulting error bound is sharp with respect to both noise level and solution accuracy.
📝 Abstract
Recent work established that rank overparameterization eliminates spurious local minima in nonconvex low-rank matrix recovery under the restricted isometry property (RIP). But this does not fully explain the practical success of overparameterization, because real algorithms can still become trapped at nonstrict saddle points (approximate second-order points with arbitrarily small negative curvature) even when all local minima are global. Moreover, the result does not accommodate for noisy measurements, but it is unclear whether such an extension is even possible, in view of the many discontinuous and unintuitive behaviors already known for the overparameterized regime. In this paper, we introduce a novel proof technique that unifies, simplifies, and strengthens two previously competing approaches -- one based on escape directions and the other based on the inexistence of counterexample -- to provide sharp global guarantees in the noisy overparameterized regime. We show, once local minima have been converted into global minima through slight overparameterization, that near-second-order points achieve the same minimax-optimal recovery bounds (up to small constant factors) as significantly more expensive convex approaches. Our results are sharp with respect to the noise level and the solution accuracy, and hold for both the symmetric parameterization $XX^{T}$, as well as the asymmetric parameterization $UV^{T}$ under a balancing regularizer; we demonstrate that the balancing regularizer is indeed necessary.