Technology Category
Search and Optimization: Non-convex OptimizationMachine Learning: Matrix & Tensor MethodsReasoning under Uncertainty: Stochastic Optimization
Non-convex matrix sensing: Breaking the quadratic rank barrier in the sample complexity
🤖 AI Summary
This work addresses the bottleneck in nonconvex matrix sensing where sample complexity scales quadratically with rank (O(r²)). We propose an efficient factorization-based gradient descent method to recover low-rank positive semidefinite matrices from Gaussian linear measurements. By integrating spectral initialization with a novel probabilistic decoupling analysis, we achieve—for the first time in a nonconvex framework—a sample complexity with only linear rank dependence, O(r), matching the optimal rate of nuclear norm minimization. Theoretically, under Ω(rdκ²) measurements, our algorithm converges linearly to the ground-truth matrix. This result substantially improves the sample efficiency and theoretical guarantees of nonconvex optimization for matrix sensing, breaking the long-standing O(r²) rank-squared barrier inherent in prior nonconvex approaches.
Technology Category
Application Category
📝 Abstract
For the problem of reconstructing a low-rank matrix from a few linear measurements, two classes of algorithms have been widely studied in the literature: convex approaches based on nuclear norm minimization, and non-convex approaches that use factorized gradient descent. Under certain statistical model assumptions, it is known that nuclear norm minimization recovers the ground truth as soon as the number of samples scales linearly with the number of degrees of freedom of the ground-truth. In contrast, while non-convex approaches are computationally less expensive, existing recovery guarantees assume that the number of samples scales at least quadratically with the rank $r$ of the ground-truth matrix. In this paper, we close this gap by showing that the non-convex approaches can be as efficient as nuclear norm minimization in terms of sample complexity. Namely, we consider the problem of reconstructing a positive semidefinite matrix from a few Gaussian measurements. We show that factorized gradient descent with spectral initialization converges to the ground truth with a linear rate as soon as the number of samples scales with $ Omega (rdkappa^2)$, where $d$ is the dimension, and $kappa$ is the condition number of the ground truth matrix. This improves the previous rank-dependence in the sample complexity of non-convex matrix factorization from quadratic to linear. Our proof relies on a probabilistic decoupling argument, where we show that the gradient descent iterates are only weakly dependent on the individual entries of the measurement matrices. We expect that our proof technique is of independent interest for other non-convex problems.
Problem
Research questions and friction points this paper is trying to address.
Reducing sample complexity for non-convex matrix sensing
Breaking quadratic rank barrier in matrix reconstruction
Achieving linear sample complexity with non-convex methods
Innovation
Methods, ideas, or system contributions that make the work stand out.
Non-convex gradient descent with spectral initialization
Linear sample complexity scaling Ω(rdκ²)
Probabilistic decoupling argument for weak dependencies
🔎 Similar Papers
No similar papers found.
