🤖 AI Summary
This work studies the convergence of power iteration for decomposing random overcomplete tensors (where rank far exceeds dimension). Prior conjectures suggested logarithmic convergence; we establish the first tight polynomial lower bound on the required number of iterations, proving that polynomial—rather than logarithmic—steps are necessary. Methodologically, we introduce a novel analytical framework based on Gaussian conditioning, which overcomes fundamental limitations of traditional approximate message passing (AMP) analyses—specifically, their reliance on proportional limits and bounded iteration counts. Integrating tools from random matrix theory and rigorous monotonicity analysis, we prove strict monotonic increase of the objective function throughout iterations. Empirical results confirm successful recovery of true components within polynomial time. This work provides the first precise characterization of the computational complexity of power iteration for overcomplete tensor decomposition, thereby establishing a foundational theoretical basis for high-dimensional tensor learning.
📝 Abstract
Tensor decomposition serves as a powerful primitive in statistics and machine learning, and has numerous applications in problems such as learning latent variable models or mixture of Gaussians. In this paper, we focus on using power iteration to decompose an overcomplete random tensor. Past work studying the properties of tensor power iteration either requires a non-trivial data-independent initialization, or is restricted to the undercomplete regime. Moreover, several papers implicitly suggest that logarithmically many iterations (in terms of the input dimension) are sufficient for the power method to recover one of the tensor components. Here we present a novel analysis of the dynamics of tensor power iteration from random initialization in the overcomplete regime, where the tensor rank is much greater than its dimension. Surprisingly, we show that polynomially many steps are necessary for convergence of tensor power iteration to any of the true component, which refutes the previous conjecture. On the other hand, our numerical experiments suggest that tensor power iteration successfully recovers tensor components for a broad range of parameters in polynomial time. To further complement our empirical evidence, we prove that a popular objective function for tensor decomposition is strictly increasing along the power iteration path. Our proof is based on the Gaussian conditioning technique, which has been applied to analyze the approximate message passing (AMP) algorithm. The major ingredient of our argument is a conditioning lemma that allows us to generalize AMP-type analysis to non-proportional limit and polynomially many iterations of the power method.
