A Random Matrix Approach to Low-Multilinear-Rank Tensor Approximation

2024-02-05

arXiv.org

AI Summary

This paper investigates the detectability and reconstruction of low multilinear-rank signals in high-dimensional spiked tensor models near computational phase transition thresholds. To address key bottlenecks—degraded performance of conventional methods in the critical signal-to-noise ratio (SNR) regime and lack of theoretical convergence guarantees for the Higher-Order Orthogonal Iteration (HOOI) algorithm—the authors systematically apply random matrix theory to analyze the spectral properties of tensor unfoldings. They derive a novel SNR criterion characterizing statistical detectability, precisely quantify the reconstruction error of truncated multilinear singular value decomposition (MLSVD) in the nontrivial regime, and rigorously prove that, in the large-dimensional limit, HOOI converges to the global optimum in a single iteration. These results establish tight theoretical bounds for low-rank tensor estimation and provide provably efficient algorithmic guarantees.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This work presents a comprehensive understanding of the estimation of a planted low-rank signal from a general spiked tensor model near the computational threshold. Relying on standard tools from the theory of large random matrices, we characterize the large-dimensional spectral behavior of the unfoldings of the data tensor and exhibit relevant signal-to-noise ratios governing the detectability of the principal directions of the signal. These results allow to accurately predict the reconstruction performance of truncated multilinear SVD (MLSVD) in the non-trivial regime. This is particularly important since it serves as an initialization of the higher-order orthogonal iteration (HOOI) scheme, whose convergence to the best low-multilinear-rank approximation depends entirely on its initialization. We give a sufficient condition for the convergence of HOOI and show that the number of iterations before convergence tends to $1$ in the large-dimensional limit.

Problem

• Tensor Decomposition
• Low Multilinear Rank
• Computational Limits

Innovation

• Random Matrix Theory
• Multilinear Singular Value Decomposition
• High-Order Orthogonal Iteration