Orthogonal Approximate Message Passing Algorithms for Rectangular Spiked Matrix Models with Rotationally Invariant Noise

2026-02-03

AI Summary

This work proposes an Orthogonal Approximate Message Passing (OAMP) algorithm for signal estimation in rectangular spiked matrix models corrupted by general rotationally invariant noise. The method establishes a flexible OAMP framework applicable to arbitrary rotationally invariant noise distributions, derives an optimal variant that minimizes the mean squared error at each iteration, and develops a rigorous state evolution theory to precisely characterize its high-dimensional dynamics. In the special case of Gaussian noise, the fixed points of the proposed algorithm coincide with those of standard AMP. Under more general rotationally invariant noise, the algorithm achieves Bayes-optimal performance under certain conditions, demonstrating its potential for statistical optimality.

Technology Category

Machine Learning: OptimizationSearch and Optimization: Non-convex OptimizationReasoning under Uncertainty: Stochastic Optimization

Application Category

Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphsSystems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deploymentsUser Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

We propose an orthogonal approximate message passing (OAMP) algorithm for signal estimation in the rectangular spiked matrix model with general rotationally invariant (RI) noise. We establish a rigorous state evolution that exactly characterizes the high-dimensional dynamics of the algorithm. Building on this framework, we derive an optimal variant of OAMP that minimizes the predicted mean-squared error at each iteration. For the special case of i.i.d. Gaussian noise, the fixed point of the proposed OAMP algorithm coincides with that of the standard AMP algorithm. For general RI noise models, we conjecture that the optimal OAMP algorithm is statistically optimal within a broad class of iterative methods, and achieves Bayes-optimal performance in certain regimes.

Problem

• spiked matrix model
• rotationally invariant noise
• signal estimation
• rectangular matrix
• orthogonal approximate message passing

Innovation

• Orthogonal Approximate Message Passing
• State Evolution
• Rotationally Invariant Noise
• Spiked Matrix Model
• Bayes-optimal Estimation