approximate message passing

In large-scale distributed dynamic computing, each iteration accesses only partial data, causing information decay and estimation inaccuracy under linear operators. To address this, we propose an Approximate Message Passing (AMP) framework integrating autoregressive memory mechanisms with orthogonal projection. Our method introduces the first general AMP theoretical model applicable to arbitrary linear operators; designs a memory-augmented iterative update scheme and a dedicated orthogonal projection algorithm compatible with nonseparable denoisers; and leverages state evolution and Gaussian process limit theory to derive high-dimensional asymptotically exact statistical characterizations. Under rank-one sparse signals, Gaussian noise, and row-wise data updates, the algorithm achieves asymptotically optimal recovery. Theoretical predictions align closely with numerical experiments, validating both analytical tractability and robustness of the iterative process.

This work addresses the high-dimensional estimation of rank-one signal matrices corrupted by rotationally invariant noise. To overcome the performance limitations of classical PCA and existing iterative methods, we propose a novel class of Approximate Message Passing (AMP) algorithms. For the first time, we establish a rigorous high-dimensional dynamical characterization of these algorithms under rotationally invariant noise. By jointly modeling the spectral structure of the noise and the signal prior, we derive the optimal iterative denoiser and an asymptotically optimal estimator. Theoretically, our algorithm achieves the information-theoretic limit—the minimal asymptotic estimation error—in the spiked matrix model, substantially outperforming PCA and state-of-the-art methods. Moreover, it enjoys provable convergence and asymptotic optimality guarantees.

This work investigates the non-asymptotic behavior of approximate message passing (AMP) algorithms in high-dimensional generalized Gaussian random matrix models with arbitrary variance profiles, focusing on entrywise precise characterization under non-i.i.d. observations. We propose the first high-dimensional, non-asymptotic, leave-one-out (LOO) AMP representation framework, departing from classical low-dimensional state evolution paradigms. Leveraging LOO analysis, integration by parts, concentration inequalities, and inductive arguments, we rigorously derive the finite-sample distribution of ridge estimators under heterogeneous covariates and establish an explicit relationship between the vector-valued effective noise and regularization parameters—governed by a high-dimensional system of equations. Our key contribution is the first entrywise exact modeling of AMP under non-i.i.d. Gaussian designs, providing a unified non-asymptotic theoretical foundation for high-dimensional regularized estimation.

In large-scale Gaussian process (GP) hyperparameter optimization, iterative linear solvers—such as conjugate gradient (CG)—induce inefficiency in computing gradients of the marginal likelihood due to repeated, costly matrix-vector operations. Method: We propose a general-purpose optimization framework integrating pathwise gradient estimation, solver warm-starting, and budget-aware early stopping. The framework is agnostic to the underlying iterative solver and supports CG, alternating projections, and stochastic gradient descent. Contribution/Results: Our approach substantially alleviates the accuracy–efficiency trade-off in gradient estimation. Experiments demonstrate up to 72× speedup over standard CG when solving to full convergence. Under early stopping, the average residual norm drops to one-seventh of that achieved by baseline methods, significantly shortening hyperparameter optimization time while preserving convergence stability and gradient estimation accuracy.

Parameter estimation in high-dimensional structured generalized linear models suffers from low efficiency, particularly under realistic design matrices exhibiting anisotropy and strong correlations. Method: This paper introduces a novel spectral estimation framework based on Approximate Message Passing (AMP). Contribution/Results: We provide the first exact asymptotic characterization of spectral estimators under correlated Gaussian designs. We identify a universally optimal covariance-adaptive preprocessing strategy, partially resolving a long-standing conjecture on optimal spectral estimation for rotationally invariant models. Theoretically and empirically, our approach substantially reduces sample complexity and achieves provably statistically optimal estimation accuracy—outperforming existing heuristic methods on canonical designs from computational imaging and genomics.

This work addresses the limited applicability of Bayes-GAMP in complex-valued settings and under arbitrary nonlinear observation models, where the required posterior denoisers typically lack closed-form expressions. By characterizing the message-passing dynamics of GAMP through state evolution, the authors introduce a score-matching framework to train a neural network that replaces analytically intractable denoisers. This approach enables approximate Bayesian-optimal inference using only forward evaluations of the observation mapping, without requiring explicit knowledge of its functional form. Notably, it is the first method to support nearly arbitrary complex-valued nonlinear observation models, eliminating dependence on closed-form denoisers or explicit model specifications. Under ideal training conditions, the proposed algorithm asymptotically approaches the performance of the true Bayes-GAMP, substantially broadening the scope of GAMP-based inference.

This work establishes, for the first time, a direct theoretical connection between Approximate Message Passing (AMP) and the Convex Gaussian Minimax Theorem (CGMT) in the proportional high-dimensional regime. By analyzing the duality consistency between the primary and auxiliary optimization problems in CGMT, the authors directly derive the AMP fixed-point equations—including the Onsager correction term—and reveal that the Gaussian vector appearing in the auxiliary problem corresponds precisely to the Gaussian perturbation inherent in AMP iterations. This unified framework naturally yields the scalar variance evolution equations for both AMP and its generalized form (GAMP), and demonstrates their equivalence in settings such as regularized linear regression and M-estimation. The results provide a rigorous theoretical foundation and novel insights for designing AMP-type algorithms in non-standard scenarios.

This study addresses the failure of classical state evolution in approximate message passing (AMP) when employing discontinuous denoisers. We propose a modified Onsager coefficient incorporating a jump-density-weighted boundary term. Through Lipschitz approximation analysis and iterative optimization over Gaussian matrices, this work overcomes traditional smoothness assumptions and rigorously establishes that the modified AMP still adheres to state evolution theory. Simulations demonstrate that the proposed coefficient enables hard-thresholding AMP to approach Bayes-optimal performance, effectively correcting both the analytical discrepancies and algorithmic failures induced by standard coefficients in non-smooth settings.

This work addresses the problem of recovering high-accuracy approximate message passing (AMP) solutions from a spiked matrix model corrupted by adversarial sparse perturbations. The authors propose an efficient algorithm that integrates spectral preprocessing with robust spectral initialization, enabling accurate reconstruction using only the contaminated observation matrix. For the first time, they theoretically establish that AMP and several of its variants—including sparse PCA, non-negative PCA, and ℤ₂ synchronization—are inherently robust when equipped with appropriate initialization. Specifically, under adversarial perturbations affecting an εn×εn submatrix, the recovered vector achieves an error of Õ(√ε) relative to the ideal AMP output, providing a rigorous theoretical guarantee of robustness.

This study addresses the long-standing open conjecture posed by Maillard et al. regarding the lack of a rigorous proof for the quantitative universality of the Approximate Message Passing (AMP) algorithm under rank-one quadratic sensing matrices. To resolve this, we propose a column-wise Lindeberg replacement and leave-one-out analysis framework, integrating smooth spectral expansions with higher-order moment concentration inequalities to systematically compare general ensembles against Gaussian ensembles and quantify their error discrepancies. This work provides the first rigorous proof of AMP’s quantitative universality in this setting, thereby resolving the aforementioned conjecture. We demonstrate that differences in statistics such as the normalized signal overlap are of order O(d^{-1/2}), establishing universal predictions for the mean squared error and spectral distribution. These findings offer solid theoretical guarantees for AMP algorithms in quadratic regression.