matrix square-root computation

This work investigates the identification of shared and view-specific latent structures in multi-view data, focusing on scenarios where symmetric positive definite matrices exhibit common eigendirections. By analyzing the spectral properties of matrix interpolants of the form \(A^{1-x}B^x\), the study establishes that the operator norm displays strict or approximate log-linear behavior if and only if the matrices share dominant eigenvectors. Leveraging this theoretical insight, the authors develop a multi-manifold learning framework that integrates complex matrix interpolation, spectral analysis, and singular vector alignment to effectively disentangle common and specific latent variables across views. This approach provides a novel theoretical foundation and a practical methodology for multi-view representation learning.

This work addresses stochastic optimization on the generalized Stiefel manifold—defined as the set of matrices $X$ satisfying $X^ op B X = I_p$—which arises in canonical correlation analysis (CCA), independent component analysis (ICA), and generalized eigenvalue problems (GEVP). Conventional Riemannian approaches require exact knowledge of $B$ and enforce the constraint via costly retraction or projection at each iteration. We propose the first projection-free stochastic algorithm: it operates using only unbiased stochastic estimates of $B$, employs a constrained-stable stochastic gradient update dominated by matrix multiplications, and converges almost surely to constrained critical points. Theoretically, its convergence rate matches that of full-information Riemannian methods under standard assumptions. Empirically, each iteration incurs significantly lower computational cost while achieving comparable accuracy.

Parameter estimation in high-dimensional structured generalized linear models suffers from low efficiency, particularly under realistic design matrices exhibiting anisotropy and strong correlations. Method: This paper introduces a novel spectral estimation framework based on Approximate Message Passing (AMP). Contribution/Results: We provide the first exact asymptotic characterization of spectral estimators under correlated Gaussian designs. We identify a universally optimal covariance-adaptive preprocessing strategy, partially resolving a long-standing conjecture on optimal spectral estimation for rotationally invariant models. Theoretically and empirically, our approach substantially reduces sample complexity and achieves provably statistically optimal estimation accuracy—outperforming existing heuristic methods on canonical designs from computational imaging and genomics.

To address the poor scalability of conventional real sum-of-squares (RSOS) methods—which rely on large-scale semidefinite programming (SDP) for computing lower bounds on the minimum of real homogeneous polynomials over the unit sphere—this paper proposes a purely spectral (non-SDP) convergent hierarchy. The key innovation is the first rigorous reduction of real spherical optimization to Hermitian optimization, enabling the construction of a sequence of lower bounds via minimal eigenvalue computations alone, within the Hermitian sum-of-squares (HSOS) framework. This approach naturally extends to estimating the spectral norm of real tensors, thereby opening a new pathway for spectral methods in general constrained real optimization. Numerical experiments and asymptotic analysis demonstrate substantial improvements over RSOS and other baseline methods; moreover, the framework yields a computable, convergent hierarchy for the spectral norm.

This work investigates the impact of matrix concatenation operations on singular value spectra, aiming to bridge a theoretical gap in the structural stability of SVDs under concatenation. Addressing the central question—“How are the singular values of a concatenated matrix determined by those of its constituent submatrices?”—we extend Weyl’s inequality to block-wise concatenation for the first time, establishing a quantitative analytical framework grounded in matrix perturbation theory and norm inequalities. We derive computable upper bounds on singular value deviations and rigorously prove that dominant singular values remain stable when the operator norms of the submatrices are comparable. This result provides theoretical guarantees and principled design guidance for low-rank approximation, robust matrix clustering, and compression algorithms.

This work addresses the optimization challenges in low-rank matrix Hadamard product modeling, which arise from row- and column-scaling symmetries. The authors formulate the problem as an optimization over a Riemannian quotient manifold and introduce a block-diagonal Riemannian metric derived from the pullback of the Frobenius inner product. This metric is invariant under the full symmetry group of scaling transformations. They further devise a parameter-free Gauss–Newton step-size strategy that leverages this geometric structure. The resulting algorithm achieves linear computational complexity with respect to the number of observed entries and demonstrates both efficiency and effectiveness on synthetic and real-world datasets.

This work addresses the challenge that real-world data in generalized linear models often violate the independent and identically distributed (i.i.d.) assumption. Under the relaxed assumption that the design matrix is orthogonally invariant—meaning its singular vectors are uniformly distributed while singular values remain arbitrary—the paper proposes an efficient parameter estimation method combining optimal spectral initialization with Approximate Message Passing (AMP). The proposed approach achieves the information-theoretically optimal sample complexity for weak recovery and attains the fundamental lower bound on estimation error, thereby extending beyond the classical i.i.d. Gaussian design setting. Rigorous theoretical analysis provides strong performance guarantees, and numerical experiments confirm both the algorithm’s effectiveness and the accuracy of the theoretical predictions on orthogonally invariant as well as more general correlated data.

This work addresses the Hadamard decomposition problem—recovering two low-rank matrices whose element-wise product approximates a given matrix. By reformulating the problem as a structured matrix factorization with explicit constraints, the authors propose three efficient algorithms: a direct decomposition method based on Manopt, a block projected gradient approach, and a projection-free Riemannian gradient descent scheme. A novel initialization strategy is also introduced to enhance solution accuracy. The proposed methods are particularly well-suited for large-scale sparse data and demonstrate significant improvements over truncated SVD and existing Hadamard decomposition techniques on both synthetic and real-world datasets, confirming their computational efficiency and competitive performance.

This work addresses the failure of conventional quadratic forms involving precision matrices in high-dimensional settings where the feature dimension \(p\) exceeds the sample size \(n\), a regime plagued by rank deficiency and excessive complexity. The authors propose a novel framework based on spectral moment representations and constrained optimization, which, under mild moment conditions, achieves the first consistent estimator for such quadratic forms. This approach overcomes fundamental limitations of existing methods and provides a unified treatment for a broad class of high-dimensional statistics. Both theoretical analysis and numerical simulations demonstrate that the proposed method substantially outperforms traditional approaches in applications including optimal portfolio Sharpe ratio estimation and multiple correlation coefficients in regression, effectively resolving the estimation breakdown inherent to the \(p > n\) setting.

Existing Riemannian metrics for symmetric positive-definite (SPD) operators lack a unified geometric framework in infinite-dimensional Hilbert spaces, leading to theoretical limitations and poor geometric stability in high-dimensional or functional data analysis. Method: We propose the Generalized Alpha-Procrustes Riemannian metric—a novel framework integrating normalized Hilbert–Schmidt operators, an extended Mahalanobis norm, and learnable regularization parameters. Contribution/Results: This is the first unified infinite-dimensional formulation encompassing generalized Bures–Wasserstein, Log-Hilbert–Schmidt (a natural extension of Log-Euclidean geometry), and Wasserstein-type distances. It resolves fundamental structural deficiencies of prior metrics in infinite dimensions, substantially improving geometric stability and scale robustness for comparing high-dimensional and functional data. Extensive experiments across multi-scale benchmarks demonstrate consistent superiority over state-of-the-art methods. The framework provides a theoretically rigorous yet practically effective Riemannian foundation for machine learning and functional data analysis.