matrix eigendecomposition

This work addresses the challenge of efficiently computing leading eigenvectors in dynamic graphs, where frequent updates to the adjacency or Laplacian matrix render traditional eigendecomposition methods computationally prohibitive. To overcome this limitation, the authors propose a fast spectral embedding update framework based on Rayleigh-Ritz projection. By leveraging eigenvector perturbation analysis, the method constructs a low-dimensional approximate invariant subspace that preserves high approximation accuracy while substantially reducing computational and memory costs. Experimental results demonstrate that the proposed approach outperforms existing techniques in both the quality of leading eigenvector approximation and performance on downstream tasks—such as influential node identification and node clustering—offering a compelling balance between efficiency and accuracy.

In “inaccessible” scenarios—such as distributed learning or data-constrained settings—massive kernel and covariance matrices cannot be explicitly constructed nor support matrix-vector multiplication. Method: This paper introduces “free decompression,” a novel spectral estimation paradigm grounded in free probability theory. It samples only a minimal principal submatrix from the original matrix, estimates its empirical spectral density, and reconstructs the global eigenvalue distribution via free convolution modeling and spectral extrapolation. Contribution/Results: Unlike conventional methods relying on implicit matrix-vector oracle access, free decompression requires no such oracle. On synthetic data, it approaches theoretical limits; on real-world datasets, it accurately recovers the full-matrix empirical spectrum using extremely small submatrices. This significantly enhances both the feasibility and accuracy of spectral estimation for ultra-large-scale matrices.

Parameter estimation in high-dimensional structured generalized linear models suffers from low efficiency, particularly under realistic design matrices exhibiting anisotropy and strong correlations. Method: This paper introduces a novel spectral estimation framework based on Approximate Message Passing (AMP). Contribution/Results: We provide the first exact asymptotic characterization of spectral estimators under correlated Gaussian designs. We identify a universally optimal covariance-adaptive preprocessing strategy, partially resolving a long-standing conjecture on optimal spectral estimation for rotationally invariant models. Theoretically and empirically, our approach substantially reduces sample complexity and achieves provably statistically optimal estimation accuracy—outperforming existing heuristic methods on canonical designs from computational imaging and genomics.

This work investigates the impact of matrix concatenation operations on singular value spectra, aiming to bridge a theoretical gap in the structural stability of SVDs under concatenation. Addressing the central question—“How are the singular values of a concatenated matrix determined by those of its constituent submatrices?”—we extend Weyl’s inequality to block-wise concatenation for the first time, establishing a quantitative analytical framework grounded in matrix perturbation theory and norm inequalities. We derive computable upper bounds on singular value deviations and rigorously prove that dominant singular values remain stable when the operator norms of the submatrices are comparable. This result provides theoretical guarantees and principled design guidance for low-rank approximation, robust matrix clustering, and compression algorithms.

This paper addresses the weak theoretical foundations of matrix decomposition in machine learning by systematically constructing a self-consistent, comprehensive, and modern-application-oriented pedagogical framework. Methodologically, it grounds the exposition in numerical linear algebra and matrix analysis, unifying classical decompositions—including LU, QR, SVD, and block triangular factorizations—while integrating numerical stability analysis and Hermitian/Hilbert space theory. Crucially, it bridges traditional numerical analysis with deep learning’s backpropagation setting, emphasizing differentiability and computational robustness of decompositions in algorithm design and model optimization. The primary contribution is a compact, dual-purpose (teaching and research) knowledge system that fills critical gaps in both theoretical coherence and machine-learning relevance present in existing literature, thereby providing rigorous mathematical foundations for high-dimensional data modeling and efficient training.

This work addresses the high computational cost of computing multiple nonlinear eigenvectors in nonlinear spectral clustering by proposing a direct multiway spectral clustering algorithm based on the p-norm (with p ∈ (1,2]). The algorithm is implemented for the first time within the C++ GraphBLAS framework, unifying its core operations into sparse linear algebraic expressions. By integrating shared-memory parallelism and p-norm-specific optimizations, the approach achieves substantial gains in computational efficiency. Experimental results on a large-scale graph with 8 million nodes and 48 million edges demonstrate excellent strong scaling performance, while the clustering quality surpasses that of existing methods in terms of balanced graph cut metrics.

This work addresses the gap between algorithmic prototypes and efficient implementations in scientific research by proposing a lightweight approach to translate statistical and machine learning algorithms—such as kernel ridge regression and stochastic gradient descent matrix factorization—from mathematical formulations into readable, high-performance C++ code. Leveraging the Eigen template library for core linear algebra operations—including kernel matrix construction, regularized solvers, and vectorized updates—the implementation seamlessly integrates into the Python ecosystem via pybind11, enabling efficient interoperability with NumPy arrays. The project provides concise, reproducible code examples that encapsulate common computational patterns in research, significantly lowering the barrier for researchers to adopt C++ for high-performance development while balancing performance, readability, and usability.

This work addresses the lack of efficient solutions for computing a large batch of small-scale singular value decompositions (SVDs) on GPUs. The authors propose a GPU-accelerated batched SVD solver based on the one-sided Jacobi algorithm, co-designed with hardware architecture to exploit fine-grained parallelism, optimize memory access patterns, and support multiple floating-point precisions. Implemented on both NVIDIA and AMD GPU platforms, the solver demonstrates exceptional robustness and scalability across diverse matrix shapes, conditioning numbers, and precision configurations. Experimental results show that the proposed method significantly outperforms existing vendor-provided libraries and open-source solvers in terms of computational performance while maintaining numerical reliability.

Learning the spectral decomposition of the Laplacian operator from unstructured high-dimensional data—such as 3D point clouds or image manifolds—typically requires explicit discretization, mesh construction, or solving eigenvalue problems, limiting scalability and applicability to unknown geometries. Method: We propose an end-to-end framework that jointly learns the implicit spectral basis, eigenvalues, and density-induced metric of the Laplacian directly from raw data—without constructing differential operators, discretizing domains, or solving eigenproblems. Leveraging optimal approximation theory, we parameterize the operator and its spectrum via neural networks, minimizing reconstruction error under a probe function distribution. Contribution/Results: The method is fully unsupervised, mesh-free, dimension-agnostic, and geometry-agnostic. Experiments demonstrate that the learned spectral basis exhibits Laplacian-like properties—yielding interpretable, generalizable, and scalable representations across diverse unstructured datasets, including high-dimensional settings.

This work addresses the computational inefficiency of traditional spectral graph partitioning methods, which rely on computing the Fiedler vector via eigenvalue decomposition—a process that incurs prohibitive time and memory costs for large-scale graphs. To overcome this bottleneck, the paper proposes, for the first time, a lightweight neural network architecture that directly approximates the Fiedler vector without performing explicit eigendecomposition. By integrating the structural properties of the graph Laplacian with principles from spectral graph theory, the method achieves partitioning quality comparable to classical spectral bisection across multiple large-scale graph datasets, while substantially reducing both computational time and memory consumption. The approach thus offers a scalable and efficient alternative to conventional spectral partitioning techniques.