cholesky factorization

This work addresses the efficient and high-accuracy solution of dense linear systems and regression problems exhibiting low-rank structure: given a $d imes d$ positive-definite matrix or an $n imes d$ design matrix $A$ with $k$ large singular values, we propose the first randomized algorithm achieving high-probability convergence at nearly optimal computational complexity—$widetilde{O}(d^2 + k^omega)$ for linear systems and $widetilde{O}(mathrm{nnz}(A) + d^2 + k^omega)$ for regression. Our method introduces a unified recursive preconditioning framework comprising three variants, integrating matrix sketching with low-rank updates to adapt to problem structure. It breaks classical time–accuracy trade-offs by relaxing statistical assumptions to a weaker generalized mean condition. Moreover, we deliver the first near-linear-time multiplicative approximation to the nuclear norm of arbitrary dense matrices. Compared to prior approaches, our algorithm achieves significant improvements in accuracy, runtime, and broad applicability across structured dense problems.

Cholesky decomposition of symmetric positive-definite matrices in Gaussian process inference suffers from numerical instability and computational inefficiency. Method: We propose a novel pivoted Cholesky pivoting strategy that integrates entropy maximization from Bayesian nonparametric inference with information gain principles from active learning, enabling an online-updatable, low-overhead diagonal adaptation mechanism. This strategy is synergistically combined with preconditioned iterative solvers and sparse regression frameworks. Contribution/Results: The proposed method significantly improves both uncertainty quantification accuracy and computational speed in sparse Gaussian process inference. Empirical evaluation across diverse benchmark tasks demonstrates consistent superiority over standard pivoted Cholesky baselines, with negligible additional computational overhead. The approach is theoretically grounded in probabilistic inference principles and exhibits strong practical applicability in large-scale GP modeling.

This paper addresses the weak theoretical foundations of matrix decomposition in machine learning by systematically constructing a self-consistent, comprehensive, and modern-application-oriented pedagogical framework. Methodologically, it grounds the exposition in numerical linear algebra and matrix analysis, unifying classical decompositions—including LU, QR, SVD, and block triangular factorizations—while integrating numerical stability analysis and Hermitian/Hilbert space theory. Crucially, it bridges traditional numerical analysis with deep learning’s backpropagation setting, emphasizing differentiability and computational robustness of decompositions in algorithm design and model optimization. The primary contribution is a compact, dual-purpose (teaching and research) knowledge system that fills critical gaps in both theoretical coherence and machine-learning relevance present in existing literature, thereby providing rigorous mathematical foundations for high-dimensional data modeling and efficient training.

This work investigates the impact of matrix concatenation operations on singular value spectra, aiming to bridge a theoretical gap in the structural stability of SVDs under concatenation. Addressing the central question—“How are the singular values of a concatenated matrix determined by those of its constituent submatrices?”—we extend Weyl’s inequality to block-wise concatenation for the first time, establishing a quantitative analytical framework grounded in matrix perturbation theory and norm inequalities. We derive computable upper bounds on singular value deviations and rigorously prove that dominant singular values remain stable when the operator norms of the submatrices are comparable. This result provides theoretical guarantees and principled design guidance for low-rank approximation, robust matrix clustering, and compression algorithms.

This work addresses efficiency bottlenecks in solving large-scale linear systems and approximating matrix norms. We propose a multilevel randomized sketching preconditioned iterative method, integrating Nyström low-rank approximation, sparse random sketching, and multilevel preconditioning. It establishes the first multilevel sketched preconditioning framework grounded in the natural average condition number. Theoretical contributions include: (1) optimal complexity $ ilde{O}(n^2 + d_lambda^omega)$ for solving regularized linear systems; (2) accelerated complexity $ ilde{O}(n^{2.065} + k^omega)$ for systems with $k$ outlying singular values; and (3) Schatten-$p$ norm approximation—particularly the nuclear norm—at $ ilde{O}(n^{2.11})$, improving upon the prior best $ ilde{O}(n^{2.18})$. These advances significantly enhance computational efficiency for key subproblems in applications such as Gaussian process regression.

This work addresses the problem of constructing black-box dynamic system models from data with guaranteed asymptotic stability. It proposes a novel weight projection method based on the real Schur decomposition, which dynamically projects the quasi-upper-triangular factor of the state matrix onto the nearest stable matrix during training. This approach strictly enforces asymptotic stability in discrete-time state-space neural networks while remaining compatible with backpropagation. By uniquely integrating Schur decomposition with dynamic projection, the method achieves minimal over-parameterization and offers a pre-factorized alternative. Experimental results demonstrate that the model attains accuracy and convergence rates comparable to state-of-the-art stable identification techniques on synthetic linear systems, and achieves efficient training with high accuracy using fewer parameters in stacked architectures on real-world datasets.

This work proposes a systematic framework to enhance the computational efficiency and numerical stability of evaluating high-degree matrix polynomials. Specifically, for polynomial degrees eight and higher, the method generates and validates stable coefficient sets that reduce the number of required matrix multiplications by one compared to the classical Paterson–Stockmeyer scheme. To address instability issues in the original formulation, the authors introduce structural variants and design a reliability metric to assess the expected numerical accuracy of candidate coefficient sets. Nonlinear polynomial systems are solved using variable-precision arithmetic (VPA), and an in-house tool, MatrixPolEval1, enables efficient screening and validation. Applied to matrix exponentials and geometric series, the approach achieves a saving of one matrix multiplication while maintaining comparable numerical accuracy.

This work proposes a novel approach to overcoming the computational complexity bottleneck in matrix multiplication by explicitly exploiting the intrinsic structural properties of tensor decompositions. By designing tensor decompositions with specialized algebraic structures and integrating techniques from algebraic complexity theory with numerical optimization, the study achieves a reduction in the exponent for 6×6 matrix multiplication from 2.8075 to 2.8016, while maintaining a reasonable leading constant. Notably, this result yields an effective exponent below the theoretical lower bound implied by conventional tensor rank considerations and significantly enhances practical algorithmic efficiency. The findings establish a new structured design paradigm for fast matrix multiplication algorithms, offering both theoretical advancement and practical relevance.

This work addresses the Hadamard decomposition problem—recovering two low-rank matrices whose element-wise product approximates a given matrix. By reformulating the problem as a structured matrix factorization with explicit constraints, the authors propose three efficient algorithms: a direct decomposition method based on Manopt, a block projected gradient approach, and a projection-free Riemannian gradient descent scheme. A novel initialization strategy is also introduced to enhance solution accuracy. The proposed methods are particularly well-suited for large-scale sparse data and demonstrate significant improvements over truncated SVD and existing Hadamard decomposition techniques on both synthetic and real-world datasets, confirming their computational efficiency and competitive performance.

Solving large-scale, ill-conditioned, and indefinite sparse linear systems is critical in multiphysics simulation, machine learning, and related fields, yet poses significant challenges in computational efficiency and scalability. This work proposes an optimized sparse direct solver framework that integrates task and data parallelism with low-rank approximation and hierarchical matrix compression techniques. Implemented on heterogeneous high-performance computing platforms, the approach substantially reduces communication overhead and computational complexity while preserving the numerical robustness inherent to direct solvers. The resulting method achieves marked improvements in both strong and weak scalability, delivering a highly efficient, reliable, and scalable linear solver toolchain tailored for modern heterogeneous architectures.