low-rank frame parameterization

This work addresses structured orthogonal dictionary learning, specifically for dictionaries parameterized as a single Householder matrix or a product of multiple Householder matrices. We propose an efficient learning algorithm grounded in Householder reflections, which explicitly enforces the structured orthogonality constraint. The algorithm achieves optimal theoretical computational complexity—O(nd)—while attaining high recovery accuracy. We establish, for the first time, approximate recoverability of Householder dictionaries under the ℓ∞ norm and extend this guarantee to multi-reflection products. Through sample complexity analysis and derivation of ℓ∞ error bounds, we prove that our method matches or surpasses state-of-the-art approaches in estimation accuracy—especially in low-sample regimes—while substantially reducing computational cost. Extensive experiments corroborate both the theoretical guarantees and superior practical performance.

To address the high computational cost of SVD approximation updates in streaming data and the scalability limitations of existing incremental/truncated SVD methods at large truncation ranks, this paper proposes a low-rank update framework based on Bidirectional Diagonal Decomposition (BDD). Our method enables efficient rank-$r$ updates via three key innovations: (1) a compact Householder transformation reducing memory usage by 50%; (2) Givens rotations enabling $O(r^2)$-complexity rank-$r$ updates; and (3) a hybrid sparse-plus-low-rank separation strategy for accurate and scalable matrix approximation. Experiments on recommendation systems and network subspace tracking demonstrate that our approach significantly outperforms LAPACK SVD and state-of-the-art incremental SVD methods—achieving superior accuracy, real-time performance even at high truncation ranks, and balanced throughput–precision trade-offs.

To address numerical instability in block orthogonalization within s-step GMRES, this work introduces— for the first time—the integration of randomized sketching into the block QR orthogonalization process, eliminating the need for classical reorthogonalization while rigorously bounding the overall orthogonality error of basis vectors at the machine-precision level. The method synergistically combines randomized projection with distributed block QR and is implemented within the Trilinos framework to enable GPU acceleration (A100) and large-scale parallel scalability. Experiments on the Perlmutter supercomputer demonstrate substantial improvements in both orthogonalization and overall solver numerical stability, with negligible overhead in execution time. This work establishes a novel orthogonalization paradigm for highly parallel Krylov methods that simultaneously ensures high numerical accuracy and computational efficiency.

This paper addresses the weak theoretical foundations of matrix decomposition in machine learning by systematically constructing a self-consistent, comprehensive, and modern-application-oriented pedagogical framework. Methodologically, it grounds the exposition in numerical linear algebra and matrix analysis, unifying classical decompositions—including LU, QR, SVD, and block triangular factorizations—while integrating numerical stability analysis and Hermitian/Hilbert space theory. Crucially, it bridges traditional numerical analysis with deep learning’s backpropagation setting, emphasizing differentiability and computational robustness of decompositions in algorithm design and model optimization. The primary contribution is a compact, dual-purpose (teaching and research) knowledge system that fills critical gaps in both theoretical coherence and machine-learning relevance present in existing literature, thereby providing rigorous mathematical foundations for high-dimensional data modeling and efficient training.

This work addresses efficiency bottlenecks in solving large-scale linear systems and approximating matrix norms. We propose a multilevel randomized sketching preconditioned iterative method, integrating Nyström low-rank approximation, sparse random sketching, and multilevel preconditioning. It establishes the first multilevel sketched preconditioning framework grounded in the natural average condition number. Theoretical contributions include: (1) optimal complexity $ ilde{O}(n^2 + d_lambda^omega)$ for solving regularized linear systems; (2) accelerated complexity $ ilde{O}(n^{2.065} + k^omega)$ for systems with $k$ outlying singular values; and (3) Schatten-$p$ norm approximation—particularly the nuclear norm—at $ ilde{O}(n^{2.11})$, improving upon the prior best $ ilde{O}(n^{2.18})$. These advances significantly enhance computational efficiency for key subproblems in applications such as Gaussian process regression.

本文研究了一类低参数正交矩阵的黎曼结构，并提出有效算法解决其在深度学习中的计算问题。

This work addresses the problem of efficiently maintaining the rank, a column basis, and a maximum-rank submatrix of a matrix under dynamic updates—either to individual entries or entire columns—and applies these techniques to the dynamic maximum matching problem in graphs. The paper presents the first dynamic algorithm whose update time depends on the current rank \( r \) rather than the matrix dimension \( n \). By integrating sparse update strategies with rank-sensitive complexity analysis, it achieves an amortized update time of \( \tilde{O}(r^{1.405}) \) for single-entry modifications and \( \tilde{O}(r^{1.528} + z) \) for column updates, where \( z \) denotes the number of changed entries. This approach is the first to simultaneously support dynamic maintenance of rank, basis, and maximum-rank submatrix, yielding an edge update time of \( \tilde{O}(|M|^{1.405}) \) for dynamic graph matching and significantly improving upon prior methods.

本文通过构建具有最小支撑的张量基，解决了成对比较理论中加性一致子空间的正交基问题，并提出了新的对数、Saaty和SVD投影公式。

This study addresses the longstanding challenge of computing generalized inverses for non-square Jacobian matrices in automatic differentiation and solving for preimages in affine spaces. To this end, it proposes the Null-A mode, which employs a compositional algorithm to efficiently compute affine preimages of matrix products. This mode supports dynamic variables and non-square Jacobians, extending reverse-mode automatic differentiation to full affine space solutions while leveraging quasi-local algorithms alongside CPU/GPU parallel acceleration. The primary contribution lies in achieving efficient generalized inverse computation for both scalar functions and aggregated array operations, such as convolutions and attention mechanisms. Ultimately, this work overcomes critical bottlenecks in reverse solving for linearized numerical computations, offering a robust framework that significantly broadens the applicability and computational efficiency of automatic differentiation in complex machine learning architectures.

This study addresses the problem of reducing the additive complexity of $3\times3$ matrix multiplication. It proposes a rank-23 matrix multiplication kernel that combines linear programming reductions with sparse basis transformation search to optimize the computational structure, providing machine-verifiable certificates of correctness through exact coefficient expansion. By exploiting alternative bases, the method reduces the number of additions to 51, while achieving a low-complexity computation requiring only 56 additions in standard coordinates. This work presents the first rigorously and formally verified low-additive-complexity algorithm for $3\times3$ matrix multiplication, establishing a reliable benchmark and a novel paradigm for exploring theoretical lower bounds in this domain.