sparse matrix assembly

This paper addresses the efficient solution of integer linear systems $Ax = b$ over finite-dimensional linear spaces. We present the first randomized solver for polynomially bounded integer inputs that is independent of the matrix condition number. Our algorithm operates over the rationals, achieving $ ilde{O}(n^2 cdot ext{nnz}(A))$ time complexity and $O(n log n)$ workspace—breaking the classical accuracy–space trade-off. The core techniques integrate bit-complexity optimization, near-linear-space data structures, and a synergistic design of exact integer arithmetic with approximation theory. As a result, our method provides a unified, highly efficient primitive for fundamental numerical tasks—including linear regression, linear programming, and eigenvalue/singular value decomposition—delivering exact or high-accuracy approximate solutions in polynomial time while using near-linear space. This significantly expands the tractability frontier for large-scale numerical linear algebra problems.

This project addresses efficient randomized algorithms for three fundamental matrix computations under resource constraints: (1) low-rank positive semidefinite (PSD) matrix approximation, aiming for exact low-rank reconstruction with minimal element accesses; (2) estimation of implicit matrix properties—trace, diagonal entries, and row norms—using only matrix-vector products; and (3) numerically robust floating-point solutions to overdetermined linear least-squares problems. Methodologically, we propose a randomized pivoted Cholesky decomposition to drastically reduce element accesses in PSD approximation; introduce a novel leave-one-out framework for property estimation, achieving optimal sample complexity; and design the first randomized least-squares solver that simultaneously attains asymptotic acceleration—O(nd log d) time for n × d matrices—and numerical backward stability, matching the accuracy of direct methods. These contributions advance both theoretical guarantees and practical efficiency in large-scale, memory- or query-constrained numerical linear algebra.

Sparse-dense and sparse-sparse matrix multiplication is a fundamental operator in graph analytics, graph neural networks, and biological sequence alignment—often requiring execution over arbitrary algebraic semirings (including heterogeneous algebras). Existing systems lack unified support for such generalized algebraic semantics. This paper introduces the first unified computational framework for sparse matrix multiplication under generic algebraic semantics. Grounded in semiring theory, it proposes an extensible sparse tensor algebra model supporting user-defined scalar operations, heterogeneous input domains, and operator fusion; it further leverages high-performance sparse storage formats and compiler optimizations. Experimental evaluation across machine learning, computational biology, and scientific computing demonstrates substantial improvements in both expressive power and computational efficiency. The framework establishes a scalable, formally verifiable infrastructure for generalized linear algebra.

This work addresses the high computational cost of sparse matrix ordering in linear systems arising from triangular meshes. We propose an efficient ordering algorithm that accelerates nested dissection and rapidly constructs elimination trees by moderately relaxing constraints on partition balance and optimality, while integrating local block ordering with a separator-based quotient graph compression strategy. The method innovatively trades a controlled degradation in ordering quality for substantial computational speedup, preserving the fill-reducing structure required for Cholesky factorization while bypassing its most expensive phases. When integrated into commercial CPU/GPU sparse Cholesky solvers, our approach significantly reduces ordering time in graphics applications and achieves up to a 6.27× improvement in overall solver performance.

Computing exact leverage scores for Kronecker-product structured matrices in large-scale least-squares problems is computationally prohibitive, while existing approximation methods incur statistical bias and high overhead. Method: We propose the first efficient exact leverage score algorithm tailored to Kronecker-structured matrices. Leveraging the inherent tensor structure, our method designs a near-linear-time framework for exact leverage score computation and sampling—bypassing costly full-matrix SVD or biased sketching approximations. Contribution/Results: Theoretically and empirically, our algorithm achieves significantly lower sampling error than state-of-the-art approximate methods (e.g., FJLT- or CountSketch-accelerated approaches), while maintaining substantially lower time complexity than full SVD. This work establishes the first scalable, exact, and efficient leverage score sampling scheme for Kronecker-structured matrices, enabling improved structured random projections and large-scale regression.

Solving large-scale, ill-conditioned, and indefinite sparse linear systems is critical in multiphysics simulation, machine learning, and related fields, yet poses significant challenges in computational efficiency and scalability. This work proposes an optimized sparse direct solver framework that integrates task and data parallelism with low-rank approximation and hierarchical matrix compression techniques. Implemented on heterogeneous high-performance computing platforms, the approach substantially reduces communication overhead and computational complexity while preserving the numerical robustness inherent to direct solvers. The resulting method achieves marked improvements in both strong and weak scalability, delivering a highly efficient, reliable, and scalable linear solver toolchain tailored for modern heterogeneous architectures.

This work addresses the inefficiency of large-scale sparse tensor computations by proposing a tensor-relational hybrid computation model. The model introduces a novel case-sensitive Einstein summation (EinSum) notation that automatically decomposes sparse tensor expressions into a combination of relational algebra operations and dense tensor computations: sparsity is efficiently managed by a relational database system, while dense numerical kernels handle the compute-intensive parts. This approach enables co-optimization of sparse structure handling and high-performance computing, significantly improving overall performance without sacrificing sparse processing efficiency. It represents the first end-to-end automatic compiler capable of translating high-level EinSum expressions into efficient hybrid execution plans.

This work addresses the problem of efficiently constructing an approximate matrix with a prescribed binary sparsity pattern using only black-box access via matrix-vector product queries. The authors introduce the degeneracy of the sparsity pattern as a unified and tight measure of query complexity, overcoming limitations inherent in traditional graph coloring approaches. Leveraging this notion, they propose an adaptive querying strategy together with a polynomial-time algorithm that achieves a near-optimal approximation using only Õ(degen(S)) queries while avoiding computational bottlenecks. Furthermore, they establish an information-theoretic lower bound of Ω(degen(S)) on the query complexity for any sparsity pattern S, thereby proving the optimality of their approach.

This work addresses the challenge of unifying diverse structured sparsity patterns for efficient model compression and acceleration. The authors propose S³, an algebraic framework that formally integrates three core components—View (tensor reshaping), Block (atomic pruning units), and Scope (sparsity decision range)—to express a wide spectrum of sparsity patterns, ranging from fine-grained N:M sparsity to coarse-grained channel pruning, within a single formalism. Notably, S³ enables cross-tensor collaborative sparsification. Building upon this framework, the authors incorporate Optimal Brain Damage and Surgeon algorithms to develop structured variants of OBS/OBD. These methods significantly outperform current state-of-the-art second-order heuristic approaches in terms of output reconstruction accuracy.

This work addresses the straggler problem in distributed matrix chain multiplication, which severely limits performance, and tackles the challenge that existing coding strategies face—namely, rapidly escalating computational and storage overheads when extended to multi-matrix settings. The paper introduces multivariate polynomial coding to this domain for the first time, proposing a novel encoding scheme that significantly reduces storage requirements while preserving fault tolerance. By deliberately introducing a controlled increase in computational load, the method effectively balances computational and storage efficiency. This approach overcomes the scalability limitations inherent in conventional univariate coding schemes for long matrix chains, offering a more practical and efficient solution for large-scale distributed linear algebra computations.