Faster Linear Algebra Algorithms with Structured Random Matrices

2025-08-28

AI Summary

Structured random matrices lack a unified theoretical analysis and a general design framework in randomized linear algebra. Method: This paper introduces the “Oblivious Subspace Injection” (OSI) property, establishing the first decoupled abstract analytical framework that separates correctness proofs of algorithms from instantiation-specific verification. Contribution/Results: We prove that sparse random matrices, random triangular transforms, and tensor-product-structured matrices all satisfy OSI, thereby unifying their dimensionality-reduction fidelity guarantees for tasks such as low-rank approximation and least-squares regression. Leveraging this framework, we design accelerated algorithms with near-optimal time complexity. Empirical evaluation on synthetic datasets and scientific computing benchmarks confirms both efficiency and practical utility.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

To achieve the greatest possible speed, practitioners regularly implement randomized algorithms for low-rank approximation and least-squares regression with structured dimension reduction maps. Despite significant research effort, basic questions remain about the design and analysis of randomized linear algebra algorithms that employ structured random matrices. This paper develops a new perspective on structured dimension reduction, based on the oblivious subspace injection (OSI) property. The OSI property is a relatively weak assumption on a random matrix that holds when the matrix preserves the length of vectors on average and, with high probability, does not annihilate any vector in a low-dimensional subspace. With the OSI abstraction, the analysis of a randomized linear algebra algorithm factors into two parts: (i) proving that the algorithm works when implemented with an OSI; and (ii) proving that a given random matrix model has the OSI property. This paper develops both parts of the program. First, it analyzes standard randomized algorithms for low-rank approximation and least-squares regression under the OSI assumption. Second, it identifies many examples of OSIs, including random sparse matrices, randomized trigonometric transforms, and random matrices with tensor product structure. These theoretical results imply faster, near-optimal runtimes for several fundamental linear algebra tasks. The paper also provides guidance on implementation, along with empirical evidence that structured random matrices offer exemplary performance for a range of synthetic problems and contemporary scientific applications.

Problem

• Designing faster randomized linear algebra algorithms with structured matrices
• Analyzing algorithms using the Oblivious Subspace Injection (OSI) property
• Identifying practical OSI examples for efficient low-rank approximation

Innovation

• Oblivious subspace injection property for structured matrices
• Analyzing algorithms with OSI assumption for efficiency
• Identifying OSI examples including sparse trigonometric transforms