leverage score estimation

Computing exact leverage scores for Kronecker-product structured matrices in large-scale least-squares problems is computationally prohibitive, while existing approximation methods incur statistical bias and high overhead. Method: We propose the first efficient exact leverage score algorithm tailored to Kronecker-structured matrices. Leveraging the inherent tensor structure, our method designs a near-linear-time framework for exact leverage score computation and sampling—bypassing costly full-matrix SVD or biased sketching approximations. Contribution/Results: Theoretically and empirically, our algorithm achieves significantly lower sampling error than state-of-the-art approximate methods (e.g., FJLT- or CountSketch-accelerated approaches), while maintaining substantially lower time complexity than full SVD. This work establishes the first scalable, exact, and efficient leverage score sampling scheme for Kronecker-structured matrices, enabling improved structured random projections and large-scale regression.

This project addresses efficient randomized algorithms for three fundamental matrix computations under resource constraints: (1) low-rank positive semidefinite (PSD) matrix approximation, aiming for exact low-rank reconstruction with minimal element accesses; (2) estimation of implicit matrix properties—trace, diagonal entries, and row norms—using only matrix-vector products; and (3) numerically robust floating-point solutions to overdetermined linear least-squares problems. Methodologically, we propose a randomized pivoted Cholesky decomposition to drastically reduce element accesses in PSD approximation; introduce a novel leave-one-out framework for property estimation, achieving optimal sample complexity; and design the first randomized least-squares solver that simultaneously attains asymptotic acceleration—O(nd log d) time for n × d matrices—and numerical backward stability, matching the accuracy of direct methods. These contributions advance both theoretical guarantees and practical efficiency in large-scale, memory- or query-constrained numerical linear algebra.

Weighted Low-Rank Approximation (WLRA) seeks a rank-$k$ matrix $XY^ op$ minimizing the weighted Frobenius norm $|W circ (M - XY^ op)|_F$, given a matrix $M$ and a nonnegative weight matrix $W$. This problem is NP-hard and hard to approximate. This paper proposes the first alternating minimization framework for WLRA that simultaneously achieves strong theoretical guarantees and high efficiency. Our method integrates a high-accuracy multi-response regression solver into each alternating update step, enabling approximate yet controllable subproblem solving. Crucially, it preserves global convergence while reducing the per-iteration time complexity from $O(|W|_0 k^2)$ to $O(|W|_0 k)$, where $|W|_0$ denotes the number of nonzero entries in $W$—yielding substantial speedups for sparse weighting patterns. Experiments demonstrate state-of-the-art performance on matrix completion and noise-robust recovery tasks.

This work addresses efficiency bottlenecks in solving large-scale linear systems and approximating matrix norms. We propose a multilevel randomized sketching preconditioned iterative method, integrating Nyström low-rank approximation, sparse random sketching, and multilevel preconditioning. It establishes the first multilevel sketched preconditioning framework grounded in the natural average condition number. Theoretical contributions include: (1) optimal complexity $ ilde{O}(n^2 + d_lambda^omega)$ for solving regularized linear systems; (2) accelerated complexity $ ilde{O}(n^{2.065} + k^omega)$ for systems with $k$ outlying singular values; and (3) Schatten-$p$ norm approximation—particularly the nuclear norm—at $ ilde{O}(n^{2.11})$, improving upon the prior best $ ilde{O}(n^{2.18})$. These advances significantly enhance computational efficiency for key subproblems in applications such as Gaussian process regression.

Existing iterative solvers for large-scale linear systems suffer from strong dependence on the global condition number and coarse-grained complexity analyses. Method: We introduce the *spectral tail condition number* $kappa_ell$, a new fine-grained spectral measure, and develop a refined time-complexity framework. Our approach formally defines $kappa_ell$, integrates it with the Sketch-and-Project paradigm, Nesterov acceleration, determinant point process sampling, and universality theory for Gaussian matrices, thereby exposing an intrinsic connection between iteration complexity and the matrix multiplication exponent $omega$. Contribution/Results: Our analysis achieves a sharper separation between deterministic and randomized algorithms, yielding an $ ilde{O}(kappa_ell n^2 log(1/varepsilon))$ bound for computing an $varepsilon$-accurate solution—valid for $ell$ up to $O(n^{0.729})$. This significantly improves the fine-grained analysis of the conjugate gradient method and establishes a novel theoretical benchmark for iterative algorithm design.

This work addresses the problem of efficiently and accurately computing ℓ_p-Lewis weights (for p ≥ 4) of a matrix, which quantify the importance of its rows. By alternating between primal and dual formulations of the underlying optimization problem and integrating leverage score iteration with a locally relative smooth gradient descent method, the authors propose a novel algorithm that significantly reduces computational overhead while maintaining high accuracy. Specifically, the proposed approach improves the iteration complexity from O(p³ log(m/ε)) to O(p² log(m/ε)), achieving the current best-known bound on the number of iterations required for ε-approximate Lewis weight computation.

This work addresses the grouped distributionally robust (GDR) least squares problem, which seeks to minimize the worst-case loss across multiple data groups. The authors introduce block Lewis weights—a novel geometric tool—to reformulate the problem as a specially weighted least squares instance. By integrating an accelerated proximal algorithm with a structured linear system solver tailored for systems of the form \(A^\top B A\), they achieve an efficient solution method that unifies optimization frameworks for both average and robust losses. The proposed approach outperforms interior-point methods at moderate accuracy levels. Theoretically, it attains a \((1+\varepsilon)\)-approximate solution using only \(\widetilde{O}(\min\{\mathrm{rank}(A), m\}^{1/3} \varepsilon^{-2/3})\) linear system solves, yielding the current best-known guarantee for the special case of \(\ell_\infty\) regression.

This work addresses the problem of efficient robust linear regression under high-dimensional Gaussian covariates with unknown covariance and adversarial contamination. The authors propose a near-linear-time algorithm that, when the contamination rate ε and the condition number κ of the covariance matrix satisfy εκ ≲ 1, achieves the minimax-optimal prediction error O(√(εκ)) using only Õ(d/ε⁴) samples—significantly improving upon prior methods. Furthermore, they establish fundamental limits for efficient algorithms by proving, via statistical query (SQ) lower bounds and low-degree polynomial hardness, that any computationally efficient SQ algorithm attaining better error must require Ω(d²) samples, thereby characterizing the sample complexity frontier for efficient estimation in this setting.

This work addresses the high computational cost of traditional leverage-score-based algorithms for approximating the John ellipsoid to within a $(1+\varepsilon)$ factor, whose iteration complexity scales with $\varepsilon^{-1}$ and thus hinders high-precision efficiency. By reframing the problem as D-optimal design, the paper reveals that the $\varepsilon^{-1}$ dependence stems solely from conventional certification mechanisms. It proposes a novel strategy that focuses exclusively on the final iteration and avoids solution averaging. Leveraging leverage scores as a first-order gradient oracle, the method combines warm-start acceleration with a damped Newton algorithm exploiting the self-concordance of the barrier function to accurately recover the Hessian. This approach achieves a $(1+\varepsilon)$-approximation using only $O(d^2 \log\log(1/\varepsilon))$ oracle queries and incurs a preprocessing cost $C(A)$ independent of $\varepsilon$, substantially reducing the overhead for high-accuracy computation.

This work addresses the high computational cost of ordinary least squares (OLS) estimation in large-scale linear models by proposing the Sequentially Preconditioned Conjugate Gradient (SPCG) method. SPCG constructs a sequence of increasingly larger randomized sketch subproblems, each solved via an inner preconditioned conjugate gradient (PCG) iteration, and leverages warm-starting from the previous solution to accelerate convergence. The final solution is refined on the full-scale problem. SPCG uniquely integrates incremental sketching with warm-started PCG, significantly reducing both iteration count and CPU time while preserving OLS prediction accuracy. Theoretical analysis provides rigorous guarantees on convergence and computational complexity, and empirical results demonstrate its superiority over full-data PCG and the Iterative Double Sketching (IDS) method.