🤖 AI Summary
This work addresses the high computational cost of ordinary least squares (OLS) estimation in large-scale linear models by proposing the Sequentially Preconditioned Conjugate Gradient (SPCG) method. SPCG constructs a sequence of increasingly larger randomized sketch subproblems, each solved via an inner preconditioned conjugate gradient (PCG) iteration, and leverages warm-starting from the previous solution to accelerate convergence. The final solution is refined on the full-scale problem. SPCG uniquely integrates incremental sketching with warm-started PCG, significantly reducing both iteration count and CPU time while preserving OLS prediction accuracy. Theoretical analysis provides rigorous guarantees on convergence and computational complexity, and empirical results demonstrate its superiority over full-data PCG and the Iterative Double Sketching (IDS) method.
📝 Abstract
We propose a randomized iterative method for the ordinary least-squares estimation problem in large-scale linear statistical models, namely the Sequential Preconditioned Conjugate Gradient Method (SPCG). SPCG constructs a sequence of sketched least-squares subproblems with increasing sketch sizes, applies PCG as the inner solver, and warm-starts each subproblem from the previous solution. A final refinement stage is then performed on the full-scale problem. Since most iterations are carried out on smaller subproblems, the overall computational cost is significantly reduced. We establish the convergence theory, prove that SPCG attains OLS prediction accuracy, and derive per-subproblem iteration bounds and complexity estimates. Numerical experiments show that SPCG reaches the target prediction accuracy with fewer iterations and less CPU time than full-data PCG and Iterative Double Sketching (IDS).