low-rank approximation

This paper addresses the low-rank approximation of large-scale binary matrices. We propose a scalable, deterministic binary CUR decomposition algorithm. Its core innovation is a block-wise Adaptive Cross Approximation (Block-ACA) strategy that enables parallelized, greedy selection of dominant rows and columns—achieving near-optimal reconstruction accuracy under binary constraints for the first time while ensuring strong scalability. Unlike existing approaches, our method avoids stochastic sampling, thereby providing both theoretical guarantees and high computational efficiency. Experiments demonstrate excellent performance: on a 16,384 × 16,384 binary matrix, runtime decreases from 12.37 seconds to 1.02 seconds as the number of processes increases from 2 to 64. On Hilbert matrices and synthetic low-rank binary matrices, reconstruction error approaches the theoretical optimum, significantly outperforming state-of-the-art binary low-rank approximation methods.

This work addresses the computational bottleneck of power iteration in low-rank approximation under high target ranks, where expensive matrix multiplications dominate the cost. To overcome this limitation, the authors propose an efficient algorithmic framework that integrates fast sketching techniques with power iteration, substantially reducing computational overhead while remaining applicable to singular value decomposition, low-rank approximation, and Nyström methods. The key theoretical contribution lies in a novel analysis based on regularized spectral approximation, which offers greater flexibility than conventional arguments and enables rigorous generalization of theoretical guarantees for power iteration. Empirical evaluations on standard benchmark datasets demonstrate that the proposed method achieves excellent performance, algorithmic simplicity, and provable efficiency.

This paper addresses the problem of jointly estimating the low-rank and diagonal components of a linear operator with low-rank-plus-diagonal (LoRD) structure, using only a small number of matrix-vector products. To this end, we propose SKETCHLORD—a novel method that formulates joint estimation as a scalable convex optimization problem, thereby avoiding error accumulation inherent in sequential estimation approaches. Leveraging randomized sketching and matrix decomposition principles, SKETCHLORD achieves structured approximation solely via black-box matrix-vector multiplication. Theoretically, it provides robustness guarantees under noise. Empirically, on both synthetic benchmarks and large-scale operators—including Hessian matrices from deep learning models—SKETCHLORD significantly outperforms existing methods: it achieves higher recovery accuracy and superior computational efficiency at the same query cost. By unifying estimation and optimization within a sketching-based framework, SKETCHLORD establishes a new paradigm for high-dimensional structured operator approximation.

This paper addresses classification under high-dimensional sparse settings. We propose a two-step discriminant method based on principal component analysis (PCA), grounded in an implicit low-rank factor model and featuring adaptive selection of the number of principal components. We establish, for the first time, a general risk analysis framework for high-dimensional two-step classifiers and rigorously derive the minimax-optimal convergence rate (up to logarithmic factors) for the PCA-based classifier—even when dimensionality far exceeds sample size. Theoretically, the excess risk achieves the optimal rate; simulations demonstrate robustness under model misspecification; and empirical evaluation on three real-world high-dimensional datasets shows significant improvement over state-of-the-art discriminant methods. Key contributions include: (i) a unified theoretical analysis paradigm for two-step classification, (ii) minimax-optimal rate guarantees, (iii) a data-driven, theoretically justified dimension-selection mechanism, and (iv) consistent empirical superiority across diverse high-dimensional benchmarks.

This work addresses efficiency bottlenecks in solving large-scale linear systems and approximating matrix norms. We propose a multilevel randomized sketching preconditioned iterative method, integrating Nyström low-rank approximation, sparse random sketching, and multilevel preconditioning. It establishes the first multilevel sketched preconditioning framework grounded in the natural average condition number. Theoretical contributions include: (1) optimal complexity $ ilde{O}(n^2 + d_lambda^omega)$ for solving regularized linear systems; (2) accelerated complexity $ ilde{O}(n^{2.065} + k^omega)$ for systems with $k$ outlying singular values; and (3) Schatten-$p$ norm approximation—particularly the nuclear norm—at $ ilde{O}(n^{2.11})$, improving upon the prior best $ ilde{O}(n^{2.18})$. These advances significantly enhance computational efficiency for key subproblems in applications such as Gaussian process regression.

This study addresses the challenge of latent dimension selection in high-dimensional factor models when the model holds only approximately or the signal-to-noise ratio is low. The authors propose EigenBayes, a novel method that uniquely integrates spectral estimation with adaptive empirical Bayes to construct a structured shrinkage prior, enabling efficient shrinkage and uncertainty quantification in over-parameterized factor models. The approach yields an analytically tractable posterior distribution, obviating the need for MCMC sampling, and adaptively accounts for varying signal-to-noise ratios across both observed variables and latent dimensions. Theoretical analysis establishes favorable asymptotic properties, while numerical experiments and a genomics application demonstrate superior performance over state-of-the-art methods.

This work addresses the challenge of efficiently computing leading eigenvectors in dynamic graphs, where frequent updates to the adjacency or Laplacian matrix render traditional eigendecomposition methods computationally prohibitive. To overcome this limitation, the authors propose a fast spectral embedding update framework based on Rayleigh-Ritz projection. By leveraging eigenvector perturbation analysis, the method constructs a low-dimensional approximate invariant subspace that preserves high approximation accuracy while substantially reducing computational and memory costs. Experimental results demonstrate that the proposed approach outperforms existing techniques in both the quality of leading eigenvector approximation and performance on downstream tasks—such as influential node identification and node clustering—offering a compelling balance between efficiency and accuracy.

This work addresses the problem of maintaining a consistent low-rank approximation for matrices whose rows arrive in a streaming fashion, with the dual objective of preserving approximation accuracy while minimizing recourse—the total amount of change in the output sequence over time. The authors propose a novel recursive subspace update algorithm that, at each time step, produces a near-optimal rank-$k$ approximation. They establish the first tight theoretical bounds characterizing the trade-off between recourse and both additive and multiplicative approximation errors. Leveraging techniques from streaming matrix processing, low-rank approximation, and condition number analysis, their theoretical framework significantly sharpens upper bounds under certain conditions. Empirical evaluations on real-world datasets demonstrate the algorithm’s efficiency and stability in practice.

This study addresses the challenging problem of recovering a low-rank latent factor structure linked through an unknown monotonic nonlinear function from incomplete and noisy observations, which is hindered by severe non-convexity and identifiability ambiguities. The authors generalize linear factor models to a nonparametric nonlinear setting by assuming the link function resides in a reproducing kernel Hilbert space (RKHS) and introduce explicit regularization to resolve scale and rotational indeterminacies. They propose a projected block coordinate descent algorithm that jointly estimates the latent factors, loading matrix, and link function, providing theoretical convergence guarantees in both noiseless and noisy regimes. Moreover, the update of the link function enjoys a sublinear regret bound. Synthetic experiments demonstrate the method’s effectiveness and robustness.

This work addresses the sensitivity of classical spectral methods to noise, outliers, and model perturbations, which often leads to inaccurate estimation of low-rank projection subspaces and consequently degrades community detection and clustering performance. To overcome this limitation, the authors propose the Regularized Projection Matrix Approximation (RPMA) framework, formulating robust low-rank projection estimation as a regularized optimization problem on the Grassmann manifold. They establish, for the first time, first- and second-order optimality conditions for this problem and prove the local stability of the regularized dominant subspace. Furthermore, they introduce the Cayley-SMW gradient method, which leverages the Sherman–Morrison–Woodbury formula to circumvent repeated eigendecompositions, substantially improving computational efficiency. Experiments demonstrate that RPMA consistently outperforms existing spectral methods on both synthetic and real-world datasets, achieving more accurate projection recovery and enhanced clustering robustness in noisy environments.