nystrom approximation

Under heavy-tailed spectral decay, conventional Nyström methods suffer from computational overload and explosive effective dimension growth, leading to failure of low-rank approximation. To address this, we propose Block-Nyström: a block-diagonal low-rank decomposition framework that reduces computational complexity while preserving approximation accuracy via randomized sampling and explicit tail-spectrum estimation. We introduce a novel recursive preconditioned inverse solver, dramatically accelerating kernel ridge regression (KRR) computation. Furthermore, we derive a new statistical learning bound for generalized approximate KRR. Our method achieves significantly improved spectral tail estimation accuracy under identical computational budgets, enabling scalable, high-precision preconditioning for large-scale second-order optimization. Key innovations include (i) the block-structured Nyström decomposition framework, (ii) a recursive matrix preconditioning mechanism, and (iii) a theory-driven error control system ensuring rigorous approximation guarantees.

To address the $O(n^2)$ computational bottleneck of kernelized Stein discrepancy (KSD) under large-scale data—arising from its reliance on U- or V-statistics—this paper introduces, for the first time, the Nyström low-rank kernel approximation into KSD estimation, yielding a scalable and accelerated KSD estimator. The proposed method reduces time complexity to $O(mn + m^3)$, where $m ll n$, and establishes $sqrt{n}$-consistency under sub-Gaussian assumptions. Theoretical analysis is grounded in the Stein operator and reproducing kernel Hilbert space (RKHS) framework, balancing statistical efficiency with computational tractability. Extensive benchmark experiments demonstrate that the new estimator retains statistical power comparable to the original KSD while substantially enhancing practicality for large-scale goodness-of-fit testing. This work provides an efficient, theoretically sound tool for high-dimensional distribution fitting and hypothesis testing.

To address the low computational efficiency of large-scale radial kernel summations, this paper proposes a randomized slicing acceleration method that integrates quasi-Monte Carlo (QMC) sampling with spherical numerical integration: high-dimensional radial kernel sums are first reduced to one dimension via random projection, then efficiently computed using the fast Fourier transform (FFT). This work is the first to incorporate QMC sequences and spherical quadrature rules into the slicing framework. We derive a theoretical error upper bound and prove that the proposed method achieves a faster convergence rate than both standard Monte Carlo and non-QMC slicing approaches. Experiments on standard benchmark datasets demonstrate that, while maintaining linear time complexity, our method significantly outperforms randomized and orthogonal Fourier features as well as existing slicing methods in approximation accuracy—thereby achieving a synergistic improvement in both computational efficiency and approximation fidelity.

Existing asymmetric kernel singular value decomposition (KSVD) methods rely on finite-dimensional approximations, limiting their ability to handle infinite-dimensional feature maps, and their variational objectives may be unbounded. Method: We propose the Coupled Covariance Eigenproblem (CCE) framework—the first rigorous variational formulation of KSVD in infinite-dimensional Hilbert spaces—unifying asymmetric KSVD with covariance operator theory and accommodating arbitrary non-Mercer, asymmetric kernels. We further derive an asymmetric Nyström method based on coupled adjoint eigenfunctions, overcoming classical limitations of symmetric kernel approximation or linear SVD modeling. Contribution/Results: Experiments demonstrate that our method significantly outperforms symmetric-baseline and linear-SVD approaches across multiple tasks, achieving faster training convergence and improved generalization. This work provides the first empirical validation of the practical utility of asymmetric kernel learning.

Kernel ridge regression (KRR) suffers from prohibitive memory and computational costs in large-scale settings. This paper focuses on the low-rank approximation theory of KRR and makes four key contributions: (i) it derives, for the first time, a tight lower bound on the minimal rank required to preserve prediction consistency—providing rigorous, optimal theoretical guarantees for Nyström-type approximations; (ii) it proves that the computational complexity of the Nyström approximation is nearly linear in the number of samples; (iii) it establishes an approximation error bound for kernel functions in the range space of the integral operator and characterizes the growth behavior of the associated weight function norm; and (iv) it significantly expands the admissible range of regularization parameters. Collectively, these results unify the analytical framework for reliability, efficiency, and stability of low-rank KRR approximations, offering both foundational theoretical insights and practical guidance for scalable kernel learning.

This work addresses the scalability limitations of traditional kernel methods, which require constructing and inverting large kernel matrices, and the lack of generality in existing denoising approaches that often rely on restrictive assumptions about signals or noise. The authors propose an efficient operator learning algorithm based on Nyström subsampling for vector-valued regression in reproducing kernel Hilbert spaces, unifying denoising within a general operator learning framework. They innovatively relax classical Hölder-type and operator monotonicity constraints by introducing an indicator function to characterize more general source conditions, and for the first time apply Nyström approximation systematically to operator learning with functional outputs and universal denoising tasks. Experiments demonstrate that the method achieves performance comparable to full-kernel approaches at substantially reduced computational cost across diverse applications—including signal, audio, and image denoising, Radon inversion reconstruction, and energy efficiency prediction—while attaining minimax optimal convergence rates.

This work addresses the convergence challenge in unsupervised domain adaptation under covariate shift when the target function lies outside the reproducing kernel Hilbert space (i.e., the misspecified setting). By integrating Tikhonov regularization with Nyström subsampling projection, the paper establishes, for the first time, a high-probability excess risk upper bound for Nyström-type domain adaptation methods in this misspecified regime. Leveraging source conditions, effective dimension estimates, and approximation of the Radon–Nikodym derivative, the proposed approach achieves the same convergence rate as in the well-specified setting, requiring only a minimal number of additional samples even when the Radon–Nikodym derivative is unknown.

This work addresses the computational inefficiency of traditional kernel Stein discrepancy (KSD) tests, which suffer from quadratic time complexity due to their reliance on U- or V-statistics and require computationally intensive bootstrap procedures to approximate the null distribution. To overcome these limitations, the authors propose an accelerated KSD test based on the Nyström approximation. They provide the first theoretical guarantee that this approach preserves asymptotic type-I error control and local consistency within a bootstrap framework, while substantially reducing computational cost. Empirical evaluations on spherical and functional data demonstrate that the accelerated method achieves statistical performance comparable to the original KSD test but with significantly improved computational efficiency, thereby enabling scalable and theoretically sound nonparametric goodness-of-fit testing.

This work addresses the limited flexibility of existing Hyperdimensional Computing (HDC) approaches, which rely on fixed mappings from raw data to high-dimensional space and struggle to adapt to diverse machine learning tasks. To overcome this limitation, the authors propose NysHD, a novel method that, for the first time, integrates the Nyström approximation from kernel methods into HDC. NysHD leverages any user-defined positive semi-definite similarity function to automatically construct effective high-dimensional embeddings, thereby establishing a theoretical bridge between kernel methods and HDC. By synergistically combining Nyström approximation, positive definite kernel design, and high-dimensional random encoding, the approach substantially enhances model expressivity. Experimental results on graph and string datasets demonstrate that NysHD improves classification accuracy by 11% and 17% on average over state-of-the-art HDC encoding schemes.

This work addresses the cubic computational complexity of Gaussian process regression and the limited accuracy of conventional Nyström approximations due to static landmark selection. The authors propose an adaptive Nyström method that dynamically selects landmarks via a greedy strategy minimizing the trace residual of the kernel approximation error. Notably, they introduce for the first time an alternating coupling between landmark selection and hyperparameter optimization, enabling an adaptive low-rank approximation that responds to changes in the covariance structure. The resulting approach maintains linear computational complexity while significantly outperforming random landmark strategies across five benchmark functions, achieving predictive accuracy nearly on par with exact Gaussian processes and demonstrating superior efficiency, accuracy, and stability.