rkhs projection

This work addresses the scalability limitations of traditional kernel methods, which require constructing and inverting large kernel matrices, and the lack of generality in existing denoising approaches that often rely on restrictive assumptions about signals or noise. The authors propose an efficient operator learning algorithm based on Nyström subsampling for vector-valued regression in reproducing kernel Hilbert spaces, unifying denoising within a general operator learning framework. They innovatively relax classical Hölder-type and operator monotonicity constraints by introducing an indicator function to characterize more general source conditions, and for the first time apply Nyström approximation systematically to operator learning with functional outputs and universal denoising tasks. Experiments demonstrate that the method achieves performance comparable to full-kernel approaches at substantially reduced computational cost across diverse applications—including signal, audio, and image denoising, Radon inversion reconstruction, and energy efficiency prediction—while attaining minimax optimal convergence rates.

Traditional neural networks suffer from limited interpretability and weak theoretical foundations. Method: This paper proposes a novel machine learning paradigm grounded in infinite-dimensional Hilbert spaces, centering on linear operators. It integrates reproducing kernel Hilbert spaces (RKHS), spectral operator learning, wavelet representations, scattering transforms, and Koopman operator theory to formulate learning tasks as sampling, approximation, and dynamical inference in infinite-dimensional function spaces. Contribution/Results: We establish the first unified Hilbert-space-theoretic framework bridging spectral learning and symbolic reasoning. The approach significantly enhances mathematical rigor and model interpretability by grounding learning in well-defined functional-analytic principles. Moreover, it provides a rigorous mathematical foundation and new methodological pathways for deep interdisciplinary integration between signal processing and machine learning—enabling principled analysis of structured data, hierarchical feature extraction, and nonlinear dynamical system modeling.

This work addresses the curse of dimensionality and model misspecification in learning nonlinear operators between infinite-dimensional spaces. We propose a stochastic approximation framework based on Mercer operator-valued kernels. Methodologically, the approach unifies treatment of general kernel structures—including compact and diagonal kernels—by integrating vector-valued reproducing kernel Hilbert spaces (RKHS), spectral decomposition, and interpolation space theory to construct a vector-valued interpolation space with quantifiable model error. Theoretically, we establish, for the first time, dimension-independent polynomial convergence rates, overcoming the linear-rate limitation inherent to scalar-valued kernels (K = kI) and providing rigorous theoretical guarantees for genuinely nonlinear operator learning. Numerical experiments on the two-dimensional Navier–Stokes equations demonstrate high-accuracy modeling and confirm the framework’s effectiveness in mitigating the curse of dimensionality.

Existing data-driven Koopman operator methods struggle to ensure approximate invariance of subspaces under the operator in non-Euclidean settings, limiting predictive accuracy. This work addresses this challenge by extending principal vector–guided subspace pruning to reproducing kernel Hilbert spaces (RKHS) for the first time. By precisely computing principal angles and vectors in RKHS, we introduce Kernel-SPV and its computationally efficient Nyström approximation–based variant, Approximate Kernel-SPV. These approaches overcome the limitations of traditional Euclidean formulations, significantly enhancing the invariance of Koopman-invariant subspaces while maintaining scalability and substantially improving prediction accuracy.

Existing neural operators lack reliability and theoretical guarantees when handling out-of-distribution input functions. This work proposes an extended framework grounded in reproducing kernel Hilbert spaces (RKHS), leveraging kernel approximation techniques to achieve robust approximation of both out-of-distribution functions and their derivatives. The key innovation lies in establishing a theoretical connection between kernel selection and Sobolev eigenfunction spaces, thereby providing predictable guarantees on generalization error and derivative accuracy for neural operators. When applied to solving elliptic partial differential equations—particularly on manifolds represented as point clouds—the method demonstrates significantly enhanced geometric awareness, improved extrapolation accuracy, and greater computational efficiency.

This work addresses the lack of a systematic connection between classical integral operators and reproducing kernel Hilbert spaces (RKHS) in algebraic signal processing, which has hindered theoretical advances in graph signal processing and learnable filters. The paper establishes, for the first time, an algebraic correspondence between integral operators and RKHS by constructing a unital kernel algebra via the box product of operator symbols, thereby deriving the associated reproducing kernel and characterizing its spectral and algebraic properties. This framework enables exact alignment between graph signal spectral decomposition and RKHS representation, extends naturally to directed graphs, and proves that when the spectral domain of a regularized learning problem is a subset of the signal domain, the optimal filter admits a finite-dimensional RKHS representation—providing a rigorous theoretical foundation for learnable filters in neural architectures based on integral operators.

This work addresses the challenge of resource allocation among the number of training samples (N), input observation points (n), and output resolution (m) in operator learning. The authors propose a two-stage sampling framework: in the offline stage, a discrete representation of the operator is learned via kernel regression; in the online stage, the output function is reconstructed from predicted observations, enhanced by physics-informed constraints to improve accuracy. The study establishes a novel quantitative scaling law and error decomposition mechanism linking N, n, and m, and introduces a physics-informed online reconstruction strategy that avoids retraining. Theoretical analysis provides convergence guarantees and an error-balancing criterion, while numerical experiments validate the proposed scaling law and demonstrate the method’s superior performance in preserving physical consistency and achieving high reconstruction accuracy.

This study addresses the high computational cost and energy consumption of supervised learning in the era of big data by investigating nonparametric supervised learning within a reproducing kernel Hilbert space. The authors propose a Horvitz–Thompson reweighted subsampling estimator based on empirical risk minimization. Through asymptotic analysis, they derive—for the first time—the optimal subsampling probabilities under the trace norm of the covariance operator and provide an efficient plug-in implementation. Both theoretical analysis and empirical evaluations demonstrate that the proposed method substantially reduces computational overhead while preserving estimation accuracy across synthetic and real-world datasets, offering an efficient and environmentally sustainable solution for large-scale nonparametric learning.

This work addresses the lack of theoretical guarantees for nonparametric regression in reproducing kernel Hilbert spaces under model misspecification, high-dimensional settings, and nonconvex losses. It establishes a unified theoretical framework for regularized M-estimators encompassing a broad class of both convex and nonconvex loss functions. By introducing a novel complexity measure, the analysis achieves an explicit bias–variance decomposition. Leveraging tools from functional analysis and empirical process theory, the study proves the existence, measurability, and asymptotic linearity of the estimator without requiring closed-form solutions or global Lipschitz assumptions. Notably, within tensor-product Sobolev spaces, the framework reveals a mechanism to circumvent the curse of dimensionality, yielding minimax-optimal convergence rates that depend on mixed smoothness of the underlying function. The variance component is shown to be robust to model misspecification, and numerical experiments in C++ corroborate the theoretical findings.

This work addresses dynamic regret minimization in online regression over a reproducing kernel Hilbert space (RKHS) when the optimal comparator sequence varies over time. The authors extend the finite-dimensional discounted Vovk–Azoury–Warmuth (VAW) algorithm to RKHS by running an ensemble of discounted VAW predictors on adaptively constructed finite-dimensional subspaces. A unified orthogonal truncation framework is introduced to accommodate various approximation strategies—including explicit feature expansions, Mercer spectral truncation, and kernel section projections—under a single analysis. The approach applies to kernels such as Gaussian, analytic dot-product, and Matérn, and establishes dynamic regret bounds that depend on the comparator’s path length and the decay rate of kernel eigenvalues, with distinct guarantees under both fast and slow learning rates.