kernels

This paper addresses the fragmentation and weak geometric intuition in existing kernel method theory by establishing a unified functional analytic framework grounded in Hilbert space geometry. Starting from the definition of positive-definite kernels, it rigorously unifies reproducing kernel Hilbert spaces (RKHS) and Hilbert–Schmidt operators, thereby reconstructing fundamental statistical concepts—including covariance, regression, and information-theoretic measures—within a coherent geometric setting. The work innovatively embeds kernel density estimation, distributional kernel embeddings, and maximum mean discrepancy (MMD) into a single RKHS paradigm, yielding a self-consistent theory bridging statistical estimation and probabilistic representation. This framework provides geometric interpretations for Gaussian processes and kernel Bayesian inference, and establishes a rigorous mathematical foundation for future theoretical advances in kernel-based learning.

This work addresses the challenge of incorporating prior features into kernel methods without penalizing them, thereby enhancing regression performance. To this end, the authors propose Conditional Kernel Ridge Regression (Conditional KRR), which decomposes the target function into a prior component modeled within a prescribed function class and a residual component, applying kernel regularization only to the latter. Theoretical analysis reveals that this approach is equivalent to standard Kernel Ridge Regression augmented with a controllable error term, and it achieves improved statistical risk under settings such as principal components or random features. When the prior component dominates the target function, Conditional KRR substantially outperforms standard KRR, a finding corroborated by both theoretical guarantees and empirical experiments.

This paper addresses the long-standing conceptual divide between Gaussian processes (GPs) and reproducing kernel Hilbert space (RKHS) methods—two fundamental paradigms grounded in positive-definite kernels. We establish a unified theoretical framework rooted in the rigorous isometric isomorphism between the Gaussian Hilbert space and the RKHS. Our method formally proves equivalence between GP-based and RKHS-based solutions across diverse tasks: regression, interpolation, numerical integration, distributional discrepancy measurement (e.g., maximum mean discrepancy), statistical dependence quantification, and sample path analysis. Crucially, this framework bridges the epistemological gap between Bayesian probabilistic modeling and deterministic kernel methods. The results provide a coherent foundation for kernelized Bayesian inference, kernel manifold learning, and other cross-disciplinary applications—enabling principled integration of probabilistic and functional-analytic perspectives within kernel methods. (149 words)

This work addresses the lack of a systematic connection between classical integral operators and reproducing kernel Hilbert spaces (RKHS) in algebraic signal processing, which has hindered theoretical advances in graph signal processing and learnable filters. The paper establishes, for the first time, an algebraic correspondence between integral operators and RKHS by constructing a unital kernel algebra via the box product of operator symbols, thereby deriving the associated reproducing kernel and characterizing its spectral and algebraic properties. This framework enables exact alignment between graph signal spectral decomposition and RKHS representation, extends naturally to directed graphs, and proves that when the spectral domain of a regularized learning problem is a subset of the signal domain, the optimal filter admits a finite-dimensional RKHS representation—providing a rigorous theoretical foundation for learnable filters in neural architectures based on integral operators.

Traditional neural networks suffer from limited interpretability and weak theoretical foundations. Method: This paper proposes a novel machine learning paradigm grounded in infinite-dimensional Hilbert spaces, centering on linear operators. It integrates reproducing kernel Hilbert spaces (RKHS), spectral operator learning, wavelet representations, scattering transforms, and Koopman operator theory to formulate learning tasks as sampling, approximation, and dynamical inference in infinite-dimensional function spaces. Contribution/Results: We establish the first unified Hilbert-space-theoretic framework bridging spectral learning and symbolic reasoning. The approach significantly enhances mathematical rigor and model interpretability by grounding learning in well-defined functional-analytic principles. Moreover, it provides a rigorous mathematical foundation and new methodological pathways for deep interdisciplinary integration between signal processing and machine learning—enabling principled analysis of structured data, hierarchical feature extraction, and nonlinear dynamical system modeling.

This work addresses the computational and memory bottlenecks of traditional Grassmannian kernel methods, which require constructing full Gram matrices and thus struggle with high-dimensional subspace data. To overcome these limitations, the authors propose a scalable kernel approximation framework based on random rank-one projections combined with bounded nonlinear transformations—either periodic or binary—that yield compact one-bit subspace feature representations. This approach enables continuous interpolation between the inverse Binet–Cauchy kernel and Gaussian-like kernels while effectively preserving the intrinsic geometry of subspaces. The method substantially reduces computational, memory, and storage costs. Experimental results on synthetic data and the ETH-80 classification benchmark demonstrate that the proposed technique accurately maintains Grassmannian geometric relationships with high fidelity, confirming its efficiency and practical utility.

Simulating high-dimensional nonstationary Gaussian processes is challenging due to high computational complexity, and conventional spectral methods are limited by the assumption that the spectral density must be a probability measure, which hinders their applicability to nonstationary settings. This work proposes a regularized Fourier features approach that directly discretizes the spectral representation of harmonizable processes, yielding complex-valued Fourier features without requiring the spectral density to be a probability measure. The method enables efficient low-rank approximations while preserving the correlation structure encoded in spectral weights. It naturally supports learning kernel functions from data and demonstrates strong empirical performance on both locally stationary and harmonizable mixture kernels, significantly improving the efficiency and structural fidelity of nonstationary Gaussian process simulation.

This work investigates the generalization performance of random feature methods under operator-valued kernels, with particular emphasis on the misspecified setting where the target function lies outside the associated reproducing kernel Hilbert space (RKHS). To this end, the authors develop a unified spectral regularization framework that encompasses both neural operators and neural networks within the neural tangent kernel (NTK) perspective for theoretical analysis. They extend random feature methods to operator-valued kernels for the first time and establish minimax optimal convergence rates in both well-specified and misspecified regimes. Key contributions include deriving optimal learning rates, quantifying the number of neurons required to achieve a prescribed accuracy, and strengthening the theoretical foundations of operator-valued kernel methods.

This work addresses the gap between algorithmic prototypes and efficient implementations in scientific research by proposing a lightweight approach to translate statistical and machine learning algorithms—such as kernel ridge regression and stochastic gradient descent matrix factorization—from mathematical formulations into readable, high-performance C++ code. Leveraging the Eigen template library for core linear algebra operations—including kernel matrix construction, regularized solvers, and vectorized updates—the implementation seamlessly integrates into the Python ecosystem via pybind11, enabling efficient interoperability with NumPy arrays. The project provides concise, reproducible code examples that encapsulate common computational patterns in research, significantly lowering the barrier for researchers to adopt C++ for high-performance development while balancing performance, readability, and usability.

This study investigates the theoretical mechanism by which the Gaussian RBF kernel converges to the linear kernel as the bandwidth parameter σ tends to infinity, and its implications for kernel PCA. Through asymptotic analysis in reproducing kernel Hilbert space (RKHS), the work establishes—for the first time—a quantitative relationship between the data geometric eccentricity ρ and the convergence rate of kernel methods. It is shown that, under large bandwidths, the RKHS embedding asymptotically aligns with the principal component frame of Euclidean space via a similarity transformation. Theoretically, Gaussian kernel PCA thus converges to classical linear PCA in this limit. Experiments confirm that the eccentricity ρ effectively predicts the convergence behavior of the leading principal directions across diverse datasets, highlighting the pivotal role of data geometry in determining the asymptotic performance of kernel methods.