🤖 AI Summary
This work addresses the long-standing lack of a clear geometric interpretation for pivoted Cholesky decomposition in kernel methods. From the perspective of reproducing kernel Hilbert spaces (RKHS), it establishes that the pivot selection in pivoted Cholesky is equivalent to farthest-point sampling under the kernel-induced metric, while the construction of the Cholesky factors implicitly performs a Gram–Schmidt orthogonalization process. By revealing this geometric equivalence between pivoted Cholesky and farthest-point sampling, the study provides an intuitive theoretical explanation accompanied by an exceptionally concise mathematical derivation. Furthermore, a lightweight Python implementation is provided, effectively bridging the gap between theoretical analysis and practical application in kernel-based learning.
📝 Abstract
Low-rank approximations of large kernel matrices are ubiquitous in machine learning, particularly for scaling Gaussian Processes to massive datasets. The Pivoted Cholesky decomposition is a standard tool for this task, offering a computationally efficient, greedy low-rank approximation. While its algebraic properties are well-documented in numerical linear algebra, its geometric intuition within the context of kernel methods often remains obscure. In this note, we elucidate the geometric interpretation of the algorithm within the Reproducing Kernel Hilbert Space (RKHS). We demonstrate that the pivotal selection step is mathematically equivalent to Farthest Point Sampling (FPS) using the kernel metric, and that the Cholesky factor construction is an implicit Gram-Schmidt orthogonalization. We provide a concise derivation and a minimalist Python implementation to bridge the gap between theory and practice.