🤖 AI Summary
While theoretical equivalence between the Laplace kernel and the Neural Tangent Kernel (NTK) is established only in the infinite-width limit, empirical validation under realistic finite-width and high-dimensional Euclidean space (ℝᵈ) settings remains lacking. Method: This work conducts the first systematic regression experiments to assess their practical equivalence, employing two rigorous criteria: exact kernel function matching and consistency of Gaussian process posterior predictions—both evaluated under finite-width and ℝᵈ conditions. Contribution/Results: We demonstrate strong empirical agreement between the Laplace kernel and NTK in regression performance and generalization behavior, even when idealized assumptions (e.g., infinite width) are relaxed. This robust similarity provides solid empirical justification for approximating the computationally expensive NTK with the efficient Laplace kernel. The findings advance the practical deployment of lightweight kernel methods in deep learning modeling, bridging theoretical insights with scalable real-world applications.
📝 Abstract
The neural tangent kernel is a kernel function defined over the parameter distribution of an infinite width neural network. Despite the impracticality of this limit, the neural tangent kernel has allowed for a more direct study of neural networks and a gaze through the veil of their black box. More recently, it has been shown theoretically that the Laplace kernel and neural tangent kernel share the same reproducing kernel Hilbert space in the space of $mathbb{S}^{d-1}$ alluding to their equivalence. In this work, we analyze the practical equivalence of the two kernels. We first do so by matching the kernels exactly and then by matching posteriors of a Gaussian process. Moreover, we analyze the kernels in $mathbb{R}^d$ and experiment with them in the task of regression.