Sharp Asymptotic Theory of Maximum Likelihood Estimation for Gaussian Processes with an RBF Kernel

2026-10-07

AI Summary

This study addresses the previously unclear asymptotic behavior of maximum likelihood estimation (MLE) for Gaussian processes with radial basis function kernels under fixed-domain asymptotics. By integrating fixed-domain asymptotic analysis, MLE theory, and non-standard statistical inference techniques, it overcomes challenges arising from strongly dependent observations and nonlinear covariance matrices to establish a complete asymptotic theory for the joint MLE of spatial variance, length scale, and noise variance. This work provides the first comprehensive characterization of this parameter combination in the fixed-domain regime. It rigorously proves estimator consistency, derives the convergence rates for each parameter, and establishes their joint asymptotic normality. Furthermore, the obtained convergence rates are shown to be minimax optimal.

Abstract

Gaussian processes (GPs) are widely used across machine learning, spatial statistics, time-series analysis, optimization, Bayesian statistics, and scientific applications. A central component of a GP model is its kernel, which is typically specified through a parametric family. Among the most widely used choices is the radial basis function (RBF), also known as the squared exponential or Gaussian kernel, owing to its simple form, smoothness, and flexibility. In practice, the kernel parameters are routinely estimated by the maximum likelihood estimators (MLEs), as implemented by standard GP software. Despite this widespread use, the asymptotic behavior of the MLEs remains poorly understood under fixed-domain asymptotics, even for the RBF kernel. The main difficulty arises from the increasingly strong dependence among densely sampled observations and the nonlinear dependence of the covariance matrix on the kernel parameters. In this paper, we address this gap by providing, to the best of our knowledge, the first complete asymptotic characterization of the joint MLE of the spatial variance, lengthscale, and nugget variance under fixed-domain asymptotics. We establish consistency, derive convergence rates for all three parameters, prove joint asymptotic normality, and show that these rates are minimax optimal.

Problem

• Gaussian Processes
• Maximum Likelihood Estimation
• Radial Basis Function Kernel
• Fixed-domain Asymptotics
• Asymptotic Theory

Innovation

• Gaussian Processes
• Maximum Likelihood Estimation
• Fixed-domain Asymptotics
• RBF Kernel
• Minimax Optimality