Adaptive finite element type decomposition of Gaussian processes

2025-05-29

AI Summary

This paper addresses the adaptivity of posterior contraction rates in approximate Gaussian process (GP) regression. It identifies a fundamental suboptimality of the stationary SPDE–finite element method with fixed smoothness when the true function is smoother than assumed, whereas lattice-based GP methods with inverse-Gamma bandwidth priors achieve adaptive optimal rates. To unify these paradigms, we propose a general GP approximation framework centered on linear combinations of compactly supported basis functions. We establish, for the first time, that lattice GPs achieve optimal posterior contraction over all unknown smoothness classes under a prior on the number of basis functions, while the SPDE approach fails to adapt due to its fixed smoothness assumption. We develop a unified theory of posterior concentration rates and design computationally efficient inference strategies. Numerical experiments corroborate the theoretical guarantees and demonstrate the practical advantages of our framework.

Technology Category

Machine Learning: Bayesian LearningReasoning under Uncertainty: Stochastic OptimizationSearch and Optimization: Mixed Discrete/Continuous Search

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingWeb Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

In this paper, we investigate a class of approximate Gaussian processes (GP) obtained by taking a linear combination of compactly supported basis functions with the basis coefficients endowed with a dependent Gaussian prior distribution. This general class includes a popular approach that uses a finite element approximation of the stochastic partial differential equation (SPDE) associated with Mat'ern GP. We explored another scalable alternative popularly used in the computer emulation literature where the basis coefficients at a lattice are drawn from a Gaussian process with an inverse-Gamma bandwidth. For both approaches, we study concentration rates of the posterior distribution. We demonstrated that the SPDE associated approach with a fixed smoothness parameter leads to a suboptimal rate despite how the number of basis functions and bandwidth are chosen when the underlying true function is sufficiently smooth. On the flip side, we showed that the later approach is rate-optimal adaptively over all smoothness levels of the underlying true function if an appropriate prior is placed on the number of basis functions. Efficient computational strategies are developed and numerics are provided to illustrate the theoretical results.

Problem

• Study approximate Gaussian processes using compact basis functions
• Compare SPDE and lattice-based approaches for scalability
• Analyze posterior concentration rates for optimal performance

Innovation

• Uses compactly supported basis functions
• Applies dependent Gaussian prior distribution
• Adapts inverse-Gamma bandwidth for scalability