Don't Get Your Kroneckers in a Twist: Gaussian Processes on High-Dimensional Incomplete Grids

2026-05-08

AI Summary

This work addresses the scalability limitations of Gaussian process regression (GPR) in high-dimensional settings, where computational complexity hinders application to large-scale data and high-dimensional inputs, particularly on incomplete grids. The authors propose CUTS-GPR, a novel method that integrates additive kernels, an incomplete tensor grid structure, and an efficient kernel matrix–vector multiplication algorithm to achieve, for the first time, exact GPR inference with near-linear or even linear time complexity. The approach enables rapid hyperparameter optimization and full posterior inference, completing the entire pipeline within hours on a dataset with N = 447,265 observations and D = 24 input dimensions. Demonstrated on high-dimensional potential energy surface modeling, CUTS-GPR substantially overcomes the longstanding scalability barrier of GPR, extending its applicability to problems involving billions of samples and thousands of features.

Abstract

We introduce CUTS-GPR, a new method for performing numerically exact Gaussian process regression (GPR) in high-dimensional settings. The key component of CUTS-GPR is an extremely fast kernel matrix-vector product, which exhibits near-linear or even linear scaling with the amount of training data, $N$, and low-order polynomial scaling with dimensionality, $D$. This is obtained by combining an additive kernel with an incomplete grid and exploiting the resulting structure of the kernel matrix. We demonstrate the scalability of the matrix-vector product by running benchmarks with billions of data points and thousands of dimensions. Full GPR calculations, including hyperparameter optimization, are completed in a matter of hours for $N = 447 265$ and $D = 24$. We demonstrate that our CUTS-GPR enables Bayesian modeling of high-dimensional potential energy surfaces - a longstanding challenge in computational chemistry.

Problem

• Gaussian process regression
• high-dimensional data
• incomplete grids
• scalability
• potential energy surfaces

Innovation

• Gaussian Process Regression
• Additive Kernel
• Incomplete Grid
• Scalable Kernel Matrix-Vector Product
• High-Dimensional Bayesian Modeling