Adaptive Nyström for Gaussian Process Regression

2026-07-29

AI Summary

This work addresses the cubic computational complexity of Gaussian process regression and the limited accuracy of conventional Nyström approximations due to static landmark selection. The authors propose an adaptive Nyström method that dynamically selects landmarks via a greedy strategy minimizing the trace residual of the kernel approximation error. Notably, they introduce for the first time an alternating coupling between landmark selection and hyperparameter optimization, enabling an adaptive low-rank approximation that responds to changes in the covariance structure. The resulting approach maintains linear computational complexity while significantly outperforming random landmark strategies across five benchmark functions, achieving predictive accuracy nearly on par with exact Gaussian processes and demonstrating superior efficiency, accuracy, and stability.

Abstract

Gaussian Process Regression (GPR) is a robust framework for uncertainty quantification, yet its $O(n^3)$ complexity limits its scalability. Low-rank Nyström approximations can reduce this burden to $O(nm^2)$, but their accuracy depends heavily on the selection of landmark points. We propose an adaptive Nyström approach that greedily selects landmarks to minimize the trace residual of the kernel approximation error. Unlike static approximations, our method interleaves landmark expansion with hyperparameter optimization, allowing the selection process to adapt as the covariance structure is refined. Numerical experiments on five benchmark functions demonstrate that this method significantly outperforms random landmark selection in both accuracy and stability. It achieves predictive performance comparable to exact GP inference while maintaining linear scaling with respect to the sample size, providing a principled and efficient framework for large-scale computer experiments.

Problem

• Gaussian Process Regression
• Nyström approximation
• landmark selection
• scalability
• kernel approximation

Innovation

• Adaptive Nyström
• Gaussian Process Regression
• Landmark Selection
• Kernel Approximation
• Scalable Inference