Fast and Provably Accurate Sequential Designs using Hilbert Space Gaussian Processes

2026-04-22

AI Summary

This work addresses the computational burden of integrated mean squared error (IMSE)-based acquisition functions in Gaussian process sequential design, which lack closed-form solutions and are thus expensive to evaluate. The authors propose an efficient approximation method grounded in Hilbert space Gaussian processes, leveraging truncated feature expansions to derive, for the first time, a closed-form IMSE approximation with rigorous non-asymptotic error bounds for isotropic kernels. A γ-stabilization strategy is incorporated to ensure numerical stability. The resulting approach achieves provably accurate approximations while substantially accelerating computation. Empirical evaluations across multiple benchmark tasks demonstrate that the method not only reduces predictive error but also significantly shortens runtime, outperforming existing alternatives in both accuracy and efficiency.

Technology Category

• Machine Learning: Kernel Methods
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Gaussian processes are widely used for accurate emulation of unknown surfaces in sequential design of expensive simulation experiments. Integrated mean squared error (IMSE) is an effective acquisition function for sequential designs based on Gaussian processes. However, existing approaches struggle with its implementation because the required integrals often lack closed-form expressions for most kernel functions. We propose a novel and computationally efficient Hilbert space Gaussian process approximation for the IMSE acquisition function, where a truncated eigenbasis representation of the integral enables closed-form evaluation. We establish sharp global non-asymptotic bounds for both the approximation error of isotropic kernels and the resulting error in the acquisition function. In a series of numerical experiments with $γ$-stabilizing, the proposed method achieves substantially lower prediction error and reduced computation time compared to existing benchmarks. These results demonstrate that the proposed Hilbert space Gaussian process framework provides an accurate and computationally efficient approach for Gaussian process based sequential design.

Problem

• sequential design
• Gaussian processes
• integrated mean squared error
• acquisition function
• kernel functions

Innovation

• Hilbert space Gaussian process
• IMSE acquisition function
• sequential design
• closed-form integration
• non-asymptotic error bounds