gaussian process regression

This work addresses the scalability challenge in Gaussian process quantile regression arising from non-conjugate likelihoods by proposing an efficient inference framework based on sparse Gaussian processes. The method introduces inducing variables to model the quantile function and leverages Laplace approximation for scalable posterior inference. It further decomposes predictive uncertainty into a conditional prior variance and a posterior inducing variance, enabling a novel dual-adaptive mechanism that dynamically optimizes inducing point locations and guides sequential data acquisition. Experimental results demonstrate that the proposed approach maintains the accuracy of Laplace approximation while significantly outperforming fixed sampling strategies, offering clear advantages in both inducing point placement efficiency and model complexity control.

This paper addresses the challenge of posterior inference in large-scale nonparametric regression. We propose a spatially adaptive distributed Gaussian process (GP) approximation method. The approach partitions the input space into disjoint subsets, fits independent GP posteriors on each subset using a Matérn kernel and an integrated Brownian motion prior, and incorporates a Bayesian prior on the length scale to enhance regularization of local smoothness. A novel weighted spatial aggregation scheme is then introduced to fuse these sub-posteriors into a global approximation of the full-data posterior. Theoretically, we establish that the resulting approximate posterior achieves a convergence rate that automatically adapts to the local smoothness of the true regression function—matching the minimax optimal rate attainable under the full-data setting. Empirically, our method significantly outperforms existing distributed GP approaches on both synthetic and real-world datasets, while effectively capturing heterogeneous local regularity and overcoming the smoothness rigidity inherent in standard GP models.

Gaussian process regression (GPR) is often treated as a black-box surrogate model, limiting its pedagogical utility and interpretability in uncertainty quantification (UQ) for beginners. Core UQ tasks—including uncertainty propagation, risk estimation, Bayesian optimization, parameter inference, and sensitivity analysis—require deeper engagement with GPR’s inherent probabilistic structure. Method: This work develops a systematic, pedagogically grounded GPR-based UQ framework that integrates UQ-specific techniques: Bayesian quadrature, active learning, and surrogate-based sensitivity analysis. It emphasizes principled covariance kernel design, Bayesian hyperparameter estimation, and reproducible implementation. Contribution/Results: The framework lowers the barrier to applying GPR in complex UQ scenarios, enhances model transparency and decision reliability, and provides a theoretically rigorous yet practically actionable paradigm for UQ education and research across engineering and scientific disciplines.

Gaussian processes (GPs) struggle to rigorously incorporate uncountably infinite-dimensional functional prior information—such as boundary conditions or global physical constraints satisfied by PDE solutions. Method: This paper proposes a unified modeling framework grounded in reproducing kernel Hilbert spaces (RKHS), establishing for the first time a rigorous equivalence between the GP conditional expectation and orthogonal projection in RKHS. This enables direct embedding of functional constraints (e.g., Dirichlet or Neumann boundary conditions) into the GP prior, bypassing conventional pseudo-point approximations. Contribution/Results: We provide theoretical guarantees on existence, uniqueness, and convergence of the constrained GP posterior. Computationally, we design a practical numerical approximation algorithm. Experiments on PDE inverse problems demonstrate substantial improvements in uncertainty quantification accuracy and posterior consistency. The framework delivers a rigorous, general, and computationally tractable paradigm for integrating domain knowledge into Bayesian modeling.

Standard Gaussian processes (GPs) in Bayesian optimization suffer from inaccurate predictions in target regions (e.g., low-function-value areas) due to their inherent stationarity assumption, which fails on non-stationary objective functions. Method: We propose relaxed Gaussian processes (reGP), a novel GP modeling framework that weakens interpolation constraints outside the region of interest while enforcing bounded mean predictions within it. reGP further introduces a target-region-weighted prediction mechanism. Contribution/Results: This is the first systematic integration of goal-oriented principles into GP kernel design. We theoretically prove that reGP—when combined with the Expected Improvement (EI) acquisition function—guarantees global convergence. Leveraging reproducing kernel Hilbert space (RKHS) analysis, reGP significantly improves predictive accuracy and convergence speed on non-stationary functions. Empirical evaluation across diverse benchmark tasks demonstrates that reGP consistently outperforms standard stationary GPs, yielding higher-quality optima with fewer iterations.

This work addresses the challenges in high-dimensional Gaussian process regression, where joint inference of multiple length-scale hyperparameters is difficult and weakly informative input variables are hard to identify. The authors propose a time-step-dependent adaptive Wishart prior that operates directly on the covariance matrix rather than on individual hyperparameters. The scale matrix of this prior is dynamically constructed using the history of Markov chain Monte Carlo (MCMC) iterations, enabling efficient Bayesian learning of hyperparameters. This approach significantly enhances the model’s ability to detect irrelevant inputs, improves learning stability, and strengthens diagnostic performance. Empirical evaluations on both synthetic and real-world datasets demonstrate the effectiveness of the proposed prior.

This work proposes an empirical Gaussian process framework that overcomes the limitations of traditional Gaussian processes, which rely on handcrafted kernel functions and struggle to capture complex covariance structures in real-world data. By directly learning both mean and covariance functions from multi-source, heterogeneous historical datasets, the method constructs a data-driven prior that transcends the constraints of parametric kernels. An expectation–maximization (EM) algorithm with closed-form updates is derived via likelihood estimation, effectively addressing the challenge of inconsistent observation locations across datasets and approximating the true data-generating process in terms of KL divergence. Empirical evaluations demonstrate competitive performance in learning curve extrapolation and time series forecasting tasks.

This work addresses the computational bottleneck of large-scale Gaussian processes, whose O(n³) complexity hinders efficient prediction. The authors propose a conditioning strategy based on carefully constructed data contrasts that exploits the low-rank structure of covariance matrices induced by smooth kernels. Within connected domains, the full conditional distribution can be accurately approximated using only a small number of linear combinations. This approach achieves O(T r²) offline precomputation and O(1) online prediction complexity at arbitrary locations, enabling real-time responses to unseen query points. Remarkably, the method attains near-linear overall computational cost while preserving machine-precision accuracy.

Standard Gaussian process regression tends to yield overconfident posterior intervals and biased decisions when inputs are subject to measurement errors. To address this issue, this work models noisy inputs as probability measures and proposes the Deterministic Projection Wasserstein ARD Gaussian Process (PWAGP). By leveraging the Wasserstein distance, the method constructs a closed-form, positive-definite, and scalable covariance function for distributional inputs. Unlike approaches relying on latent variables or Monte Carlo sampling, PWAGP avoids stochastic approximations, thereby enhancing the transparency and robustness of uncertainty quantification. The resulting framework maintains computational efficiency while significantly improving predictive reliability under input noise.

Gaussian process (GP) regression suffers from high computational complexity on autocorrelated data (e.g., time-series or spatial data), and conventional approximation methods often fail due to overfitting in temporal or structural dependencies. Method: This paper proposes a fast GP approximation framework tailored for block-wise decorrelated data. Its core innovation lies in the systematic adaptation of sparse GP methods and structured covariance decomposition techniques to data preprocessed via blocking and decorrelation—ensuring theoretical consistency while mitigating temporal overfitting. Contribution/Results: The method achieves both computational efficiency and high predictive accuracy. On multiple real-world autocorrelated datasets, it accelerates inference by one to two orders of magnitude compared to exact GP inference, while attaining prediction errors comparable to those of exact GP and significantly outperforming state-of-the-art GP approximations. This work establishes a new paradigm for scalable GP modeling under non-i.i.d. settings.