embedding-space gp regression

Estimating predictive uncertainty for quantized neural networks in safety-critical applications remains challenging, particularly when model parameters or gradients are inaccessible. Method: We propose a post-hoc uncertainty calibration method that requires only the backbone network’s raw inputs and its frozen outputs—no parameter access or gradient computation. For regression tasks, we formulate an input–frozen-output joint conditional framework to directly infer Gaussian posterior distribution parameters. We introduce the first formally falsifiable posterior optimization objective, rigorously characterize conditions for output-driven uncertainty estimation validity, and theoretically prove that frozen outputs encode generalization error information. Using maximum likelihood estimation and sequential fitting, we train lightweight regression surrogates on data-augmented subsets. Results: Our method significantly improves out-of-distribution detection and probabilistic calibration on UCI and deep regression benchmarks, preserves input-dependent uncertainty modeling, incurs zero sampling or approximation overhead during inference, and maintains the backbone’s original prediction accuracy.

This study addresses the miscalibration of Gaussian processes (GPs) in uncertainty quantification (UQ)—specifically, their frequent lack of probabilistic calibration, which undermines convergence in downstream tasks such as Bayesian optimization. To tackle this, we propose Kernel Covariance Validation (KCV), the first framework to systematically exploit the multivariate normal structure of GP predictions for interpretable, computationally tractable calibration diagnostics. KCV integrates multivariate normality testing, uncertainty calibration analysis, and adaptive target design to quantitatively detect and localize model misspecification. Extensive experiments across 1D to high-dimensional GPs demonstrate KCV’s ability to identify canonical calibration failure modes. Results show that KCV significantly improves predictive reliability and enhances both the convergence speed and stability of optimization algorithms. By enabling principled, diagnostic-driven calibration, KCV establishes a new paradigm for trustworthy UQ in GP-based modeling.

In Gaussian process (GP) regression, systematic input bias errors—e.g., from drifting mobile sensor localization—degrade prediction accuracy; existing approaches require full model retraining upon input correction, incurring prohibitive computational cost. Method: We propose a training-free online GP model refinement method that dynamically corrects for time-varying input biases. Leveraging the differentiability of the squared-exponential kernel, we introduce a second-order Taylor expansion of the GP predictive mean and variance with respect to input perturbations, efficiently computed via pre-estimated Jacobian and Hessian matrices of the kernel. Input bias is estimated in real time using a Kalman filter. Contribution/Results: The method significantly improves both point prediction accuracy and uncertainty quantification quality. In two simulation studies, it achieves millisecond-scale model refinement without retraining, while relaxing the restrictive assumption of zero-mean input noise inherent in conventional GP formulations.

This paper addresses the challenge of posterior inference in large-scale nonparametric regression. We propose a spatially adaptive distributed Gaussian process (GP) approximation method. The approach partitions the input space into disjoint subsets, fits independent GP posteriors on each subset using a Matérn kernel and an integrated Brownian motion prior, and incorporates a Bayesian prior on the length scale to enhance regularization of local smoothness. A novel weighted spatial aggregation scheme is then introduced to fuse these sub-posteriors into a global approximation of the full-data posterior. Theoretically, we establish that the resulting approximate posterior achieves a convergence rate that automatically adapts to the local smoothness of the true regression function—matching the minimax optimal rate attainable under the full-data setting. Empirically, our method significantly outperforms existing distributed GP approaches on both synthetic and real-world datasets, while effectively capturing heterogeneous local regularity and overcoming the smoothness rigidity inherent in standard GP models.

Standard Gaussian processes (GPs) in Bayesian optimization suffer from inaccurate predictions in target regions (e.g., low-function-value areas) due to their inherent stationarity assumption, which fails on non-stationary objective functions. Method: We propose relaxed Gaussian processes (reGP), a novel GP modeling framework that weakens interpolation constraints outside the region of interest while enforcing bounded mean predictions within it. reGP further introduces a target-region-weighted prediction mechanism. Contribution/Results: This is the first systematic integration of goal-oriented principles into GP kernel design. We theoretically prove that reGP—when combined with the Expected Improvement (EI) acquisition function—guarantees global convergence. Leveraging reproducing kernel Hilbert space (RKHS) analysis, reGP significantly improves predictive accuracy and convergence speed on non-stationary functions. Empirical evaluation across diverse benchmark tasks demonstrates that reGP consistently outperforms standard stationary GPs, yielding higher-quality optima with fewer iterations.

This work proposes an embedded model error correction framework to address prediction biases arising from simplifying assumptions in computational models of physical systems, which often introduce errors entangled with model parameters. The approach characterizes the spatiotemporal correlations of model discrepancy using a weight-space representation of Gaussian processes and incorporates orthogonality constraints tailored for nonlinear systems to effectively decouple model and error parameters. By integrating a likelihood-informed subspace method, the framework efficiently handles high-dimensional problems. Under Bayesian inference, the method successfully corrects predictions to align with observed trends in both linear and nonlinear test cases, reverts to the prior predictive distribution during extrapolation, and yields nearly uncorrelated posteriors for model and error parameters—thereby substantially enhancing model reliability and interpretability.

This work addresses the scalability challenge in Gaussian process quantile regression arising from non-conjugate likelihoods by proposing an efficient inference framework based on sparse Gaussian processes. The method introduces inducing variables to model the quantile function and leverages Laplace approximation for scalable posterior inference. It further decomposes predictive uncertainty into a conditional prior variance and a posterior inducing variance, enabling a novel dual-adaptive mechanism that dynamically optimizes inducing point locations and guides sequential data acquisition. Experimental results demonstrate that the proposed approach maintains the accuracy of Laplace approximation while significantly outperforming fixed sampling strategies, offering clear advantages in both inducing point placement efficiency and model complexity control.

Standard Gaussian process regression tends to yield overconfident posterior intervals and biased decisions when inputs are subject to measurement errors. To address this issue, this work models noisy inputs as probability measures and proposes the Deterministic Projection Wasserstein ARD Gaussian Process (PWAGP). By leveraging the Wasserstein distance, the method constructs a closed-form, positive-definite, and scalable covariance function for distributional inputs. Unlike approaches relying on latent variables or Monte Carlo sampling, PWAGP avoids stochastic approximations, thereby enhancing the transparency and robustness of uncertainty quantification. The resulting framework maintains computational efficiency while significantly improving predictive reliability under input noise.

This work addresses the challenge of applying Gaussian processes (GPs) to high-dimensional inputs, where they are prone to the curse of dimensionality, and existing two-stage dimensionality reduction approaches often compromise either predictive accuracy or reliable uncertainty quantification. To overcome this limitation, the authors propose an end-to-end Bayesian joint modeling framework that seamlessly integrates input dimensionality reduction within the GP formulation. By placing a prior on the Stiefel manifold to enforce orthogonality of the projection matrix and employing Riemannian Hamiltonian Monte Carlo for posterior inference along geodesics, the method achieves, for the first time, a unified Bayesian treatment of GP regression and dimensionality reduction. The framework is further extended to deep Gaussian processes to enhance representational capacity. Experimental results demonstrate superior performance over conventional two-stage methods in both prediction accuracy and uncertainty calibration, albeit at increased computational cost.