A Bayesian Framework for Built-in Input Dimension Reduction for Gaussian Process Modeling

2026-07-21

AI Summary

This work addresses the challenge of applying Gaussian processes (GPs) to high-dimensional inputs, where they are prone to the curse of dimensionality, and existing two-stage dimensionality reduction approaches often compromise either predictive accuracy or reliable uncertainty quantification. To overcome this limitation, the authors propose an end-to-end Bayesian joint modeling framework that seamlessly integrates input dimensionality reduction within the GP formulation. By placing a prior on the Stiefel manifold to enforce orthogonality of the projection matrix and employing Riemannian Hamiltonian Monte Carlo for posterior inference along geodesics, the method achieves, for the first time, a unified Bayesian treatment of GP regression and dimensionality reduction. The framework is further extended to deep Gaussian processes to enhance representational capacity. Experimental results demonstrate superior performance over conventional two-stage methods in both prediction accuracy and uncertainty calibration, albeit at increased computational cost.

Abstract

Gaussian process (GP) modeling is widely used in computational science and engineering. However, fitting a GP to high-dimensional inputs remains challenging due to the curse of dimensionality. While various methods have been proposed to reduce input dimensionality, they typically follow a two-stage approach, performing dimension reduction and GP fitting separately. We introduce a Bayesian framework that seamlessly integrates dimensionality reduction with GP modeling and inference. Our approach, built on a hierarchical Bayesian model with priors on the Stiefel manifold, enforces orthonormality on the projection matrix and enables posterior inference via Hamiltonian Monte Carlo with geodesic flow. Additionally, we extend this framework by incorporating Deep Gaussian Processes (DGP) with built-in dimension reduction, providing a more flexible and powerful tool for complex datasets. Through extensive numerical studies, we demonstrate that while the proposed Bayesian method incurs higher computational costs, it improves predictive performance and uncertainty quantification, providing a principled and robust alternative to existing methods.

Problem

• Gaussian process
• dimensionality reduction
• high-dimensional inputs
• Bayesian framework
• curse of dimensionality

Innovation

• Bayesian framework
• Gaussian process
• dimensionality reduction
• Stiefel manifold
• Deep Gaussian Processes