🤖 AI Summary
This work addresses the challenges in high-dimensional Gaussian process regression, where joint inference of multiple length-scale hyperparameters is difficult and weakly informative input variables are hard to identify. The authors propose a time-step-dependent adaptive Wishart prior that operates directly on the covariance matrix rather than on individual hyperparameters. The scale matrix of this prior is dynamically constructed using the history of Markov chain Monte Carlo (MCMC) iterations, enabling efficient Bayesian learning of hyperparameters. This approach significantly enhances the model’s ability to detect irrelevant inputs, improves learning stability, and strengthens diagnostic performance. Empirical evaluations on both synthetic and real-world datasets demonstrate the effectiveness of the proposed prior.
📝 Abstract
In probabilstic supervised learning of an input-output relationship - as a sample function of a Gaussian Process (GP) - priors are typically specified for the hyperparameters of the kernel that parametrises the covariance function of the GP, where the induced covariance matrix of the (resulting multivariate Normal) likelihood, governs the learning and prediction. When the sought function is highly multivariate, multiple lengthscale parameters must be learnt simultaneously, making inference difficult. We develop a ``self-assembled'' Wishart prior for the covariance matrix, while undertaking Bayesian inference on the kernel hyperparameters using MCMC. The construction uses a look-back window over recent MCMC iterations to define a time-step dependent scale matrix, thereby introducing adaptiveness to the chain. Results suggest that direct prior specification on the covariance matrix can be useful for diagnosing weakly informative inputs within the GP-based learning paradigm. We support our prior development with two distinct empirical illustrations - one on synthetic data, and another on a real-world dataset.