Feature Space Adaptation for Effortless Gaussian Process Flows

2026-10-08

AI Summary

This study addresses the high computational cost of conditional sampling in nonlinear Gaussian processes and their reliance on manually specified kernel hyperparameters. We propose FlowGP, a framework that integrates feature-space adaptation with kernel approximation techniques. By leveraging diffusion steering to estimate marginal likelihoods for variational inference, this approach achieves, for the first time, automatic optimization of internal model hyperparameters. The proposed method substantially reduces dependence on manual tuning while enhancing scalability in high-resolution domains. FlowGP has been successfully applied to probabilistic downscaling, partial differential equation inference on irregular domains, and the reconstruction of sea level anomaly fields from non-Gaussian satellite observations.

Abstract

Outside the linear-Gaussian regime, conditional sampling from Gaussian processes (GPs) is challenging. Recent methods such as FlowGP (Moss et al., (2026)) can condition on arbitrary non-linear and non-Gaussian statements, but at considerable cost: an expensive iterative and high-dimensional diffusion that requires hand-specified kernel hyperparameters. In this paper, we alleviate two significant drawbacks of FlowGP by (1) introducing kernel approximations that enable scaling to high-resolution domains and (2) proposing a way to obtain the marginal likelihood by measuring the work needed to steer the diffusion towards conditioning statements. We enable, for the first time, hyperparameter optimisation within FlowGP and demonstrate our approach on probabilistic downscaling from areal summary statistics, PDE solution inference on irregular domains, and recovery of sea level anomaly fields from non-Gaussian satellite observations.

Problem

• Gaussian processes
• conditional sampling
• non-linear non-Gaussian conditioning
• hyperparameter optimization
• diffusion

Innovation

• Gaussian Process Flows
• Kernel Approximation
• Marginal Likelihood
• Hyperparameter Optimization
• Diffusion Models