Generalized Matheron Variational Implicit Processes

2026-10-06

AI Summary

This study addresses the challenge of posterior inference arising from the intractable density of function spaces under implicit process priors. To this end, we propose GMVIP, a path-wise variational family that constructs posterior samples via inducing point corrections. By leveraging Matheron’s rule and whitened inducing coefficients, our approach enables efficient computation of the Kullback–Leibler divergence in coefficient space. Crucially, GMVIP provides a unified framework for both Gaussian processes and general implicit priors while strictly preserving the variational structure of the original process. Experimental results demonstrate that GMVIP achieves superior performance across regression, classification, and forecasting tasks, consistently matching or surpassing state-of-the-art methods.

Abstract

Implicit-process priors specify distributions over functions through sample-forward mechanisms such as Bayesian neural networks and stochastic simulators, but their function-space densities are typically unavailable. We introduce Generalized Matheron Variational Implicit Processes (GMVIP), a pathwise variational family for posterior inference with such priors. For Gaussian-process priors, GMVIP recovers the standard inducing-variable variational GP construction; for general implicit priors, its empirical covariance construction preserves the prior mean and covariance in the population limit. GMVIP constructs posterior samples by drawing a function from the prior and applying a correction anchored at a set of inducing inputs. The effect of this correction away from the inducing inputs is determined directly from prior samples, allowing the posterior to retain the structure and variability of the original implicit process. The (surrogate) prior and variational posterior use the same pathwise construction and differ only in the distribution of whitened inducing coefficients, yielding a tractable coefficient-space Kullback-Leibler divergence. Experiments on regression, classification, and forecasting with simulator-defined and retrieval-conditioned empirical trajectory priors show that GMVIP is broadly competitive with existing methods.

Problem

• implicit-process priors
• posterior inference
• variational inference
• Bayesian neural networks
• stochastic simulators

Innovation

• Implicit Processes
• Variational Inference
• Pathwise Construction
• Inducing Variables
• Matheron's Rule