Amortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models

2026-10-02

AI Summary

This study addresses the limitation of mean-field variational approximations in Gaussian Process Latent Variable Models (GPLVMs), which constrain manifold uncertainty estimation. To overcome this, we propose an amortized structured stochastic variational inference method. The core innovation lies in moving beyond the traditional mean-field assumption by leveraging neural network amortizers to model the conditional dependence between inducing points and latent variables. This enables the latent space posterior distribution to be conditioned on inducing points, thereby significantly enhancing model flexibility. Experimental results demonstrate that the proposed approach yields substantial improvements in both data manifold reconstruction metrics and the quality of uncertainty estimation. Consequently, this work establishes a superior structured Gaussian process inference paradigm for complex data representation.

Abstract

Many machine learning methods aim to approximate the lower-dimensional manifold on which the data lives. A desirable feature of such methods is that they should capture the epistemic uncertainty of this learned manifold. One model that achieves this is the Gaussian Process Latent Variable Model, in which a Gaussian Process (GP) mapping from the latent space provides an estimate of the uncertainty of the manifold. However, the effectiveness of this uncertainty estimation is limited by the mean-field variational approximation between the GP inducing points and the latent variables. In this work, we apply Amortized Structured Stochastic Variational Inference to allow the variational posterior for the latent space to be conditionally dependent on the value of the inducing points. We demonstrate that this more flexible variational posterior improves several metrics relating to the reconstruction of points on the data manifold.

Problem

• Gaussian Process Latent Variable Model
• Epistemic Uncertainty
• Mean-field Variational Approximation
• Manifold Learning

Innovation

• Gaussian Process Latent Variable Model
• Amortized Structured Stochastic Variational Inference
• Epistemic Uncertainty
• Variational Posterior
• Inducing Points