Semi-Implicit Approaches for Large-Scale Bayesian Spatial Interpolation

2025-10-22

AI Summary

Gaussian processes (GPs) suffer from cubic computational complexity $O(n^3)$ in spatial statistics, severely limiting scalability to large datasets. To address this, we propose a scalable Bayesian interpolation framework that integrates nearest-neighbor Gaussian processes (NNGPs) with semi-implicit variational inference (SIVI). NNGP approximates the full GP covariance structure via sparse conditional independence, while SIVI enables efficient, fully parametric variational inference over all hyperparameters and latent variables. The framework further incorporates automatic differentiation and Pathfinder initialization to accelerate optimization. Experiments demonstrate substantial gains: on 500 training points, inference time drops from 6 hours (HMC) to 130 seconds; for a temperature mapping task across 150,000 spatial locations, full posterior estimation completes in under 2 minutes. This represents the first demonstration of minute-scale, fully Bayesian inference at the million-point scale.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Probabilistic Inference
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Spatial statistics often rely on Gaussian processes (GPs) to capture dependencies across locations. However, their computational cost increases rapidly with the number of locations, potentially needing multiple hours even for moderate sample sizes. To address this, we propose using Semi-Implicit Variational Inference (SIVI), a highly flexible Bayesian approximation method, for scalable Bayesian spatial interpolation. We evaluated SIVI with a GP prior and a Nearest-Neighbour Gaussian Process (NNGP) prior compared to Automatic Differentiation Variational Inference (ADVI), Pathfinder, and Hamiltonian Monte Carlo (HMC), the reference method in spatial statistics. Methods were compared based on their predictive ability measured by the CRPS, the interval score, and the negative log-predictive density across 50 replicates for both Gaussian and Poisson outcomes. SIVI-based methods achieved similar results to HMC, while being drastically faster. On average, for the Poisson scenario with 500 training locations, SIVI reduced the computational time from roughly 6 hours for HMC to 130 seconds. Furthermore, SIVI-NNGP analyzed a simulated land surface temperature dataset of 150,000 locations while estimating all unknown model parameters in under two minutes. These results highlight the potential of SIVI as a flexible and scalable inference technique in spatial statistics.

Problem

• Addressing high computational costs of Gaussian processes in spatial statistics
• Proposing scalable Bayesian interpolation using semi-implicit variational inference
• Achieving comparable accuracy to reference methods with drastically reduced runtime

Innovation

• Uses Semi-Implicit Variational Inference for scalability
• Combines SIVI with Gaussian Process and NNGP priors
• Achieves comparable accuracy to HMC with drastic speedup