Low-rank computation of the posterior mean in Multi-Output Gaussian Processes

2025-04-30

AI Summary

Standard inference for multi-output Gaussian processes (MOGPs) suffers from prohibitive computational complexity in computing posterior means, especially for large-scale spatiotemporal data. Method: This paper proposes an efficient inference framework tailored to separable spatiotemporal covariance structures. It reformulates noisy MOGP inference as solving a large-scale Stein equation and introduces the first integration of low-rank preconditioned conjugate gradient (LRPCG) with a Kronecker-product-based iterative solver (KPIK) specifically designed for Stein equations. Additionally, a degree-weighted average covariance matrix is incorporated to accelerate convergence. The method synergistically combines low-rank approximation, Kronecker factorization, and graph-filtering-based covariance modeling. Results: Evaluated on real-world road network data, the approach significantly reduces memory footprint and accelerates posterior mean computation by an order of magnitude, enabling scalable, real-time MOGP inference—a novel paradigm for large-scale multi-output spatial statistics.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Gaussian processes (GP) are a versatile tool in machine learning and computational science. We here consider the case of multi-output Gaussian processes (MOGP) and present low-rank approaches for efficiently computing the posterior mean of a MOGP. Starting from low-rank spatio-temporal data we consider a structured covariance function, assuming separability across space and time. This separability, in turn, gives a decomposition of the covariance matrix into a Kronecker product of individual covariance matrices. Incorporating the typical noise term to the model then requires the solution of a large-scale Stein equation for computing the posterior mean. For this, we propose efficient low-rank methods based on a combination of a LRPCG method with the Sylvester equation solver KPIK adjusted for solving Stein equations. We test the developed method on real world street network graphs by using graph filters as covariance matrices. Moreover, we propose a degree-weighted average covariance matrix, which can be employed under specific assumptions to achieve more efficient convergence.

Problem

• Efficient low-rank computation of MOGP posterior mean
• Solving large-scale Stein equation for structured covariance
• Improving convergence with degree-weighted average covariance matrix

Innovation

• Low-rank methods for MOGP posterior mean
• Kronecker product decomposition of covariance
• LRPCG and KPIK for Stein equation solution