Robust, Online, and Adaptive Decentralized Gaussian Processes

2025-09-22

AI Summary

Gaussian processes (GPs) face three key limitations in large-scale, dynamic, and noisy settings: high computational complexity (O(N³)), restrictive stationarity assumptions, and insufficient robustness to outliers. To address these, this paper proposes a decentralized Random Fourier Features (RFF)-based GP framework formulated in information filter form. The method integrates a robust weighting mechanism to suppress outlier influence and introduces a dynamic adaptive strategy for tracking time-varying functions. Crucially, it enables stable, recursive online learning and globally consistent sequential inference—without requiring a central node—in fully distributed networks. Experiments on large-scale Earth system modeling demonstrate substantial improvements in prediction accuracy and system stability, while maintaining real-time adaptability and in-situ learning capability.

Technology Category

• Machine Learning: Kernel Methods
• Reasoning under Uncertainty: Relational Probabilistic Models
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Gaussian processes (GPs) offer a flexible, uncertainty-aware framework for modeling complex signals, but scale cubically with data, assume static targets, and are brittle to outliers, limiting their applicability in large-scale problems with dynamic and noisy environments. Recent work introduced decentralized random Fourier feature Gaussian processes (DRFGP), an online and distributed algorithm that casts GPs in an information-filter form, enabling exact sequential inference and fully distributed computation without reliance on a fusion center. In this paper, we extend DRFGP along two key directions: first, by introducing a robust-filtering update that downweights the impact of atypical observations; and second, by incorporating a dynamic adaptation mechanism that adapts to time-varying functions. The resulting algorithm retains the recursive information-filter structure while enhancing stability and accuracy. We demonstrate its effectiveness on a large-scale Earth system application, underscoring its potential for in-situ modeling.

Problem

• Addresses cubic scaling issues in Gaussian processes for large datasets
• Enhances robustness against outliers in dynamic noisy environments
• Enables adaptive modeling of time-varying functions in distributed systems

Innovation

• Robust filtering update downweights atypical observations
• Dynamic adaptation mechanism handles time-varying functions
• Retains recursive information-filter structure for distributed computation