BONE: a unifying framework for Bayesian online learning in non-stationary environments

2024-11-15

arXiv.org

AI Summary

This paper addresses the lack of a unified modeling framework for probabilistic online learning under nonstationary environments. We propose the Bayesian Online Nonstationary Learning framework (BONE), comprising three core components: an interpretable measurement model, an auxiliary process explicitly modeling nonstationarity, and a context-dependent conditional prior. To enable efficient online posterior updates, we design a two-layer variational inference algorithm. BONE is the first modular framework to unify diverse tasks—including continual learning, sequential forecasting, and contextual bandits—within a single Bayesian paradigm; it subsumes several existing methods and inspires novel algorithms while clarifying their applicability boundaries. Comprehensive evaluation across multiple benchmark datasets demonstrates BONE’s expressive power, generalization capability, and utility in guiding algorithm selection. The framework establishes a reusable theoretical foundation and practical guidelines for nonstationary online learning.

Technology Category

• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

We propose a unifying framework for methods that perform Bayesian online learning in non-stationary environments. We call the framework BONE, which stands for (B)ayesian (O)nline learning in (N)on-stationary (E)nvironments. BONE provides a common structure to tackle a variety of problems, including online continual learning, prequential forecasting, and contextual bandits. The framework requires specifying three modelling choices: (i) a model for measurements (e.g., a neural network), (ii) an auxiliary process to model non-stationarity (e.g., the time since the last changepoint), and (iii) a conditional prior over model parameters (e.g., a multivariate Gaussian). The framework also requires two algorithmic choices, which we use to carry out approximate inference under this framework: (i) an algorithm to estimate beliefs (posterior distribution) about the model parameters given the auxiliary variable, and (ii) an algorithm to estimate beliefs about the auxiliary variable. We show how this modularity allows us to write many different existing methods as instances of BONE; we also use this framework to propose a new method. We then experimentally compare existing methods with our proposed new method on several datasets; we provide insights into the situations that make one method more suitable than another for a given task.

Problem

• Develops a framework for Bayesian online learning in non-stationary environments.
• Addresses online continual learning, prequential forecasting, and contextual bandits.
• Provides modular structure for model and algorithmic choices in dynamic settings.

Innovation

• BONE framework for Bayesian online learning
• Modular structure for non-stationary environments
• Jax library for open-source implementation