FBMS: An R Package for Flexible Bayesian Model Selection and Model Averaging

2025-08-31

AI Summary

Bayesian generalized nonlinear models (BGNLMs) suffer from multimodal posteriors, complicating model selection and Bayesian model averaging (BMA). Method: We propose GMJMCMC—a novel algorithm integrating mode-jumping MCMC with genetic evolutionary strategies to enable efficient exploration across model spaces; a learnable nonlinear feature generation mechanism that automatically constructs flexible basis functions for approximating complex response surfaces; and precise posterior probability estimation coupled with cooperative co-evolution of model ensembles to enhance mixture stability. Results: Evaluated on benchmark tasks including Gaussian regression, GMJMCMC consistently outperforms conventional BMA and point-estimation approaches, achieving significant improvements in predictive accuracy, posterior coverage calibration, and structural discovery capability—demonstrating robustness and scalability in high-dimensional nonlinear modeling.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations

Abstract

The FBMS R package facilitates Bayesian model selection and model averaging in complex regression settings by employing a variety of Monte Carlo model exploration methods. At its core, the package implements an efficient Mode Jumping Markov Chain Monte Carlo (MJMCMC) algorithm, designed to improve mixing in multi-modal posterior landscapes within Bayesian generalized linear models. In addition, it provides a genetically modified MJMCMC (GMJMCMC) algorithm that introduces nonlinear feature generation, thereby enabling the estimation of Bayesian generalized nonlinear models (BGNLMs). Within this framework, the algorithm maintains and updates populations of transformed features, computes their posterior probabilities, and evaluates the posteriors of models constructed from them. We demonstrate the effective use of FBMS for both inferential and predictive modeling in Gaussian regression, focusing on different instances of the BGNLM class of models. Furthermore, through a broad set of applications, we illustrate how the methodology can be extended to increasingly complex modeling scenarios, extending to other response distributions and mixed effect models.

Problem

• Facilitates Bayesian model selection and averaging in complex regression settings
• Implements efficient MJMCMC algorithm for multi-modal posterior landscapes
• Enables nonlinear feature generation for generalized nonlinear models

Innovation

• Mode Jumping MCMC algorithm for Bayesian models
• Genetically modified MJMCMC for nonlinear features
• Flexible Bayesian model selection and averaging