On the Choice of Model Space Priors and Multiplicity Control in Bayesian Variable Selection: An Application to Streaming Logistic Regression

2025-12-27

AI Summary

This paper investigates the impact of model-space priors on sparsity control and multiplicity correction in Bayesian variable selection (BVS), specifically within streaming logistic regression. We propose a decomposable approximation to the Matryoshka Doll (MD) prior, enabling independent modeling of inclusion indicators and scalable dynamic inference. Theoretically and empirically, we show that no single prior is universally optimal; the MD prior achieves an intermediate level of sparsity between standard Beta-Binomial priors, yielding a more balanced trade-off between sparsity and sensitivity. Its BIC-approximated marginal likelihood facilitates efficient posterior inclusion probability estimation. Compared to baseline methods, the MD prior significantly improves variable selection stability and coefficient estimation accuracy during both intermediate and final stages of streaming inference.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Bayesian variable selection (BVS) depends critically on the specification of a prior distribution over the model space, particularly for controlling sparsity and multiplicity. This paper examines the practical consequences of different model space priors for BVS in logistic regression, with an emphasis on streaming data settings. We review some popular and well-known Beta--Binomial priors alongside the recently proposed matryoshka doll (MD) prior. We introduce a simple approximation to the MD prior that yields independent inclusion indicators and is convenient for scalable inference. Using BIC-based approximations to marginal likelihoods, we compare the effect of different model space priors on posterior inclusion probabilities and coefficient estimation at intermediate and final stages of the data stream via simulation studies. Overall, the results indicate that no single model space prior uniformly dominates across scenarios, and that the recently proposed MD prior provides a useful additional option that occupies an intermediate position between commonly used Beta--Binomial priors with differing degrees of sparsity.

Problem

• Evaluating model space priors for Bayesian variable selection
• Comparing priors' effects on inclusion probabilities and estimation
• Assessing performance in streaming logistic regression contexts

Innovation

• Introduces scalable approximation to MD prior
• Compares model space priors via BIC approximations
• Evaluates priors in streaming logistic regression context