metabeta - A fast neural model for Bayesian mixed-effects regression

2025-10-08

AI Summary

To address the computational expense and lack of real-time applicability of MCMC-based inference in Bayesian mixed-effects regression, this paper introduces MetaBeta—a Transformer-based neural posterior estimation framework for rapid Bayesian inference on grouped hierarchical data. The approach shifts computation-intensive operations to an offline pretraining phase, where a neural network learns the mapping from data to posterior distributions via simulation-based training, enabling millisecond-scale online inference. On both synthetic and real-world datasets, MetaBeta matches the accuracy and stability of traditional MCMC while accelerating inference by two to three orders of magnitude. Its core contribution lies in the first systematic integration of neural posterior estimation into Bayesian hierarchical modeling, thereby overcoming the long-standing scalability–practicality trade-off inherent in mixed-effects models.

Technology Category

• Machine Learning: Bayesian Learning
• Search and Optimization: Metareasoning and Metaheuristics
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Hierarchical data with multiple observations per group is ubiquitous in empirical sciences and is often analyzed using mixed-effects regression. In such models, Bayesian inference gives an estimate of uncertainty but is analytically intractable and requires costly approximation using Markov Chain Monte Carlo (MCMC) methods. Neural posterior estimation shifts the bulk of computation from inference time to pre-training time, amortizing over simulated datasets with known ground truth targets. We propose metabeta, a transformer-based neural network model for Bayesian mixed-effects regression. Using simulated and real data, we show that it reaches stable and comparable performance to MCMC-based parameter estimation at a fraction of the usually required time.

Problem

• Develops fast neural model for Bayesian mixed-effects regression
• Addresses computational inefficiency of MCMC methods in inference
• Provides rapid parameter estimation for hierarchical data analysis

Innovation

• Transformer-based neural network for Bayesian regression
• Amortizes computation via pre-training on simulated datasets
• Achieves comparable accuracy to MCMC with faster speed