🤖 AI Summary
Bayesian multiple testing correction for simultaneous comparisons among $n$ groups faces combinatorial explosion: the number of possible equality-constrained partitions equals the Bell number $ ext{Bell}(n)$—e.g., 115,975 for $n=10$—rendering exhaustive enumeration infeasible.
Method: We propose a scalable Bayesian framework using a Beta-binomial prior to model equality structures as group partitions, integrated with a Dirichlet process prior and a stochastic search algorithm for efficient posterior exploration.
Contribution/Results: Our approach enables, for the first time, joint inference on equality constraints over means, standard deviations, and proportions across the full partition space. We implement this methodology in EqualitySampler—a high-performance Julia package supporting large-scale, high-precision identification of equality structures. In both simulations and empirical applications, EqualitySampler significantly improves accuracy and interpretability of model selection under multiple comparisons.
📝 Abstract
Researchers frequently wish to assess the equality or inequality of groups, but this poses the challenge of adequately adjusting for multiple comparisons. Statistically, all possible configurations of equality and inequality constraints can be uniquely represented as partitions of groups, where any number of groups are equal if they are in the same subset of the partition. In a Bayesian framework, one can adjust for multiple comparisons by constructing a suitable prior distribution over all possible partitions. Inspired by work on variable selection in regression, we propose a class of flexible beta-binomial priors for multiple comparison adjustment. We compare this prior setup to the Dirichlet process prior suggested by Gopalan and Berry (1998) and multiple comparison adjustment methods that do not specify a prior over partitions directly. Our approach not only allows researchers to assess pairwise equality constraints but simultaneously all possible equalities among all groups. Since the space of possible partitions grows rapidly -- for ten groups, there are already 115,975 possible partitions -- we use a stochastic search algorithm to efficiently explore the space. Our method is implemented in the Julia package EqualitySampler, and we illustrate it on examples related to the comparison of means, standard deviations, and proportions.