🤖 AI Summary
This work addresses the challenge of combining probabilistic predictions from multiple models in few-shot settings by proposing a general weighted averaging method grounded in a minimum divergence framework. Applicable to models constructed via frequentist, Bayesian, or other fitting paradigms, the approach employs a dual-motivation weighting scheme that simultaneously minimizes the divergence between the aggregated predictive distribution and the true data-generating distribution while accounting for model complexity. Theoretical analysis elucidates the source of its advantage under limited data regimes, and empirical evaluations demonstrate that the method consistently matches or significantly outperforms conventional model averaging strategies—such as Akaike weights and stacking—in terms of predictive accuracy, exhibiting robust performance across diverse scenarios.
📝 Abstract
This paper uses a minimum divergence framework to introduce a new way of calculating model weights that can be used to average probabilistic predictions from statistical and machine learning models. The method is general and can be applied regardless of whether the models under consideration are fit to data using frequentist, Bayesian, or some other fitting method. The proposed method is motivated in two different ways and is shown empirically to perform better than or on a par with standard model averaging methods, including model stacking and model averaging that relies on Akaike-style negative exponentiated model weighting, especially when the sample size is small. Our theoretical analysis explains why the method has a small-sample advantage.