soft-voting ensembles

This study investigates optimal parallel heterogeneous ensemble strategies for small- to medium-scale tabular classification tasks. Through systematic experiments on 56 OpenML CC18 datasets, the authors find that the instability of Blending and Stacking is mutually independent, and that Robust Soft Voting significantly outperforms Hard Voting in multi-class settings. Building on these insights, they propose a robust ensemble strategy and validate its effectiveness on the TabArena platform. The recommended approach significantly surpasses the Single Best method across 28 additional tasks and matches or exceeds the performance of all individual ensemble techniques considered.

To address the unreliability of prediction confidence in deep neural network image classification, this paper proposes a lightweight meta-model classifier ensemble method that achieves efficient uncertainty calibration without requiring additional calibration data. The approach constructs a parameter-efficient meta-model to fuse outputs from multiple base classifiers and jointly evaluates calibration performance using majority voting alongside Expected Calibration Error (ECE) and Maximum Calibration Error (MCE). For the first time, both theoretical analysis and empirical evaluation demonstrate its significant calibration advantages: across diverse mainstream CNN architectures, it reduces ECE and MCE by over 40% on average while preserving classification accuracy nearly unchanged; moreover, its parameter count is 3–5× smaller than conventional model ensembles. This work establishes a novel paradigm for high-reliability, low-overhead model calibration.

Existing ensemble methods typically rely on a single evaluation criterion for weight assignment, failing to fully exploit multidimensional prior performance information of base classifiers—thus limiting overall model performance. This paper proposes a cooperative-game-theoretic multi-criteria weighted ensemble learning framework. It is the first to introduce the Shapley value into multi-criteria ensemble learning, quantifying each classifier’s marginal contribution across multiple dimensions—including accuracy, stability, and class-wise sensitivity—and integrating these via multi-criteria decision analysis for dynamic weight allocation. The approach ensures both comprehensiveness and fairness in decision-making. Evaluated on the OpenML-CC18 benchmark, it significantly outperforms mainstream weighted ensemble methods, achieving improvements in classification accuracy, robustness, and generalization ability, while effectively mitigating class imbalance and overfitting.

This work addresses the lack of a unified Python framework for ensemble learning methods grounded in Composite Fusion Analysis (CFA), particularly in integrating Rank-Score Characteristic (RSC) functions with Cognitive Diversity (CD). To bridge this gap, we propose InFusionLayer—a general-purpose machine learning architecture inspired by CFA that, for the first time, unifies RSC and CD mechanisms within a single framework compatible with PyTorch, TensorFlow, and Scikit-learn. Requiring only a small set of base models, our approach achieves substantial performance gains in both unsupervised and supervised multi-class classification tasks. Extensive experiments across multiple computer vision benchmarks validate its efficacy, and the open-sourced implementation facilitates the practical adoption and broader dissemination of CFA within mainstream deep learning ecosystems.

Reliable decision-making demands well-calibrated uncertainty estimates, yet large language models often exhibit overconfidence, and explicit ensemble methods incur prohibitive computational and memory overhead. To address this, we propose a parameter-efficient implicit ensemble framework: it shares a pretrained Transformer backbone while assigning independent low-rank self-attention projection matrices—implemented via LoRA—to each ensemble member, marking the first extension of LoRA to implicit ensembling. Crucially, our method introduces zero additional memory overhead during inference and incurs computational cost nearly identical to that of a single model. On multiple benchmarks, it achieves prediction accuracy competitive with explicit ensembles while substantially improving calibration—reducing Expected Calibration Error (ECE) by up to 37%—outperforming state-of-the-art implicit approaches such as BatchEnsemble.

This work proposes a sequential testing–based early-stopping strategy for binary ensemble classifiers to reduce inference overhead while strictly bounding the divergence rate from predictions of the full ensemble. The approach terminates evaluation as soon as a decisive majority emerges during the sequential assessment of base models. Under three optimality criteria, the strategy can be formulated as a linear programming problem, enabling efficient computation of the optimal stopping rule. Experimental results on UCI and Grinsztajn benchmark datasets demonstrate that the method achieves an average speedup exceeding 4× while consistently maintaining prediction divergence below 0.1%.

This work proposes an efficient Bayesian deep ensemble method to address the limitations of existing deep ensembles in uncertainty calibration and interpretability. By leveraging low-dimensional predictive representations, independent training strategies, and a closed-form Bayesian linear regression aggregation mechanism, the approach enables analytical posterior weight inference while maintaining high predictive accuracy. This significantly enhances model interpretability and yields well-calibrated uncertainty estimates. Notably, the computational complexity shifts from scaling with dataset size to depending only on ensemble size, substantially improving scalability. Empirical evaluations on standard regression benchmarks demonstrate that the method achieves state-of-the-art predictive performance while providing reliable and properly calibrated uncertainty quantification.

This work proposes a general, model-agnostic post-processing framework that systematically integrates ensemble learning into fairness optimization. By aggregating predictions from multiple base models without requiring access to their internal architectures, the approach uniformly supports diverse predictive tasks—including classification, regression, and survival analysis—and accommodates a wide range of fairness definitions. The method addresses the inherent trade-off between predictive performance and fairness in machine learning models. Experimental results demonstrate that it significantly enhances fairness while maintaining or only marginally compromising prediction accuracy, thereby validating its effectiveness and generalizability across multiple scenarios.

This study addresses class imbalance and insufficient model reliability in the four-class classification of liquid-based cervical cytology (LBC) under the Bethesda system. For the first time, it separates training, calibration, and evaluation data, employing weighted random sampling to train lightweight models—including Swin-Tiny, TinyViT-5M, and DenseNet121—and combines them via soft voting ensemble, followed by temperature scaling calibration on an independent subset. Experimental results demonstrate that calibration substantially reduces expected calibration error, Brier score, and negative log-likelihood while maintaining stable discriminative performance. Classification of NILM and LSIL is perfectly accurate, with errors occurring only at the boundary between HSIL and SCC. The work highlights the critical role of calibration in enhancing reliability and finds that increasing ensemble size yields no significant additional benefit after calibration.

This work addresses the challenge of modeling predictive uncertainty in subjective NLP tasks, where annotator disagreement introduces systematic divergence that conventional models struggle to capture. The authors propose a novel framework that optimizes ensemble diversity directly in prediction space, jointly learning ensemble weights, effective ensemble size, and calibration within an end-to-end training paradigm. A signed diversity regularizer is introduced to controllably preserve or suppress disagreement, thereby preventing ensemble collapse. The approach integrates Gumbel-Softmax relaxation, a soft F1 surrogate loss, class-weighted cross-entropy, and reliability-weighted diversity regularization. Evaluated on four subjective text classification benchmarks, the model achieves substantially improved probability calibration—reducing cross-entropy by 40–78% and outperforming multiple baselines in Brier score—while maintaining competitive F1 performance and more accurately aligning with the underlying annotator distribution.