ensemble methods

Existing ensemble methods typically rely on a single evaluation criterion for weight assignment, failing to fully exploit multidimensional prior performance information of base classifiers—thus limiting overall model performance. This paper proposes a cooperative-game-theoretic multi-criteria weighted ensemble learning framework. It is the first to introduce the Shapley value into multi-criteria ensemble learning, quantifying each classifier’s marginal contribution across multiple dimensions—including accuracy, stability, and class-wise sensitivity—and integrating these via multi-criteria decision analysis for dynamic weight allocation. The approach ensures both comprehensiveness and fairness in decision-making. Evaluated on the OpenML-CC18 benchmark, it significantly outperforms mainstream weighted ensemble methods, achieving improvements in classification accuracy, robustness, and generalization ability, while effectively mitigating class imbalance and overfitting.

This paper addresses the poor generalizability and low stability of multiple pretrained models—particularly across ancestrally diverse populations—under label-free settings. We propose the first unsupervised robust ensemble framework explicitly designed for heteroscedastic and adversarial models. Methodologically, we leverage random matrix theory to devise a variance-stabilization mechanism and integrate iterative sparse signal recovery for unsupervised model evaluation and adaptive weighted fusion. Crucially, the framework dynamically calibrates heterogeneous model contributions without requiring target-domain labels, thereby substantially enhancing prediction robustness. In genetic risk prediction, our approach achieves superior cross-population generalization compared to state-of-the-art unsupervised and semi-supervised ensemble methods. It has been successfully deployed for integrating publicly available polygenic score (PGS) models in the PGS Catalog.

Conformal prediction for multi-class classification often suffers from inefficiency and overly large prediction sets due to reliance on a single scoring function. To address this, we propose a weighted ensemble of multiple scoring functions within the conformal prediction framework. Our method learns data-driven weights via joint optimization grounded in empirical risk minimization, integrating Vapnik–Chervonenkis (VC) theory with convex optimization. Crucially, we establish, for the first time, a theoretical connection between weighted score aggregation and VC subgraph classes—thereby enabling provably optimal multi-score fusion. Under strict coverage guarantees (e.g., 90%), our approach significantly reduces prediction set size, achieving an average reduction of 12.6% across multiple benchmark datasets. It consistently outperforms state-of-the-art single-score conformal methods in both efficiency and predictive performance.

In model merging, representational misalignment—modeled as an orthogonal transformation—exists between fused outputs and fine-tuned classifiers in the feature space, leading to evaluation distortion and suboptimal performance. To address this, we propose a novel few-shot unsupervised classifier alignment paradigm: using only a small number of unlabeled samples, it calibrates classifier weights via orthogonal transformation to achieve feature-space alignment. Based on this, we establish a more reliable evaluation protocol for merged models. Experiments across multiple classification tasks demonstrate that our method significantly improves the accuracy of merged models and yields evaluations that more faithfully reflect the intrinsic capabilities of merging methods. This work introduces a new benchmark for model merging that jointly ensures effectiveness and evaluability.

This study addresses the issue that standard evaluation mechanisms—such as win rate—can induce model homogenization in AI markets, thereby undermining consumer utility. To counter this, the authors propose a weighted win rate mechanism that incentivizes model specialization by offering differentiated rewards for high-quality responses. Drawing on game-theoretic and mechanism design frameworks, the work combines theoretical analysis with empirical validation using real-world benchmark data. The results demonstrate that the proposed mechanism effectively promotes model diversity while significantly enhancing consumer welfare, offering a principled approach to aligning model development incentives with user interests in competitive AI ecosystems.

This work addresses the sensitivity of conventional neurons—whose linear weighted summation is vulnerable to noise—and its detrimental impact on model robustness. To overcome this limitation, the study introduces, for the first time at the neuronal level, a learnable nonlinear aggregation mechanism by proposing F-Mean neurons and Gaussian-support neurons. A differentiable hybrid architecture is constructed to enable end-to-end training that adaptively interpolates between these two neuron types. The approach demonstrates that the choice of aggregation function constitutes a critical design dimension for enhancing noise robustness. Experiments on CIFAR-10 and its noisy variants show that the triple-hybrid model substantially improves robustness (achieving a score of 0.991 versus the baseline 0.890) while also yielding modest gains on clean data. Learned parameters consistently converge to sublinear aggregation (p ≈ 0.43–0.50) and high utilization of input novelty (α ≈ 0.69–0.79).

This work addresses the limitations of conventional model fusion approaches—such as parameter averaging—which often introduce non-generalizable features when source models differ substantially and lack theoretical grounding or effective metrics for assessing fusion compatibility. The study establishes, for the first time, a theoretical connection between model fusion and ensemble learning, and introduces M-Loss, a layer- or neuron-level inconsistency metric computed using only a small amount of unlabeled data to quantify the discrepancy between parameter averaging and model ensembling. This metric effectively guides the optimization of fusion strategies and the evaluation of parameter importance, thereby enhancing pruning efficiency. Experiments demonstrate that M-Loss significantly improves alignment between fused and ensemble models, enabling efficient and accurate model integration while reducing inference cost and storage overhead.

In federated learning, aggregated local evaluation metrics often diverge from centralized assessment results, leading to misleading performance estimates. This work presents the first systematic analysis of the root causes of this discrepancy and introduces FLAM (Federated Learning Aggregatable Metrics), a general framework for consistent metric aggregation. Through rigorous mathematical derivation, FLAM establishes necessary conditions for metrics to ensure global consistency and devises a distributed evaluation protocol that enables accurate aggregation of diverse performance measures without requiring a global test set. Empirical evaluations across multiple benchmark tasks demonstrate that FLAM precisely reproduces centralized evaluation outcomes, substantially enhancing the reliability and applicability of model assessment in federated settings and overcoming the prevailing limitation of existing approaches to accuracy alone.

This study investigates optimal parallel heterogeneous ensemble strategies for small- to medium-scale tabular classification tasks. Through systematic experiments on 56 OpenML CC18 datasets, the authors find that the instability of Blending and Stacking is mutually independent, and that Robust Soft Voting significantly outperforms Hard Voting in multi-class settings. Building on these insights, they propose a robust ensemble strategy and validate its effectiveness on the TabArena platform. The recommended approach significantly surpasses the Single Best method across 28 additional tasks and matches or exceeds the performance of all individual ensemble techniques considered.

This work investigates whether ensemble methods can enhance the generation quality of unconditional score-based diffusion models, with a particular focus on perceptual metrics such as FID. We systematically evaluate strategies including deep ensembles, Monte Carlo Dropout, and random forests across CIFAR-10, FFHQ, and tabular datasets. Our findings indicate that while ensembling consistently improves score-matching loss and likelihood, it does not necessarily lead to better perceptual quality. We further elucidate the theoretical mechanism underlying score addition, establishing connections to model composition techniques like guidance, and identify an optimal aggregation strategy specifically effective for tabular data.