decision fusion

This study addresses the arbitrariness in recidivism risk prediction arising from model multiplicity by leveraging a judicial system with over 15 years of operational history. The authors formalize legal rules into algorithmic labels to construct a high-quality dataset, train interpretable models, and analyze how structural diversity among models influences predictive disagreement. For the first time in a real-world judicial setting, they quantify the relationship between model multiplicity and prediction arbitrariness, establish a theoretical lower bound, and demonstrate that actual inter-model consistency substantially exceeds worst-case expectations. Innovatively adopting a “minimum risk score across multiple models” strategy, the approach simultaneously safeguards individual rights and reduces decision arbitrariness. The resulting models not only achieve superior predictive performance and more equitable error distributions across demographic groups but also effectively capture inmates’ rehabilitation progress.

This study addresses the challenge posed by predictive multiplicity in high-risk AI systems, which often leads to inconsistent individual-level predictions that fail to meet the accuracy requirements mandated by the EU AI Act. To bridge this gap, the work integrates predictive multiplicity into the Act’s compliance framework for the first time, introducing two novel metrics—Individual Conflict Ratio and δ-ambiguity—to quantify the degree of disagreement among alternative models yielding similar aggregate performance. Building on these metrics, the authors develop a comprehensive methodology that combines ensemble model evaluation with multiplicity analysis. Furthermore, they provide actionable guidelines for model providers to assess and disclose the reliability of individual predictions, thereby enhancing transparency and facilitating compliance with regulatory expectations regarding individual-level accuracy.

When machine learning models must simultaneously withstand security, privacy, and fairness threats, existing defense combinations often suffer from methodological conflicts that degrade efficacy and incur prohibitive empirical validation costs. To address this, we propose DefCon—the first framework for assessing the effectiveness of defense combinations while satisfying four critical desiderata: accuracy, scalability, non-intrusiveness, and generality. DefCon leverages defense interaction modeling and lightweight meta-feature evaluation, enabling principle-driven compositional analysis without modifying original defense modules. It achieves 90% accuracy on eight established combinations and 81% on thirty novel ones, substantially reducing experimental overhead. Its core innovation lies in transcending heuristic, experience-based composition paradigms by establishing an interpretable, reusable theoretical criterion for combination validity.

To address the limitations of the conventional Consistency Ratio (CR) in the Analytic Hierarchy Process (AHP)—including low discriminative accuracy, rigid thresholding, sensitivity to matrix order, and poor interpretability—this paper proposes a novel consistency classification method based on triplet Preference Reversal (PR). The method explicitly models local–global inconsistency by detecting directional conflicts between local 3×3 submatrices and the global priority vector, yielding an interpretable consistency metric centered on the preference reversal ratio and maximum reversal magnitude. Implemented within the R package *AHPtools*, the approach integrates k-means clustering and logistic regression for robust classification. Experimental evaluation on synthetic pairwise comparison matrices demonstrates a classification accuracy of 97%, substantially outperforming the CR-based method (50%); the false rejection rate is reduced to 2.6%, significantly enhancing the robustness and credibility of AHP-based decision-making.

Model merging often suffers from unpredictable performance, limiting its practical utility. This work introduces the first quantifiable definition of model mergeability and systematically investigates the key factors influencing merging effectiveness, identifying the base model’s prior knowledge about the fine-tuning data as the decisive factor. Building on this insight, the authors propose a weighted parameter fusion strategy that effectively preserves weak yet relevant knowledge embedded in the base model. Experimental results demonstrate that the proposed method significantly enhances merging performance in multi-task settings, thereby validating the critical role of the base model’s knowledge level in determining the success of model merging.

This work addresses how interpretive disagreements may arise in joint decision-making among agents who share identical environments and signal likelihoods but hold divergent subjective models. The authors propose a decision-theoretic measure of disagreement by comparing the extent to which different subjective models support signal-contingent plans. They construct a prior-independent consistency ordering and show that its unique rotation-invariant scalar completion is the cosine similarity. This ordering is independent of Blackwell dominance and favors quadratic over KL-type Bregman divergences. Greater interpretive consistency narrows speculative trading wedges, expands the ex ante Pareto frontier, and enlarges the set of strategies rationalizable under a single model.

This study addresses the ambiguity and inconsistency in evaluation criteria for software engineering replication studies, which have led to contradictory interpretations and uncertainty in reported results. Through a systematic review of ten replication studies published between 2021 and 2025, combined with qualitative content analysis, statistical principles, and modeling of measurement uncertainty, this work is the first to uncover the heterogeneity and lack of standardized practices in current evaluation approaches. Building on these insights, the paper proposes a unified evaluation framework that integrates statistical theory, methodological rigor, and measurement theory. Empirical illustration demonstrates that the framework effectively enhances the transparency, consistency, comparability, and reliability of replication studies in software engineering.

This study addresses the "cherry-picked coverage omission" (CCO) problem in large language model (LLM) judges, wherein models erroneously output directional verdicts (e.g., SUPPORTS/REFUTES) instead of the appropriate CONFLICTING label when presented with mixed evidence, leading to unsafe commitments. The work formally characterizes CCO for the first time and introduces a task-contract-based diagnostic protocol. To mitigate this issue, the authors propose a dual-channel reference probe coupled with an external commitment control layer that orthogonally leverages evidence structure and model confidence, along with a NO-COMMIT routing mechanism to enhance safety. Experiments reveal that over 84% of three-way LLM judges exhibit CCO on the AVeriTeC conflict subset, and majority voting exacerbates the problem. In contrast, the dual-channel probe significantly improves CONFLICTING recognition (p < 1/2001) and demonstrates consistent effectiveness on VitaminC-Mixed.

This study challenges the prevailing assumption in AI evaluation that consistency implies correctness, systematically investigating whether self-consistency and cross-model agreement reliably indicate answer accuracy in large language models. Leveraging the GPQA Diamond and AIME datasets, the authors analyze 265,000 samples across 53 model instances using hierarchical clustering with bootstrapping, majority-correct labeling, cross-model-family validation, and margin-preserving null hypothesis testing. They find that consistency serves only as a weak positive predictor of correctness (ρ = 0.20–0.59), with its efficacy strongly dependent on model capability tier and task saturation: it is most informative for mid-tier, unsaturated models but can induce overconfidence in state-of-the-art models, yielding high agreement alongside high error rates. All data are publicly released.

This study addresses the reliability and validity of LLM-as-judge evaluations, which are susceptible to shifts in the judge model’s version even when candidate responses remain unchanged. The authors conduct a systematic audit of dense Qwen3 models (1.7B–32B) and MiniMax API iterations (M2 to M2.7) across four benchmark judgment datasets. They propose a multidimensional auditing framework incorporating multiscale judge comparisons, repeated-sampling juries, structured debate protocols, and probes for position and verbosity biases. Findings indicate that only the upgrade from Qwen3-1.7B to -4B yields consistent performance gains; stronger judges mitigate but do not eliminate systematic biases; and structured debate substantially alters verdicts, though reliable attribution requires access to detailed interaction logs.