🤖 AI Summary
This paper addresses the challenge of modeling capability disparities among heterogeneous decision-makers—human experts and AI models—in collaborative decision-making. We propose the first unified capability modeling framework: (1) representing human and AI decision capabilities via learnable capability vectors, (2) dynamically assigning context-aware weighted fusion coefficients for decision integration, and (3) introducing a learning-free global collaboration baseline as a zero-shot reference. Our method comprises capability embedding, context-sensitive weighted fusion, and multi-source decision ensemble. Evaluated on image classification and hate speech detection, our approach significantly outperforms state-of-the-art methods—particularly when non-expert participants exhibit relatively strong capabilities—yielding substantial gains in both accuracy and robustness. Empirical results validate that explicit capability modeling markedly enhances human-AI collaborative decision-making performance.
📝 Abstract
Human-AI collaborative decision making has emerged as a pivotal field in recent years. Existing methods treat human and AI as different entities when designing human-AI systems. However, as the decision capabilities of AI models become closer to human beings, it is necessary to build a uniform framework for capability modeling and integrating. In this study, we propose a general architecture for human-AI collaborative decision making, wherein we employ learnable capability vectors to represent the decision-making capabilities of both human experts and AI models. These capability vectors are utilized to determine the decision weights of multiple decision makers, taking into account the contextual information of each decision task. Our proposed architecture accommodates scenarios involving multiple human-AI decision makers with varying capabilities. Furthermore, we introduce a learning-free approach to establish a baseline using global collaborative weights. Experiments on image classification and hate speech detection demonstrate that our proposed architecture significantly outperforms the current state-of-the-art methods in image classification and sentiment analysis, especially for the case with large non-expertise capability levels. Overall, our method provides an effective and robust collaborative decision-making approach that integrates diverse human/AI capabilities within a unified framework.