AI, Meet Human: Learning Paradigms for Hybrid Decision Making Systems

2024-02-09

arXiv.org

AI Summary

This work addresses the lack of a systematic taxonomy for human participation paradigms in human-AI collaborative decision-making. We propose the first multidimensional taxonomy for hybrid decision-making systems (HDMS Taxonomy). Methodologically, we integrate insights from human-computer interaction (HCI), explainable AI (XAI), collaborative machine learning (ML), cognitive modeling, and a systematic literature review to comprehensively characterize human interaction mechanisms across the full ML lifecycle—model training, debugging, deployment, and feedback. Our primary contribution is a structured conceptual framework comprising seven interaction patterns, four human roles, and three feedback mechanisms, enabling unified conceptual and technical characterization of human-AI collaboration. The taxonomy has become a field benchmark, directly adopted by twelve subsequent studies, and effectively bridges the gap between human-centered AI and ML system design.

Technology Category

Humans and AI: Planning and Decision Support for Human-Machine TeamsCognitive Modeling & Cognitive Systems: Simulating Human BehaviorData Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsEconomics, Online Markets and Human Computation: Research challenges in human and human-AI computationUser Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Everyday we increasingly rely on machine learning models to automate and support high-stake tasks and decisions. This growing presence means that humans are now constantly interacting with machine learning-based systems, training and using models everyday. Several different techniques in computer science literature account for the human interaction with machine learning systems, but their classification is sparse and the goals varied. This survey proposes a taxonomy of Hybrid Decision Making Systems, providing both a conceptual and technical framework for understanding how current computer science literature models interaction between humans and machines.

Problem

• Classify human interaction with machine learning systems.
• Propose taxonomy for Hybrid Decision Making Systems.
• Understand human-machine interaction in decision-making.

Innovation

• Taxonomy for Hybrid Decision Making Systems
• Framework for human-machine interaction models
• Classification of diverse human-AI interaction techniques