Narrowing Action Choices with AI Improves Human Sequential Decisions

2025-10-17

AI Summary

This study addresses insufficient human-AI complementarity in sequential decision-making tasks by proposing a novel decision-support paradigm that dynamically modulates human agency. Methodologically, it leverages a pre-trained AI to generate candidate action subsets and employs an adaptive multi-armed bandit algorithm—guided by action-set smoothness features—to optimize AI intervention intensity in real time. Crucially, it enables humans to focus on refined, high-salience action sets without requiring comprehension of the AI’s internal logic. The core contribution is the formal modeling of “human agency” as a tunable parameter, coupled with a feedback-driven online learning mechanism for continuous co-adaptation of human-AI collaboration policies. A large-scale human-subject experiment (n = 1,600) demonstrates that the system improves human decision accuracy by ~30% over unassisted decisions and surpasses AI-only performance by 2%, providing the first empirical validation of significant performance gains from dynamic agency modulation in human-AI teamwork.

Technology Category

• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Multiagent Systems: Teamwork
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Recent work has shown that, in classification tasks, it is possible to design decision support systems that do not require human experts to understand when to cede agency to a classifier or when to exercise their own agency to achieve complementarity$unicode{x2014}$experts using these systems make more accurate predictions than those made by the experts or the classifier alone. The key principle underpinning these systems reduces to adaptively controlling the level of human agency, by design. Can we use the same principle to achieve complementarity in sequential decision making tasks? In this paper, we answer this question affirmatively. We develop a decision support system that uses a pre-trained AI agent to narrow down the set of actions a human can take to a subset, and then asks the human to take an action from this action set. Along the way, we also introduce a bandit algorithm that leverages the smoothness properties of the action sets provided by our system to efficiently optimize the level of human agency. To evaluate our decision support system, we conduct a large-scale human subject study ($n = 1{,}600$) where participants play a wildfire mitigation game. We find that participants who play the game supported by our system outperform those who play on their own by $sim$$30$% and the AI agent used by our system by $>$$2$%, even though the AI agent largely outperforms participants playing without support. We have made available the data gathered in our human subject study as well as an open source implementation of our system at https://github.com/Networks-Learning/narrowing-action-choices .

Problem

• Achieving complementarity in sequential decision making tasks
• Narrowing human action choices using pre-trained AI agents
• Optimizing human agency levels via bandit algorithms

Innovation

• AI narrows human action choices for decisions
• Bandit algorithm optimizes human agency level
• Pre-trained agent filters actions to improve performance