Understanding the Effects of AI-Assisted Critical Thinking on Human-AI Decision Making

2026-02-10

AI Summary

This study addresses the suboptimal performance of humans in human-AI collaborative decision-making, often stemming from insufficient reflection on their own reasoning. To mitigate this, the authors propose the AI-Augmented Critical Thinking (AACT) framework, which introduces domain-specific counterfactual analysis into human-AI interaction for the first time. By generating counterfactual explanations that challenge users’ stated rationales for their decisions, AACT prompts individuals to actively re-examine and refine their judgments. Empirical validation in a house price prediction task demonstrates that AACT significantly reduces overreliance on AI recommendations. Although it imposes a modest increase in cognitive load, the framework enhances overall decision quality, with particularly pronounced benefits for users already familiar with AI systems. This work establishes a novel paradigm for fostering critical thinking in human-AI collaboration.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Reasoning under Uncertainty: Decision/Utility Theory
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Despite the growing prevalence of human-AI decision making, the human-AI team's decision performance often remains suboptimal, partially due to insufficient examination of humans'own reasoning. In this paper, we explore designing AI systems that directly analyze humans'decision rationales and encourage critical reflection of their own decisions. We introduce the AI-Assisted Critical Thinking (AACT) framework, which leverages a domain-specific AI model's counterfactual analysis of human decision to help decision-makers identify potential flaws in their decision argument and support the correction of them. Through a case study on house price prediction, we find that AACT outperforms traditional AI-based decision-support in reducing over-reliance on AI, though also triggering higher cognitive load. Subgroup analysis reveals AACT can be particularly beneficial for some decision-makers such as those very familiar with AI technologies. We conclude by discussing the practical implications of our findings, use cases and design choices of AACT, and considerations for using AI to facilitate critical thinking.

Problem

• human-AI decision making
• critical thinking
• decision rationale
• over-reliance on AI
• cognitive load

Innovation

• AI-Assisted Critical Thinking
• counterfactual analysis
• human-AI decision making
• decision rationale
• cognitive load