Novice Reliance Calibration in AI-Assisted Decision Making: The Role of Explanations and Self-Assessment

2026-10-06

AI Summary

This study addresses the challenge that novice users struggle to calibrate their reliance on AI systems in scenarios lacking immediate feedback, thereby increasing the risk of over-reliance. To investigate this, we extend the concept of reliance calibration to human-AI collaboration contexts without real-time signals. Through a between-subjects experiment involving a clinical entity extraction task, we examine how AI-generated explanations and metacognitive self-assessment influence novices’ reliance calibration. Our findings reveal a counterintuitive effect: providing explanations can paradoxically induce over-reliance, whereas a deeper understanding of the task facilitates selective reliance. Based on these insights, we propose system design guidelines tailored for environments without immediate feedback, offering empirical evidence to optimize human-AI collaborative tools.

Abstract

Artificial Intelligence (AI) tools are widely used to support decision making in tasks and domains where no immediate performance feedback is available. In these settings, users cannot learn to adjust their reliance behavior over time through trial and error. However, little is known about how novice users calibrate reliance on AI when external feedback is unavailable, or whether AI explanations can support calibration in its absence. We introduce reliance calibration as an organizing construct for studying how novice users dynamically adjust reliance behavior, and examine how AI explanations and meta-cognitive self-assessment shape it. Through a between-subjects study with 110 participants completing a clinical entity extraction task with AI assistance and limited performance feedback, we observe that novice users exhibit systematic drift toward over-reliance in the presence of explanations, while higher self-reported task understanding is associated with more selective reliance behavior. These results extend reliance calibration research into human-AI collaboration contexts without real-time performance signals and present actionable guidelines on designing AI tools that must support appropriate reliance in these settings.

Problem

• reliance calibration
• AI-assisted decision making
• novice users
• explanations
• self-assessment

Innovation

• Reliance Calibration
• AI-Assisted Decision Making
• Explainable AI
• Over-reliance
• Meta-cognitive Self-assessment