Careful Judge: Safe and Efficient Human-AI Collaborative Decision Making

2026-10-06

AI Summary

This study addresses the inherent tension between AI decision safety and human review costs in human-AI collaboration, where adaptive learning often compromises safety calibration. To resolve this, we propose CARE (Calibrated Adaptive Revision and Escalation), an end-to-end framework featuring a novel adaptive calibration module that ensures risk controllability across arbitrary revision stages. By integrating a selective human feedback mechanism for continual learning, its modular architecture remains compatible with any black-box model. Evaluated on mission-critical datasets spanning autonomous driving, natural language processing, and robotics, CARE reduces human query volume by 25%–81% compared to baseline methods while preserving human-aligned safe decision-making, effectively reconciling safety guarantees with automation efficiency.

Abstract

In human-AI collaborative decision making, human review can prevent unsafe AI decisions, but each human judgment is costly. Treating human intervention after AI abstention as a one-off fallback misses the opportunity to improve future AI decisions for greater automation, yet AI adaptively learning from selectively queried human feedback breaks safety guardrails calibrated for old models. We approach this challenge with CARE---calibrated adaptive rectification and escalation---an end-to-end pipeline that combines AI models and human reviewers to guarantee safe, human-aligned decisions, while continuously learning from human feedback to achieve greater automation with fewer human queries. CARE is principled, general, modular, and works with any black-box AI model. Our novel adaptive calibration module guarantees risk control at every time step for any rectification module. We further show how CARE improves query efficiency when the AI model is well trained and the human-AI misalignment has a clear structure. Experiments on four safety-critical real-world datasets spanning driving, language, and robotics demonstrate that CARE achieves human-aligned decisions while reducing human queries by 25-81% relative to baselines.

Problem

• Human-AI collaboration
• Safe decision making
• Adaptive learning
• Risk control
• Query efficiency

Innovation

• Human-AI collaboration
• Adaptive calibration
• Risk control
• Query efficiency
• Continuous learning