Designing User-Centric Metrics for Evaluation of Counterfactual Explanations

2025-07-20

AI Summary

Existing counterfactual explanation (CFE) evaluation metrics—such as proximity—are manually designed and fail to capture users’ genuine preferences and constraints, limiting their practical utility. Method: Leveraging large-scale crowdsourcing and behavioral science methodologies, we conducted a two-day scenario-based user study to empirically uncover the cognitive mechanisms underlying human CFE evaluation for the first time. Based on these findings, we propose the Adaptive Weighted Preference (AWP) model—a two-stage, personalized, user-adaptive evaluation framework. Contribution/Results: AWP departs from traditional static metric paradigms by enabling human-centered modeling of the evaluation process. Experiments show that conventional proximity aligns with user choices in only 63.81% of cases, whereas AWP achieves an 84.37% prediction accuracy—significantly enhancing both the validity and interpretability of CFE assessment.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Knowledge Representation and Reasoning: Preferences
• Machine Learning: Learning Preferences or Rankings

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

Machine learning-based decision models are increasingly being used to make decisions that significantly impact people's lives, but their opaque nature leaves end users without a clear understanding of why a decision was made. Counterfactual Explanations (CFEs) have grown in popularity as a means of offering actionable guidance by identifying the minimum changes in feature values required to flip a model's prediction to something more desirable. Unfortunately, most prior research in CFEs relies on artificial evaluation metrics, such as proximity, which may overlook end-user preferences and constraints, e.g., the user's perception of effort needed to make certain feature changes may differ from that of the model designer. To address this research gap, this paper makes three novel contributions. First, we conduct a pilot study with 20 crowd-workers on Amazon MTurk to experimentally validate the alignment of existing CF evaluation metrics with real-world user preferences. Results show that user-preferred CFEs matched those based on proximity in only 63.81% of cases, highlighting the limited applicability of these metrics in real-world settings. Second, inspired by the need to design a user-informed evaluation metric for CFEs, we conduct a more detailed two-day user study with 41 participants facing realistic credit application scenarios to find experimental support for or against three intuitive hypotheses that may explain how end users evaluate CFEs. Third, based on the findings of this second study, we propose the AWP model, a novel user-centric, two-stage model that describes one possible mechanism by which users evaluate and select CFEs. Our results show that AWP predicts user-preferred CFEs with 84.37% accuracy. Our study provides the first human-centered validation for personalized cost models in CFE generation and highlights the need for adaptive, user-centered evaluation metrics.

Problem

• Evaluating alignment of CFE metrics with user preferences
• Designing user-informed metrics for counterfactual explanations
• Proposing AWP model to predict user-preferred CFEs

Innovation

• Conduct user study to validate CF evaluation metrics
• Propose AWP model for user-centric CFE evaluation
• Design adaptive metrics based on user preferences