Towards Locally Explaining Prediction Behavior via Gradual Interventions and Measuring Property Gradients

2025-03-07

AI Summary

Deep learning models exhibit strong predictive performance but lack local interpretability; existing methods either yield correlation-based attributions or provide only global causal explanations inapplicable to individual instances. This paper introduces the first local interventional explanation framework, which identifies causal factors driving predictions for specific inputs via progressive semantic intervention and attribute-wise gradient quantification. We innovatively define the “expected attribute gradient norm” as a formal interpretability metric, enabling a principled transition from pixel-level attribution to fine-grained, semantically grounded causal analysis. Our approach integrates image editing, controllable semantic intervention, gradient sensitivity analysis, and empirical causal validation. Extensive evaluation across synthetic datasets, training dynamics, skin lesion classification, and CLIP models demonstrates its effectiveness: it successfully pinpoints localized biases and uncovers unintended semantic dependencies encoded by models.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Causal Learning
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Deep learning models achieve high predictive performance but lack intrinsic interpretability, hindering our understanding of the learned prediction behavior. Existing local explainability methods focus on associations, neglecting the causal drivers of model predictions. Other approaches adopt a causal perspective but primarily provide more general global explanations. However, for specific inputs, it's unclear whether globally identified factors apply locally. To address this limitation, we introduce a novel framework for local interventional explanations by leveraging recent advances in image-to-image editing models. Our approach performs gradual interventions on semantic properties to quantify the corresponding impact on a model's predictions using a novel score, the expected property gradient magnitude. We demonstrate the effectiveness of our approach through an extensive empirical evaluation on a wide range of architectures and tasks. First, we validate it in a synthetic scenario and demonstrate its ability to locally identify biases. Afterward, we apply our approach to analyze network training dynamics, investigate medical skin lesion classifiers, and study a pre-trained CLIP model with real-life interventional data. Our results highlight the potential of interventional explanations on the property level to reveal new insights into the behavior of deep models.

Problem

• Lack of interpretability in deep learning models
• Need for local causal explanations for specific inputs
• Quantifying impact of semantic properties on predictions

Innovation

• Leverages image-to-image editing models
• Uses gradual interventions on semantic properties
• Introduces expected property gradient magnitude