Explaining the Unexplained: Revealing Hidden Correlations for Better Interpretability

2024-12-02

arXiv.org

AI Summary

Deep learning excels on unstructured data tasks, yet its “black-box” nature hinders deployment in safety- or fairness-critical applications. Existing interpretability methods often neglect feature interdependencies and lack systematic evaluation of model decision paths. To address these limitations, we propose RealExp: the first correlation-aware Shapley value disentanglement framework that decomposes feature importance into individual contributions and pairwise (or higher-order) correlation contributions. RealExp integrates feature similarity metrics with explicit decision path modeling. Furthermore, we introduce a novel path-oriented interpretability evaluation criterion—moving beyond conventional fidelity-based metrics—to holistically assess explanation quality. Extensive experiments on image classification and text sentiment analysis demonstrate that RealExp significantly outperforms state-of-the-art baselines. It enables principled selection of interpretable pre-trained models and empirically validates the feasibility of approximating fine-tuned GPT-Ada performance using a lightweight Bag-of-Words model.

Technology Category

Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP ModelsMachine Learning: Transparent, Interpretable, Explainable MLComputer Vision: Interpretability, Explainability, and Transparency

Application Category

User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalizationSearch and Retrieval-Augmented AI: Web evaluation methodologies and metricsWeb Mining and Content Analysis: Large pretrained models with web data

Abstract

Deep learning has achieved remarkable success in processing and managing unstructured data. However, its"black box"nature imposes significant limitations, particularly in sensitive application domains. While existing interpretable machine learning methods address some of these issues, they often fail to adequately consider feature correlations and provide insufficient evaluation of model decision paths. To overcome these challenges, this paper introduces Real Explainer (RealExp), an interpretability computation method that decouples the Shapley Value into individual feature importance and feature correlation importance. By incorporating feature similarity computations, RealExp enhances interpretability by precisely quantifying both individual feature contributions and their interactions, leading to more reliable and nuanced explanations. Additionally, this paper proposes a novel interpretability evaluation criterion focused on elucidating the decision paths of deep learning models, going beyond traditional accuracy-based metrics. Experimental validations on two unstructured data tasks -- image classification and text sentiment analysis -- demonstrate that RealExp significantly outperforms existing methods in interpretability. Case studies further illustrate its practical value: in image classification, RealExp aids in selecting suitable pre-trained models for specific tasks from an interpretability perspective; in text classification, it enables the optimization of models and approximates the performance of a fine-tuned GPT-Ada model using traditional bag-of-words approaches.

Problem

• Enhances interpretability in deep learning models
• Decouples Shapley Value for feature analysis
• Introduces new interpretability evaluation criterion

Innovation

• Decouples Shapley Value importance
• Incorporates feature similarity computations
• Proposes novel interpretability evaluation criterion