Now you see me! A framework for obtaining class-relevant saliency maps

2025-03-10

AI Summary

Existing saliency map methods often produce explanations with high generalizability but low discriminability, failing to precisely localize class-specific decision evidence. To address this, we propose the Cross-Class Attribution Fusion (CCAF) framework—the first to explicitly integrate inter-class attributions by decoupling shared features from discriminative ones. CCAF enhances attribution specificity in a model- and method-agnostic manner via plug-and-play aggregation of gradient- and perturbation-based attributions, guided by counterfactual contrastive masking. The framework comprises three stages: attribution aggregation, class-contrastive masking, and randomized robustness validation. Evaluated on grid-pointing localization and randomized sanity checks, CCAF significantly improves discriminative accuracy across mainstream attribution methods. It reliably identifies both class-discriminative and class-shared visual evidence on multiple benchmarks, advancing the fidelity and interpretability of post-hoc explanations.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Causality

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Neural networks are part of daily-life decision-making, including in high-stakes settings where understanding and transparency are key. Saliency maps have been developed to gain understanding into which input features neural networks use for a specific prediction. Although widely employed, these methods often result in overly general saliency maps that fail to identify the specific information that triggered the classification. In this work, we suggest a framework that allows to incorporate attributions across classes to arrive at saliency maps that actually capture the class-relevant information. On established benchmarks for attribution methods, including the grid-pointing game and randomization-based sanity checks, we show that our framework heavily boosts the performance of standard saliency map approaches. It is, by design, agnostic to model architectures and attribution methods and now allows to identify the distinguishing and shared features used for a model prediction.

Problem

• Improving specificity of saliency maps for neural networks
• Identifying class-relevant features in model predictions
• Enhancing transparency in high-stakes decision-making contexts

Innovation

• Framework for class-relevant saliency maps
• Incorporates attributions across multiple classes
• Agnostic to model architectures and methods