Show and Tell: Visually Explainable Deep Neural Nets via Spatially-Aware Concept Bottleneck Models

2025-02-27

AI Summary

Deep neural networks achieve strong performance but suffer from poor interpretability, failing to reveal the semantic concepts underlying decisions and their precise spatial locations. To address this, we propose Spatially-Aware Unsupervised Concept Bottleneck Models (SALF-CBMs), the first framework to generate concept heatmaps without human-annotated concept labels, jointly achieving semantic meaningfulness and pixel-level localization. Our method introduces: (i) a spatial-attention-guided concept projection layer; (ii) self-supervised concept disentanglement training; (iii) a concept-map-based classification head optimization; and (iv) heatmap-driven visual explanation generation. On multi-class classification benchmarks, SALF-CBMs significantly outperform existing concept-bottleneck and post-hoc explanation baselines. In zero-shot segmentation, they surpass mainstream gradient-based interpretability methods. Moreover, SALF-CBMs support interactive region querying and local concept editing, enhancing model debuggability and controllability.

Technology Category

Computer Vision: Interpretability, Explainability, and TransparencyMachine Learning: Calibration & Uncertainty QuantificationCognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalizationSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Modern deep neural networks have now reached human-level performance across a variety of tasks. However, unlike humans they lack the ability to explain their decisions by showing where and telling what concepts guided them. In this work, we present a unified framework for transforming any vision neural network into a spatially and conceptually interpretable model. We introduce a spatially-aware concept bottleneck layer that projects"black-box"features of pre-trained backbone models into interpretable concept maps, without requiring human labels. By training a classification layer over this bottleneck, we obtain a self-explaining model that articulates which concepts most influenced its prediction, along with heatmaps that ground them in the input image. Accordingly, we name this method"Spatially-Aware and Label-Free Concept Bottleneck Model"(SALF-CBM). Our results show that the proposed SALF-CBM: (1) Outperforms non-spatial CBM methods, as well as the original backbone, on a variety of classification tasks; (2) Produces high-quality spatial explanations, outperforming widely used heatmap-based methods on a zero-shot segmentation task; (3) Facilitates model exploration and debugging, enabling users to query specific image regions and refine the model's decisions by locally editing its concept maps.

Problem

• Enhances deep neural network interpretability
• Generates spatial and conceptual explanations
• Improves model exploration and debugging capabilities

Innovation

• Spatially-aware concept bottleneck layer
• Label-free interpretable concept maps
• Self-explaining model with heatmaps