FastCAV: Efficient Computation of Concept Activation Vectors for Explaining Deep Neural Networks

2025-05-23

AI Summary

Existing Concept Activation Vector (CAV) methods incur prohibitive computational overhead in large-scale, high-dimensional models, hindering real-time, scalable concept-level interpretation. This paper proposes an efficient CAV extraction framework that replaces traditional SVM-based optimization with linear-algebraic computation and statistical approximation, augmented by feature-space projection and concept orthogonality constraints. We provide theoretical guarantees showing equivalence to the SVM baseline under mild assumptions. Experiments demonstrate an average speedup of 46.4× (up to 63.6×), substantially reducing computational cost while preserving interpretive consistency and stability. The method achieves performance parity with standard CAVs in downstream tasks such as TCAV and uniquely enables dynamic tracking of concept evolution during model training. To our knowledge, this is the first concept-level diagnostic framework for large-scale deep models that simultaneously ensures efficiency, fidelity, and evolvability.

Technology Category

• Computer Vision: Large Vision Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Causal Learning

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Concepts such as objects, patterns, and shapes are how humans understand the world. Building on this intuition, concept-based explainability methods aim to study representations learned by deep neural networks in relation to human-understandable concepts. Here, Concept Activation Vectors (CAVs) are an important tool and can identify whether a model learned a concept or not. However, the computational cost and time requirements of existing CAV computation pose a significant challenge, particularly in large-scale, high-dimensional architectures. To address this limitation, we introduce FastCAV, a novel approach that accelerates the extraction of CAVs by up to 63.6x (on average 46.4x). We provide a theoretical foundation for our approach and give concrete assumptions under which it is equivalent to established SVM-based methods. Our empirical results demonstrate that CAVs calculated with FastCAV maintain similar performance while being more efficient and stable. In downstream applications, i.e., concept-based explanation methods, we show that FastCAV can act as a replacement leading to equivalent insights. Hence, our approach enables previously infeasible investigations of deep models, which we demonstrate by tracking the evolution of concepts during model training.

Problem

• Efficient computation of Concept Activation Vectors (CAVs) for deep neural networks
• Reducing computational cost and time in large-scale, high-dimensional architectures
• Maintaining performance and stability in concept-based explanation methods

Innovation

• FastCAV accelerates CAV extraction by 46.4x on average
• Theoretical foundation ensures equivalence to SVM-based methods
• Maintains performance while improving efficiency and stability