🤖 AI Summary
This work addresses the problem of interpretable concept probing—i.e., detecting human-defined semantic concepts—in neural network representations. Conventional approaches rely on heuristic, manual layer selection, resulting in unstable and poorly generalizable interpretations. To overcome this limitation, we propose an automatic layer selection method grounded in two complementary representational properties: *informativeness* (quantifying a layer’s discriminative power for the target concept) and *regularity* (measuring structural consistency of concept-related representations). We formulate a joint evaluation framework that jointly optimizes these metrics to identify the optimal probing layer. Extensive experiments across diverse architectures (e.g., ResNet, ViT) and benchmarks (ImageNet, CUB) demonstrate that our method significantly improves probing accuracy, cross-model robustness, interpretability, and reproducibility. By replacing ad hoc layer selection with a principled, representation-aware criterion, this work establishes a new paradigm for trustworthy AI layer localization.
📝 Abstract
Concept probing has recently gained popularity as a way for humans to peek into what is encoded within artificial neural networks. In concept probing, additional classifiers are trained to map the internal representations of a model into human-defined concepts of interest. However, the performance of these probes is highly dependent on the internal representations they probe from, making identifying the appropriate layer to probe an essential task. In this paper, we propose a method to automatically identify which layer's representations in a neural network model should be considered when probing for a given human-defined concept of interest, based on how informative and regular the representations are with respect to the concept. We validate our findings through an exhaustive empirical analysis over different neural network models and datasets.