🤖 AI Summary
Weak interpretability of large language models (LLMs) and the reliance of existing concept bottleneck models (CBMs) on manually annotated concepts and architectural modifications pose significant challenges. This paper proposes a learnable, plug-and-play Concept Layer that establishes a lightweight, differentiable mapping between internal LLM representations and an interpretable concept space—without altering the original model architecture. The layer supports both task-directed and unsupervised concept discovery and enables dynamic, inference-time intervention. Key contributions include: (i) the first zero-annotation concept set and seamless integration; (ii) ontology-guided automatic concept retrieval coupled with runtime regulation (e.g., bias mitigation); and (iii) validation of intervention efficacy via concept projection-reconstruction and interactive visualization. Experiments demonstrate that the method preserves original model performance and output consistency across multiple tasks while simultaneously enhancing interpretability, controllability, and practical utility.
📝 Abstract
The opaque nature of Large Language Models (LLMs) has led to significant research efforts aimed at enhancing their interpretability, primarily through post-hoc methods. More recent in-hoc approaches, such as Concept Bottleneck Models (CBMs), offer both interpretability and intervenability by incorporating explicit concept representations. However, these methods suffer from key limitations, including reliance on labeled concept datasets and significant architectural modifications that challenges re-integration into existing system pipelines. In this work, we introduce a new methodology for incorporating interpretability and intervenability into an existing model by integrating Concept Layers (CLs) into its architecture. Our approach projects the model's internal vector representations into a conceptual, explainable vector space before reconstructing and feeding them back into the model. Furthermore, we eliminate the need for a human-selected concept set by algorithmically searching an ontology for a set of concepts that can be either task-specific or task-agnostic. We evaluate CLs across multiple tasks, demonstrating that they maintain the original model's performance and agreement while enabling meaningful interventions. Additionally, we present a proof of concept showcasing an intervenability interface, allowing users to adjust model behavior dynamically, such as mitigating biases during inference.