In Defense of Information Leakage in Concept-based Models

2026-06-09

AI Summary

Conventional wisdom holds that information leakage in concept-based models undermines interpretability; however, this assumption lacks empirical validation and may unnecessarily constrain model performance in real-world settings where concepts are inherently incomplete. Challenging this prevailing view, this work introduces the novel notion of “benign leakage,” arguing that under specific conditions, such leakage can be harnessed to enhance model efficacy. To this end, we propose a new method integrating a reconstruction-based training objective, intervenability constraints, and representation learning, which deliberately accommodates benign leakage while preserving both predictive accuracy and concept-level intervenability. Empirical results demonstrate that our approach significantly outperforms existing methods that forcibly eliminate leakage, offering a new paradigm for building concept-based models that are simultaneously effective, practical, and interpretable.

Abstract

Concept-based models (CMs), deep neural networks that ground their predictions on representations aligned with human-understandable concepts (e.g., "round", "stripes", etc.), have been shown to learn representations that leak concept-irrelevant information. As the traditional narrative goes, this leakage is undesirable and should be eradicated as it leads to uninterpretable models. In this paper, we posit that this conventional view of leakage in CMs is not only ill-posed, as the evidence of how leakage makes a model less interpretable is often inconclusive, but also bound to lead to impractical CMs under common real-world constraints. Specifically, we argue that in real-world settings where concept incompleteness is the norm, some leakage is often necessary for constructing accurate and intervenable CMs. To this end, we propose that there is such a thing as benign leakage and show that, by optimizing a reframing of the typical CM training objective, CMs can encourage and exploit this form of leakage without sacrificing accuracy or intervenability.

Problem

• information leakage
• concept-based models
• interpretability
• concept incompleteness
• benign leakage

Innovation

• concept-based models
• information leakage
• benign leakage
• intervenability
• concept incompleteness