Operational Abstractions of Neural Network Concepts via Topological Representations

2026-10-04

AI Summary

This study addresses the lack of a unified editable representation in existing concept editing methods by proposing a topological concept representation framework. The approach leverages topological encoding to construct an intermediate concept space, thereby decoupling concept organization from intervention mechanisms and enabling their joint optimization. Furthermore, it introduces the first topology-based paradigm for universal concept abstraction, theoretically establishing its stability and reparameterization invariance. By integrating techniques such as topological data analysis, concept recoverability evaluation, and posterior operation abstraction, the framework achieves substantial empirical improvements. Experimental results demonstrate that the proposed method increases worst-group accuracy by an average of 21.89% and enhances concept recoverability in knowledge distillation by 5.54%.

Abstract

Concept-based methods provide a semantic level for interpreting and manipulating learned representations, but existing editing approaches are typically specialized to particular interventions and do not provide a common and editable representation of concept organization. To achieve this, we introduce Topological Concept Representations (TCR), a post-hoc operational abstraction that jointly characterizes the concepts encoded in a learned representation and their relationships. TCR constructs an intermediate concept space from concept recoverability and interaction scores, and compactly encodes its organization through topology. Interventions are expressed through modifications of this abstraction that are propagated back to the underlying learned representation. This allows different concept-level operations to share the same optimization framework and separates the desired concept organization from the mechanism to achieve it. We establish stability and reparameterization-invariance properties of TCR and its connections to existing concept-editing formulations. We use TCR to disentangle concepts as a preprocessing step for existing erasure methods, improving worst-group accuracy by 21.89 on average at comparable concept leakage. We further use TCR to transfer concepts from teacher to student models, improving concept recoverability by up to 5.54 while also improving or maintaining competitive test top-1 accuracy.

Problem

• concept-based interpretability
• neural network editing
• concept representation
• topological abstraction

Innovation

• Topological Concept Representations
• Concept Editing
• Operational Abstraction
• Concept Disentanglement
• Concept Transfer