🤖 AI Summary
This paper systematically investigates the quantification of neural network model similarity, addressing both representational similarity (intermediate-layer activations) and functional similarity (output behavior). It unifies and comparatively analyzes mainstream metrics—such as CKA, SVCCA, PWCCA, linear probes, and top-k output agreement—across these two complementary paradigms. A structured taxonomy is introduced, accompanied by theoretical analysis of each metric’s mathematical properties, interrelationships, and applicability boundaries. Empirical evaluation assesses their explanatory power and limitations in downstream tasks including model compression, ensemble learning, and robustness analysis. The core contribution is a cross-paradigm benchmark enabling rigorous metric comparison, revealing how metric selection critically influences downstream conclusions. The work further identifies open challenges and proposes principled evaluation criteria, thereby advancing methodological foundations for model behavior interpretation and trustworthy AI. (149 words)
📝 Abstract
Measuring similarity of neural networks to understand and improve their behavior has become an issue of great importance and research interest. In this survey, we provide a comprehensive overview of two complementary perspectives of measuring neural network similarity: (i) representational similarity, which considers how
activations
of intermediate layers differ, and (ii) functional similarity, which considers how models differ in their
outputs
. In addition to providing detailed descriptions of existing measures, we summarize and discuss results on the properties of and relationships between these measures, and point to open research problems. We hope our work lays a foundation for more systematic research on the properties and applicability of similarity measures for neural network models.
