Technology Category
Machine Learning: Representation LearningCognitive Modeling & Cognitive Systems: Symbolic RepresentationsComputer Vision: Representation Learning for Vision
Measuring the Measures: Discriminative Capacity of Representational Similarity Metrics Across Model Families
🤖 AI Summary
Existing studies lack a quantitative, discriminability-oriented comparison of representational similarity measures (RSMs) across diverse model families (e.g., CNNs, Transformers, biologically inspired models).
Method: We propose a unified discriminability evaluation framework grounded in signal detection theory (d′), silhouette coefficient, and ROC-AUC, integrating major RSMs—including Representational Similarity Analysis (RSA), linear predictivity, Procrustes alignment, and soft matching.
Contribution/Results: Our analysis reveals, for the first time, a positive correlation between an RSM’s alignment constraint strength and its ability to separate models by architecture or training paradigm: soft matching achieves top discriminability, while non-fitting methods (e.g., RSA) also exhibit strong separation performance. The framework provides interpretable, reproducible criteria for selecting RSMs in cross-model comparisons and model–brain alignment studies.
Technology Category
Application Category
📝 Abstract
Representational similarity metrics are fundamental tools in neuroscience and AI, yet we lack systematic comparisons of their discriminative power across model families. We introduce a quantitative framework to evaluate representational similarity measures based on their ability to separate model families-across architectures (CNNs, Vision Transformers, Swin Transformers, ConvNeXt) and training regimes (supervised vs. self-supervised). Using three complementary separability measures-dprime from signal detection theory, silhouette coefficients and ROC-AUC, we systematically assess the discriminative capacity of commonly used metrics including RSA, linear predictivity, Procrustes, and soft matching. We show that separability systematically increases as metrics impose more stringent alignment constraints. Among mapping-based approaches, soft-matching achieves the highest separability, followed by Procrustes alignment and linear predictivity. Non-fitting methods such as RSA also yield strong separability across families. These results provide the first systematic comparison of similarity metrics through a separability lens, clarifying their relative sensitivity and guiding metric choice for large-scale model and brain comparisons.
Problem
Research questions and friction points this paper is trying to address.
Evaluating discriminative power of representational similarity metrics across model families
Systematically comparing metric performance across architectures and training regimes
Assessing separability capacity of various similarity measures using quantitative framework
Innovation
Methods, ideas, or system contributions that make the work stand out.
Quantitative framework for evaluating similarity metrics
Assessing discriminative capacity with three separability measures
Systematic comparison of metrics across model families
🔎 Similar Papers
No similar papers found.
