🤖 AI Summary
Existing activation alignment methods struggle to capture differences in the sensitivity of neural representations to local stimulus perturbations and thus fail to reflect how systems leverage local evidence for discrimination. This work proposes a novel analytical framework based on locally decodable information, integrating Fisher information, pullback metrics, and log-spectral distances on the SPD manifold to construct the Spectral Riemannian Alignment Score (S-RAS). S-RAS provides, for the first time, a minimal, dataset-level summary of neural representational sensitivity from the perspective of local discriminative tasks, with guaranteed multiplicative consistency. The method successfully aligns corresponding layers across independently trained networks, enables transferable class-conditional probing, reveals representational differences between standard and robustly trained models, and uncovers stimulus coordinate family effects in mouse visual cortex.
📝 Abstract
Activation-alignment measures such as Representational Similarity Analysis (RSA), Canonical Correlation Analysis (CCA), and Centered Kernel Alignment (CKA) are widely used to compare biological and artificial neural representations. Recent theoretical work interprets many of these methods as assessing agreement between optimal linear readouts over broad families of global tasks. However, agreement at the level of global readouts does not determine how a system uses local stimulus evidence. Specifically, representations may align in activation space yet differ in their sensitivity to small perturbations. To address this challenge, we introduce a complementary framework based on local decodable information, which focuses on a representation's ability, under noise, to discriminate small perturbations within a specified stimulus-coordinate subspace. Building on Fisher information and local representation geometry, we summarize each representation using the expected projected pullback/Fisher metric over that subspace. This formulation induces a second-moment family of local discrimination tasks, for which the resulting operator provides a minimal, complete dataset-level summary of expected discriminability. We compare these regularized signatures using a log-spectral distance on the manifold of symmetric positive definite (SPD) matrices, yielding the Spectral Riemannian Alignment Score (S-RAS) and a uniform multiplicative certificate over the corresponding family of lifted task values. Empirically, this framework enables the recovery of corresponding layers across independently trained artificial neural networks, supports transferable class-conditional probes, reveals controlled dissociations between standard and robust training, and uncovers stimulus-coordinate family effects across mouse visual cortex using the Allen Brain Observatory static gratings dataset.