🤖 AI Summary
Existing Centered Kernel Alignment (CKA) methods neglect the underlying manifold structure of data and rely on heuristic designs, resulting in poor stability across data scales and limited interpretability. To address this, we propose Manifold-aware Kernel Alignment (MKA), the first framework to systematically integrate manifold geometry into kernel alignment: it constructs scale-adaptive kernels via local manifold approximation and establishes a theoretically grounded, geometrically consistent alignment criterion. MKA requires no hyperparameter tuning and significantly improves robustness and discriminability in representation comparison—especially under few-shot and multi-scale regimes—on both synthetic and real-world benchmarks. Experiments demonstrate its superiority in tasks such as representation equivalence testing and neural representational analysis, delivering more reliable and interpretable similarity metrics. By bridging manifold learning with kernel-based representation comparison, MKA provides a principled tool for representation learning and computational neuroscience.
📝 Abstract
Centered kernel alignment (CKA) is a popular metric for comparing representations, determining equivalence of networks, and neuroscience research. However, CKA does not account for the underlying manifold and relies on numerous heuristics that cause it to behave differently at different scales of data. In this work, we propose Manifold approximated Kernel Alignment (MKA), which incorporates manifold geometry into the alignment task. We derive a theoretical framework for MKA. We perform empirical evaluations on synthetic datasets and real-world examples to characterize and compare MKA to its contemporaries. Our findings suggest that manifold-aware kernel alignment provides a more robust foundation for measuring representations, with potential applications in representation learning.