mahalanobis cosine similarity

This work addresses the limitation of conventional cosine similarity in evaluating linear probes, which neglects task-specific data distributions and thus fails to reliably predict out-of-distribution (OOD) performance. The authors propose Mahalanobis Cosine Similarity (MCS), a task-aware probe similarity metric that incorporates the covariance of test data to weight the inner product. Under assumptions of Gaussian projections and class balance, they theoretically establish—for the first time—that both OOD AUROC and MCS are sigmoidal functions of the signal-to-noise ratio, yielding an approximate linear relationship between them and delineating the boundary conditions under which this relationship breaks down. Extensive experiments across diverse models, network layers, and concept domains demonstrate a strong linear correlation (R² = 0.98) between MCS and OOD AUROC, substantially outperforming standard cosine similarity.

In high-dimensional data, conventional similarity measures such as cosine similarity suffer from poor interpretability and degraded discriminability due to inherent dimensionality dependence. To address this, we propose Dimension-Independent Euclidean Measure (DIEM)—the first Euclidean-style metric with theoretically guaranteed dimension independence. DIEM rigorously characterizes the dimensional bias mechanism of cosine similarity and eliminates variance drift across dimensions via geometric normalization and dimension-adaptive scaling, ensuring consistent and comparable similarity assessments regardless of dimensionality. Extensive experiments across diverse scenarios—including electromyographic synergy analysis, PCA embedding, and clustering—demonstrate that DIEM significantly improves discriminative accuracy and result interpretability. Its coefficient of variation remains stable and substantially lower than that of cosine similarity and other baselines. DIEM establishes a robust, interpretable, and theoretically sound paradigm for comparing high-dimensional vectors.

This work addresses the limited interpretability of cosine similarity in pretrained embedding spaces, where absolute similarity values are concentrated in a narrow range due to anisotropy. The authors propose a monotonic calibration method based on isotonic regression that reparameterizes similarity scores without altering the underlying embeddings or their geometric structure. The approach strictly preserves the original ranking order of similarities, thereby maintaining all ordinal-dependent structures—such as nearest neighbors, angular rankings, and threshold graphs—intact. The calibrated similarities achieve near-perfect alignment in absolute values while retaining 98% local stability under seven types of perturbations and fully preserving rank correlation with the original similarities.

Quantifying and interpreting representational similarity between biological systems (e.g., neural activity) and artificial systems (e.g., deep neural networks) remains challenging due to ambiguities in metric choice, interpretability, and functional relevance. Method: We propose the first end-to-end differentiable optimization framework that directly maximizes representational similarity scores between model and neural representations. Using theoretical analysis and synthetic data inversion, we systematically characterize how CKA, angular Procrustes, and normalized Bures similarity (NBS) differentially weight principal component variance. Contributions: We show CKA strongly biases toward high-variance components, whereas angular Procrustes captures low-variance neural dimensions earlier; high similarity scores do not imply functional equivalence, and no universal threshold exists for neuroscientific interpretation; finally, we delineate the feasible score space and hierarchical constraint strengths under multi-metric joint optimization—establishing a theoretical benchmark and practical guidance for representational similarity assessment.

This work addresses the challenges of anomaly detection in high-dimensional data, where traditional methods often suffer from performance degradation and sensitivity to parameter settings. The authors propose a novel Multidimensional Outlier Detection (MDOD) algorithm that enhances discriminative power by extending the original data with an additional dimension filled with zeros and constructing vectors rooted at a designated observation point. Anomalies are identified through cosine similarity measures between these vectors. By innovatively integrating dimensionality expansion with an observation-point-based mechanism, MDOD significantly improves detection accuracy in high-dimensional settings. Empirical evaluations across multiple datasets demonstrate the method’s efficiency and effectiveness, and the implementation has been made publicly available as the open-source Python package “mdod” on PyPI.

This work uncovers the intrinsic equivalence between scoring rules and cosine similarity by establishing a direct geometric connection through vector geometry and Euclidean distance optimization. From a least-squares perspective, it demonstrates that the arithmetic mean of score vectors uniquely minimizes the sum of squared Euclidean distances to all individual score vectors, naturally yielding the cosine similarity formulation. This approach not only provides a clear geometric interpretation of scoring rules but also simplifies existing proofs by revealing that the alignment between scoring rules and cosine similarity arises inevitably from this underlying optimization principle. The result clarifies why these two seemingly distinct formulations must coincide, grounding their equivalence in fundamental properties of vector space geometry.

This work addresses the recovery of latent geometric structure from nonlinear observations in the space of probability measures. It introduces, for the first time, the Wasserstein Mahalanobis distance—a geometry-aware metric between distributions constructed by combining optimal transport displacement fields with covariance operators on tangent spaces. This approach establishes a covariance-adaptive geometric framework for distribution-valued data, which exactly recovers the Mahalanobis distance of latent variables under affine transformations and achieves controllable error under general smooth transformations. Both theoretical analysis and numerical experiments demonstrate that the proposed distance effectively reconstructs the underlying latent geometry in the context of nonlinear independent component analysis.

This study investigates how the geometric properties of text embedding spaces influence the choice of parameter-free similarity measures. Through a systematic evaluation of 19 similarity metrics across 19 encoders and 7 datasets, the work reveals— for the first time—that embedding anisotropy is a key determinant of optimal metric selection. The authors propose “single-dimension variance ratio” as a predictive indicator of metric performance: cosine similarity excels under isotropic conditions, whereas rank-based or L1-based metrics achieve approximately 20% higher performance under anisotropy. This indicator exhibits a strong linear correlation (r = 0.95) with metric advantage, and ablation via projection onto the principal direction eliminates this correlation, confirming its causal role.

This study addresses the limitation of fixed geometric structures in contrastive learning, which struggle to accommodate context-dependent semantic similarity. To overcome this, we propose anchor divergence, a method that integrates contrastive learning, exponential family theory, and information geometry. By establishing a correspondence between anchor distributions and Bregman geometry, our approach defines a context-aware dynamic semantic geometry over fixed representations. Specifically, it directly controls the geometric structure by modeling anchor distributions, thereby enabling adaptive similarity measurement. Experimental results demonstrate that the proposed method efficiently and accurately characterizes context-dependent semantic similarity, yielding substantial improvements in retrieval performance.

This study addresses the challenge of constructing rotation-invariant vector representations for planar shapes by proposing a method that strictly encodes star-shaped normalized contours into Euclidean vectors. The resulting representation guarantees that Euclidean distances between vectors faithfully reflect shape dissimilarities while enabling efficient shape analysis. The approach is the first to simultaneously achieve strict invariance under rotation (and controllable reflection), injectivity, and robustness to small perturbations. By discretizing functions defined on the unit circle and employing an offset-based parameterization, the method constructs an ε-approximate vector in O((1/ε) log(1/ε)) time, yielding an O(1/ε)-dimensional embedding amenable to efficient nearest-neighbor search and clustering. Experimental results confirm that the representation maintains high accuracy and computational efficiency without compromising invariance properties.