Learning over von Mises-Fisher Distributions via a Wasserstein-like Geometry

2025-04-19

AI Summary

This work addresses the lack of geometrically consistent, analytically tractable distance metrics between von Mises–Fisher (vMF) distributions on the unit hypersphere. We propose a Wasserstein-like distance explicitly adapted to the spherical geometry, which decomposes the dissimilarity between vMF distributions into two interpretable components: directional angular distance and concentration parameter divergence. Grounded in optimal transport theory, our metric leverages spherical Gaussian approximations under high-concentration regimes and incorporates geodesic distance modeling to induce a non-degenerate implicit geometric structure on the space of non-degenerate vMF distributions. Crucially, the resulting distance admits closed-form computation while preserving theoretical rigor. Furthermore, we develop an efficient algorithm for reducing vMF mixture models, demonstrating substantial improvements in distribution discriminability and interpretability on high-dimensional real-world data—including biomedical sentence representations and visual features.

Technology Category

• Machine Learning: Learning with Manifolds
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Search and Optimization: Distributed Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

We introduce a novel, geometry-aware distance metric for the family of von Mises-Fisher (vMF) distributions, which are fundamental models for directional data on the unit hypersphere. Although the vMF distribution is widely employed in a variety of probabilistic learning tasks involving spherical data, principled tools for comparing vMF distributions remain limited, primarily due to the intractability of normalization constants and the absence of suitable geometric metrics. Motivated by the theory of optimal transport, we propose a Wasserstein-like distance that decomposes the discrepancy between two vMF distributions into two interpretable components: a geodesic term capturing the angular separation between mean directions, and a variance-like term quantifying differences in concentration parameters. The derivation leverages a Gaussian approximation in the high-concentration regime to yield a tractable, closed-form expression that respects the intrinsic spherical geometry. We show that the proposed distance exhibits desirable theoretical properties and induces a latent geometric structure on the space of non-degenerate vMF distributions. As a primary application, we develop the efficient algorithms for vMF mixture reduction, enabling structure-preserving compression of mixture models in high-dimensional settings. Empirical results on synthetic datasets and real-world high-dimensional embeddings, including biomedical sentence representations and deep visual features, demonstrate the effectiveness of the proposed geometry in distinguishing distributions and supporting interpretable inference. This work expands the statistical toolbox for directional data analysis by introducing a tractable, transport-inspired distance tailored to the geometry of the hypersphere.

Problem

• Introducing a geometry-aware distance metric for von Mises-Fisher distributions
• Addressing the lack of suitable geometric metrics for comparing vMF distributions
• Developing efficient algorithms for vMF mixture reduction in high-dimensional settings

Innovation

• Wasserstein-like distance for vMF distributions
• Decomposes discrepancy into geodesic and variance terms
• Gaussian approximation for tractable closed-form expression