🤖 AI Summary
This work proposes the first projective-geometric framework for measuring representational drift, grounded in the Fubini–Study metric, which treats high-dimensional representations as points in projective space and quantifies their geometric evolution along trajectories. Conventional metrics—such as Euclidean or cosine distance—are prone to conflating genuine structural changes in data with artifacts induced by parametrization ambiguities, such as global scaling or sign flips. In contrast, the proposed approach is gauge-invariant, effectively disentangling intrinsic representational dynamics from spurious perturbations due to parameterization freedom. It further introduces a computable, monotonic quantity to rigorously quantify representational churn. Experiments on real high-dimensional data demonstrate that this framework avoids the systematic overestimation of drift inherent in traditional measures, yielding stable and interpretable diagnostics suitable for general-purpose empirical analysis pipelines.
📝 Abstract
High dimensional representation drift is commonly quantified using Euclidean or cosine distances, which presuppose fixed coordinates when comparing representations across time, training or preprocessing stages. While effective in many settings, these measures entangle intrinsic changes in the data with variations induced by arbitrary parametrizations. We introduce a projective geometric view of representation drift grounded in the Fubini Study metric, which identifies representations that differ only by gauge transformations such as global rescalings or sign flips. Applying this framework to empirical high dimensional datasets, we explicitly construct representation trajectories and track their evolution through cumulative geometric drift. Comparing Euclidean, cosine and Fubini Study distances along these trajectories reveals that conventional metrics systematically overestimate change whenever representations carry genuine projective ambiguity. By contrast, the Fubini Study metric isolates intrinsic evolution by remaining invariant under gauge-induced fluctuations. We further show that the difference between cosine and Fubini Study drift defines a computable, monotone quantity that directly captures representation churn attributable to gauge freedom. This separation provides a diagnostic for distinguishing meaningful structural evolution from parametrization artifacts, without introducing model-specific assumptions. Overall, we establish a geometric criterion for assessing representation stability in high-dimensional systems and clarify the limits of angular distances. Embedding representation dynamics in projective space connects data analysis with established geometric programs and yields observables that are directly testable in empirical workflows.