Beyond Perturbation Magnitude: Direction-Dependent Responses in Multimodal Geometric Representations
This study addresses the challenge that responses of multimodal geometric alignment scores to modality degradation cannot be adequately explained by perturbation magnitude alone, which accounts for only a small fraction of variance. To overcome this limitation, we propose Directional Geometric Response (DGR) theory, departing from conventional scalar perspectives. By leveraging Gramian volume gradient projections and first-order Taylor expansions, DGR integrates operating points, magnitudes, and directions to precisely model geometric volume variations. The framework is validated through experiments employing frozen embeddings with controlled audio-visual noise injection. Our findings demonstrate that directional dependence constitutes the primary driver of multimodal geometric responses. DGR achieves out-of-sample R² values ranging from 0.838 to 0.969 and ranking accuracy exceeding 0.864, significantly outperforming direction-agnostic baselines.