Stitching Inner Product and Euclidean Metrics for Topology-aware Maximum Inner Product Search

2025-04-21

AI Summary

To address local optima, redundant computation, and topological distortion with information loss caused by projection-based methods in high-dimensional Maximum Inner Product Search (MIPS), this paper challenges the conventional binary opposition between inner-product (IP) and Euclidean metrics. We propose a topology-aware Metric-Amphibious Graph (MAG) index and an Adaptive Navigation with Metric Switch (ANMS) algorithm. By synergistically leveraging the strengths of both IP and Euclidean metrics, our approach introduces three topology-aware statistical metrics for automatic parameter tuning. Integrating graph-based indexing, metric-space stitching, and topology-preserving modeling, MAG-ANMS achieves state-of-the-art performance across 12 real-world datasets—up to 4× faster retrieval—while demonstrating strong generalizability, adaptability, and scalability.

Technology Category

• Search and Optimization: Distributed Search
• Machine Learning: Learning with Manifolds
• Multiagent Systems: Other Foundations of Multi Agent Systems

Application Category

• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Maximum Inner Product Search (MIPS) is a fundamental challenge in machine learning and information retrieval, particularly in high-dimensional data applications. Existing approaches to MIPS either rely solely on Inner Product (IP) similarity, which faces issues with local optima and redundant computations, or reduce the MIPS problem to the Nearest Neighbor Search under the Euclidean metric via space projection, leading to topology destruction and information loss. Despite the divergence of the two paradigms, we argue that there is no inherent binary opposition between IP and Euclidean metrics. By stitching IP and Euclidean in the design of indexing and search algorithms, we can significantly enhance MIPS performance. Specifically, this paper explores the theoretical and empirical connections between these two metrics from the MIPS perspective. Our investigation, grounded in graph-based search, reveals that different indexing and search strategies offer distinct advantages for MIPS, depending on the underlying data topology. Building on these insights, we introduce a novel graph-based index called Metric-Amphibious Graph (MAG) and a corresponding search algorithm, Adaptive Navigation with Metric Switch (ANMS). To facilitate parameter tuning for optimal performance, we identify three statistical indicators that capture essential data topology properties and correlate strongly with parameter tuning. Extensive experiments on 12 real-world datasets demonstrate that MAG outperforms existing state-of-the-art methods, achieving up to 4x search speedup while maintaining adaptability and scalability.

Problem

• Addresses limitations in Maximum Inner Product Search (MIPS) methods
• Bridges Inner Product and Euclidean metrics for better MIPS performance
• Introduces topology-aware graph index and adaptive search algorithm

Innovation

• Stitching IP and Euclidean metrics for MIPS
• Introducing Metric-Amphibious Graph (MAG) index
• Adaptive Navigation with Metric Switch (ANMS)