Better Nearest Neighbor Graph Indices via (Efficient) LLM-Guided Pruning

2026-09-28

AI Summary

This study addresses the fundamental mismatch between geometric index construction and semantic retrieval in graph-based approximate nearest neighbor search by proposing a large language model (LLM)-guided graph pruning framework. For the first time, this work introduces LLMs into the index construction phase, leveraging their reasoning capabilities to directly optimize the proximity graph structure. By replacing low-value neighbors with semantically relevant ones, the proposed method transcends previous limitations where LLMs were exclusively employed for query reranking. Extensive experiments conducted on established indices, including HNSW and DiskANN, demonstrate that this approach yields significantly superior end-to-end retrieval performance across multiple benchmarks compared to conventional greedy search strategies and existing LLM-based reranking methods.

Abstract

Graph-based approximate nearest neighbor search (ANNS) is widely used for large-scale semantic search. Its indices are constructed primarily based on geometric relationships among embeddings of an input dataset (e.g., documents or images), rather than explicitly optimizing for semantic relevance. However, when using these indices for downstream query retrieval, performance is evaluated based on the semantic relevance of the retrieved results to the query. This creates a fundamental "geometry-semantic" mismatch between how the indices are constructed and how their retrieval results are evaluated. While existing LLM-based reranking methods can partially mitigate this mismatch at query time, they leave this underlying structural problem in the graph unresolved. We therefore propose LLM-Guided Graph Pruning (LGP), a general framework that addresses this mismatch directly by leveraging LLM reasoning to refine an existing ANN graph index itself. LGP identifies structurally "low-value" neighbors of nodes and replaces them with LLM-selected alternatives that provide useful semantic information while retaining desired geometric structures of the original graph, including sparsity and efficient navigability. Experiments on representative semantic retrieval benchmarks show that LGP consistently improves end-to-end retrieval performance over both vanilla greedy graph search and LLM-based reranking across widely used graph-based ANN indices such as DiskANN and HNSW.

Problem

• Approximate Nearest Neighbor Search
• Graph Index
• Semantic Relevance
• Geometry-Semantic Mismatch

Innovation

• Approximate Nearest Neighbor Search
• LLM-Guided Graph Pruning
• Graph Index Optimization
• Semantic Retrieval
• Geometry-Semantic Mismatch