nearest-neighbor search

To address the high computational and storage overhead in large-scale semantic text retrieval, this paper systematically surveys and advances deep text hashing research. Methodologically, it introduces the first taxonomy of technical components for deep text hashing and establishes a unified evaluation framework covering semantic modeling, discrete optimization, and code-quality constraints. It further proposes an end-to-end trainable architecture that jointly optimizes deep semantic encoding and Hamming distance–aware hash learning, substantially improving the semantic fidelity of binary codes. Extensive experiments on multiple standard benchmarks demonstrate superior retrieval accuracy, efficiency, and compression ratio over state-of-the-art methods. The implementation is open-sourced to ensure reproducibility. Additionally, the paper前瞻性 explores a novel paradigm integrating semantic hashing with large language models, laying groundwork for future synergistic architectures.

This work addresses a critical limitation in traditional approximate nearest neighbor (ANN) retrieval evaluation, which relies solely on recall and fails to distinguish semantically relevant from irrelevant neighbors, often misrepresenting retrieval quality. To overcome this, the authors introduce Semantic Recall—a novel metric that quantifies only those semantically relevant results theoretically recoverable via exact search—and propose Tolerant Recall as an efficient proxy for practical evaluation. This is the first systematic integration of semantic relevance into vector retrieval assessment, revealing the pervasive sparsity of relevant results within embedding spaces. Experimental results demonstrate that the proposed metrics more accurately reflect real-world retrieval effectiveness, and algorithms optimized under this framework achieve superior trade-offs between cost and quality.

To address the challenge of efficient similarity search and multi-dimensional attribute filtering over billion-scale datasets on CPU-only systems, this paper proposes a disk-resident hybrid indexing paradigm. Our method deeply integrates multi-dimensional discrete filtering into the IVF-Flat architecture for the first time, jointly encoding dense embeddings and discrete features. We design a disk-aware hierarchical filtering mechanism and a CPU-native vectorized computation pipeline. The approach supports complex queries with millisecond-level latency while significantly improving filtering accuracy by 32% and reducing memory footprint by 58% compared to state-of-the-art CPU-based methods such as FAISS and Annoy. By unifying dense and discrete representations within a disk-efficient framework, our solution achieves a superior balance among retrieval efficiency, accuracy, and resource utilization—particularly critical for large-scale, memory-constrained CPU deployments.

Traditional vector retrieval relies on approximate nearest neighbor (ANN) search, which often yields semantically redundant results and fails to meet the diversity and contextual richness requirements of applications such as retrieval-augmented generation (RAG) and multi-hop question answering. To address this, we propose a novel paradigm—“Semantic Compression and Graph-Enhanced Retrieval”—that introduces submodular optimization into vector retrieval for the first time. We formalize semantic compression to maximize information coverage while explicitly suppressing redundancy. Leveraging information-geometric similarity metrics and k-nearest neighbor (kNN) graphs, we construct a multi-hop semantic search framework, further augmented with knowledge graph integration for structured semantic querying. Our method supports hybrid indexing and significantly improves semantic diversity and coverage in high-dimensional embedding spaces, outperforming state-of-the-art ANN baselines. The implementation is open-sourced, advancing research toward semantics-centric vector search.

This work addresses the problem of cardinality estimation for similarity queries in high-dimensional spaces by proposing a novel method that balances accuracy and online efficiency. The approach leverages locality-sensitive hashing (LSH) to partition the space and integrates adaptive multi-probe bucket probing, progressive sampling, and asymmetric distance computation. It also supports dynamic data updates, making it suitable for evolving datasets. Experimental results demonstrate that the proposed scheme significantly outperforms existing methods across multiple high-dimensional datasets, achieving high estimation accuracy while substantially improving online query response time. The method is thus well-suited for large-scale applications involving both static and dynamic data.

This work addresses the challenge of efficient approximate nearest neighbor search in $\ell_p$ metric spaces for $p > 2$ under large-scale data settings. The authors propose a novel approach combining randomized data structures with metric embeddings, leveraging a careful analysis of the geometric properties of $\ell_p$ spaces to design an effective indexing strategy. Their method achieves query time nearly logarithmic in the dataset size—specifically $\text{poly}(d \log n)$—with only polynomial space overhead $\text{poly}(dn)$ and provably controlled approximation error. A key contribution is the first known approximation ratio of $p^{O(1) + \log \log p}$, which significantly improves upon or is incomparable to the best prior results (e.g., from 2019 and 2025), thereby overcoming a longstanding performance bottleneck in high-dimensional $\ell_p$ approximate nearest neighbor search.

This work addresses the well-known degradation in indexing efficiency and performance of traditional locality-sensitive hashing (LSH) methods in high-dimensional settings. To overcome this limitation, the authors propose DET-LSH, a novel LSH scheme based on a dynamic encoding tree (DE-Tree), along with its parallelized variant, PDET-LSH, which integrates multi-tree range querying with multi-core CPU parallelism. The proposed approach significantly enhances both indexing and query efficiency while preserving theoretical guarantees on retrieval accuracy. Experimental results demonstrate that, compared to state-of-the-art LSH techniques, PDET-LSH achieves up to a 40-fold speedup in index construction and up to a 62-fold acceleration in query processing, all while maintaining optimal accuracy.

This work addresses the challenge of efficiently supporting both vector similarity search and arbitrary attribute filtering in high-dimensional approximate nearest neighbor retrieval. The authors propose a lightweight graph-based indexing algorithm that seamlessly integrates attribute filtering into the graph traversal process, overcoming the efficiency bottlenecks of existing methods when handling unseen query vectors combined with complex attribute constraints. Experimental results on multiple real-world datasets demonstrate that the proposed approach significantly outperforms state-of-the-art techniques, achieving substantially faster query latency while maintaining high recall. The method thus offers a compelling balance among flexibility, efficiency, and scalability for hybrid vector-and-attribute search scenarios.

This work addresses the challenge of efficiently merging graph indices in distributed systems and real-time vector databases, a problem previously lacking systematic investigation. To this end, the authors propose FGIM, a general and efficient three-stage framework for graph index merging. FGIM first converts input navigable graphs (e.g., HNSW) into k-nearest neighbor graphs (k-NNGs), then enhances neighbor quality and graph connectivity through cross-query candidate extraction and k-NNG refinement, and finally reconstructs a high-quality navigable graph. Extensive experiments on six real-world datasets demonstrate that FGIM achieves up to 3.5× speedup over incremental HNSW construction and averages 7.9× acceleration compared to non-incremental baselines, while maintaining comparable or superior retrieval accuracy.

This work proposes a novel approach to large-scale retrieval that circumvents the prohibitive cost of full reranking by constructing query and item embeddings derived from the outputs of a reranker. Specifically, it leverages relevance scores assigned by a heavyweight reranker over a set of support items to generate lightweight embeddings, thereby enabling the reranking model to directly guide embedding learning—a capability demonstrated here for the first time. Under mild conditions, the method is theoretically shown to approximate arbitrarily complex similarity functions. Through systematic investigation of support item selection strategies and integration with approximate nearest neighbor search, the approach significantly improves candidate set quality across multiple academic and industrial datasets while maintaining computational efficiency.

This work addresses the significant performance degradation of existing quantization-based approximate nearest neighbor (ANN) methods under large-k queries, primarily caused by inefficient top-k result collection and costly re-ranking. To overcome these limitations, the authors propose a Bucket-based Collector (BBC), which organizes candidate vectors into distance-based buckets to reduce both candidate maintenance overhead and final sorting costs. Additionally, they introduce two efficient re-ranking algorithms tailored to different quantization schemes, effectively minimizing the number of items requiring re-ranking and mitigating cache misses. Experimental results demonstrate that, at a recall@k of 0.95, BBC accelerates state-of-the-art quantization-based ANN methods by up to 3.8×.