ANNS-AMP: Accelerating Approximate Nearest Neighbor Search via Adaptive Mixed-Precision Computing

2026-06-05

AI Summary

This work addresses the inefficiency of fixed-precision computation in approximate nearest neighbor search (ANNS) for large-scale high-dimensional data, which leads to significant waste of computational resources and memory bandwidth. The authors propose ANNS-AMP, the first framework to introduce adaptive mixed-precision computation into ANNS. Leveraging the cluster structure of product quantization (PQ) indexes and a lightweight runtime predictor, ANNS-AMP dynamically determines the required precision for distance computations per cluster. It efficiently executes variable-precision calculations using a bit-serial accelerator, bit-interleaved memory layout, and greedy scheduling. Experimental results demonstrate that, while limiting accuracy loss to under 2.7%, ANNS-AMP achieves average speedups of 163.76×, 10.57×, and 2.06× over CPU, GPU, and state-of-the-art custom ANNS accelerators, respectively, with corresponding energy reductions of up to 1100×, 39.41×, and 6.66×.

Abstract

Approximate nearest neighbor search(ANNS) is a critical kernel in modern applications such as LLM and recommendation systems.However,its efficiency is fundamentally limited by the need to compute distances between a query and a massive number of high-dimensional vectors,most of which are non-neighbors.Existing approaches reduce redundancy via index optimization or early termination,but remain constrained by fixed-precision computation,leading to unnecessary arithmetic and memory bandwidth overhead.This paper presents ANNS-AMP,an adaptive mixed-precision framework and accelerator that adapts the precision of distance computation to the characteristics of queries and data distribution.The key insight is that different regions of the vector space require different levels of precision to preserve top-k accuracy.ANNS-AMP leverages the clustered structure of PQ-based indices and introduces a lightweight predictor to determine cluster-level precision at runtime based on features such as scale,radius,and query distance.To efficiently realize variable-precision execution,we design a bit-serial accelerator with a bit-interleaved data layout,enabling throughput to scale with reduced precision while mitigating memory bandwidth bottlenecks and load imbalance through a greedy scheduling strategy.Moreover,the runtime predictor can also reuse the bit-serial computing array for efficient runtime prediction and can be fitted to the ANNS pipeline without performance penalty.According to our experiments on representative datasets,ANNS-AMP achieves 163.76x,10.57x,and 2.06x performance speedups on average,and reduces average energy consumption by 1100.00x,39.41x,and 6.66x compared to CPU,GPU,and customized ANNS accelerator baselines,respectively,while maintaining accuracy loss below 2.7%.These results demonstrate that adaptive mixed-precision computing is a promising direction for efficient large-scale ANNS.

Problem

• Approximate Nearest Neighbor Search
• Mixed-Precision Computing
• Memory Bandwidth Overhead
• High-Dimensional Vectors
• Fixed-Precision Computation

Innovation

• adaptive mixed-precision
• approximate nearest neighbor search
• bit-serial accelerator
• runtime precision prediction
• PQ-based indexing