Efficient and Effective In-place Graph-based Vector Index Updates

2026-07-16

AI Summary

This work addresses the challenge of balancing recall accuracy and throughput efficiency under frequent vector updates in the era of large language models. The authors propose Yi, a novel graph-based vector indexing system that pioneers support for in-place updates. Guided by a “decompose-to-integrate” design philosophy, Yi introduces a vector-level update mechanism that overcomes the bottlenecks of traditional batch merging. It integrates three core components—a tasklet execution engine, an asynchronous buffer manager, and a vector file system—to enable efficient online updates and high-quality retrieval. Experiments on a dataset of 800 million vectors demonstrate that Yi achieves 1.75× higher update throughput, 1.8× greater concurrent search throughput, and reduces peak memory usage to 73% of baseline systems, delivering superior performance even with fewer CPU cores.

Abstract

In the era of Large Language Models (LLMs), efficient vector updates are critical for capturing real-time information from rapidly evolving data. However, it is not trivial to process frequent vector insert and delete updates and maintain a high recall of the search results simultaneously. Specifically, the cluster-based vector indexing methods have high update throughput but low search result quality. Existing out-of-place graph-based vector indexing update approaches suffer from poor update throughput due to the need to periodically merge update batches into the underlying graph index. Building a vector data system that supports efficient and effective in-place updates is inherently challenging. In this work, we propose Yi to achieve it. In particular, Yi supports in-place graph-based vector indexing updates with consistently high update throughput and good search result quality. The key idea of Yi is decomposition facilitates consolidation. In particular, we introduce a vector-level update mechanism and architect Yi with three core components: (i) a tasklet-based execution engine, (ii) an asynchronous buffer manager, and (iii) a vector file system. Experimental results demonstrate that Yi achieves 1.75x higher update throughput and 1.8x higher concurrent search throughput than the state-of-the-art systems on the 800M dataset, while using only 73% of the peak memory and fewer CPU cores.

Problem

• vector indexing
• in-place updates
• graph-based index
• update throughput
• search recall

Innovation

• in-place update
• graph-based vector index
• vector-level update
• tasklet-based execution
• asynchronous buffer management