🤖 AI Summary
Traditional vector retrieval systems exhibit high encapsulation, limiting the ability of AI agents to flexibly intervene in embedding generation and scoring during query time, thereby hindering fine-grained control over retrieval. To address this, this work proposes flexvec, a novel retrieval kernel featuring the first programmable embedding modulation (PEM) mechanism, which exposes embedding matrices and score arrays as programmable interfaces to enable dynamic, composable semantic manipulations at query time. By natively integrating with SQL, supporting runtime arithmetic operations, and leveraging query materialization, flexvec efficiently executes complex modulations without relying on approximate indexing: it completes three types of compositional operations end-to-end over 240k text chunks in just 19 milliseconds, and scales to 82 milliseconds on million-scale datasets—all on a desktop CPU.
📝 Abstract
As AI agents become the primary consumers of retrieval APIs, there is an opportunity to expose more of the retrieval pipeline to the caller. flexvec is a retrieval kernel that exposes the embedding matrix and score array as a programmable surface, allowing arithmetic operations on both before selection. We refer to composing operations on this surface at query time as Programmatic Embedding Modulation (PEM). This paper describes a set of such operations and integrates them into a SQL interface via a query materializer that facilitates composable query primitives. On a production corpus of 240,000 chunks, three composed modulations execute in 19 ms end-to-end on a desktop CPU without approximate indexing. At one million chunks, the same operations execute in 82 ms.