🤖 AI Summary
Traditional relational databases struggle to efficiently process multimodal, context-rich data: relational operators lack contextual modeling capabilities, while representation learning models cannot be readily integrated into declarative query frameworks. This paper proposes **context-enhanced relational join operators**, introducing the first composable embedding operator that natively integrates vector embeddings into relational algebra. We design algebraic equivalence rules and logical/physical optimization mechanisms to build a vector-relational hybrid execution engine, and propose a协同 optimization strategy jointly leveraging sequential scans and vector indexes. Our approach preserves SQL’s declarative semantics while enabling multimodal, context-aware querying. Experiments demonstrate up to an order-of-magnitude improvement in query performance over pure vector database solutions. The system validates the effectiveness of our optimization techniques and characterizes performance trade-offs across diverse application scenarios.
📝 Abstract
Collecting data, extracting value, and combining insights from relational and context-rich sources of many modalities in data processing pipelines presents a challenge for traditional relational DBMS. While relational operators enable declarative and optimizable query specification, they are limited to unsuitable data transformations for capturing or analyzing context. On the other hand, representation learning models can map context-rich data into embeddings, enabling machine-automated context processing but requiring imperative data transformation integration with the analytical query. We present a context-enhanced relational join operator to bridge this dichotomy and introduce an embedding operator composable with relational operators. This approach enables hybrid relational and context-rich vector data processing, with algebraic equivalences compatible with relational algebra and corresponding logical and physical optimizations. We investigate model-operator interaction with vector data processing and study the characteristics of the join operator. We demonstrate the hybrid context-enhanced relational join operators with vector embeddings and evaluate it against a vector database approach. We show step-by-step the impact of logical and physical optimizations, which result in orders of magnitude execution time improvement resulting in tensor join formulation. We also outline the performance tradeoffs and cases of using scan-based processing against vector indexes.