Database-Augmented Query Representation for Information Retrieval

2024-06-23

arXiv.org

AI Summary

To address the low retrieval accuracy caused by short user queries, this paper proposes DAQu, a database-augmented query representation framework that dynamically expands query semantics using relational database metadata across multiple interlinked tables. Methodologically, it introduces a novel graph-structured unordered set encoding strategy to model hierarchical cross-table metadata relationships and integrate high-dimensional heterogeneous features; it jointly optimizes graph neural networks, metadata modeling, set encoding, query expansion, and dense retrieval. On multi-scenario retrieval tasks, DAQu achieves a 12.7% improvement in Recall@10 over state-of-the-art query enhancement methods, demonstrating the substantial benefit of leveraging structured database knowledge for query representation. The core contribution lies in formulating database metadata as a learnable, graph-structured prior and enabling end-to-end semantic-enhanced retrieval.

Technology Category

Data Mining & Knowledge Management: Intelligent Query ProcessingSearch and Optimization: Distributed SearchKnowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

Search and Retrieval-Augmented AI: Web query analysis, representation and understandingGraph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphsSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Information retrieval models that aim to search for the documents relevant to the given query have shown many successes, which have been applied to diverse tasks. However, the query provided by the user is oftentimes very short, which challenges the retrievers to correctly fetch relevant documents. To tackle this, existing studies have proposed expanding the query with a couple of additional (user-related) features related to the query. Yet, they may be suboptimal to effectively augment the query, though there is plenty of information available to augment it in a relational database. Motivated by this, we present a novel retrieval framework called Database-Augmented Query representation (DAQu), which augments the original query with various (query-related) metadata across multiple tables. In addition, as the number of features in the metadata can be very large and there is no order among them, we encode them with our graph-based set encoding strategy, which considers hierarchies of features in the database without order. We validate DAQu in diverse retrieval scenarios that can incorporate metadata from the relational database, demonstrating that ours significantly enhances overall retrieval performance, compared to existing query augmentation methods.

Problem

• Addressing short query challenge in information retrieval
• Augmenting queries with metadata from relational databases
• Encoding unordered metadata features via graph-based strategy

Innovation

• Augmenting queries with relational database metadata
• Using graph-based set-encoding for feature hierarchies
• Enhancing retrieval performance across diverse scenarios