RAGONITE: Iterative Retrieval on Induced Databases and Verbalized RDF for Conversational QA over KGs with RAG

2024-12-23

AI Summary

Contemporary conversational question-answering systems exhibit limited performance on knowledge graphs (KGs) for complex, multi-turn, and abstract queries, primarily due to their fragile reliance on SPARQL translation. To address this, we propose a dual-path iterative retrieval-augmented generation (RAG) framework: (1) automatically mapping the KG to an SQL database to enable structured querying, and (2) verbalizing RDF triples into natural language to support semantic text-based retrieval. A dynamic turn-level decision mechanism coordinates multi-turn user feedback and dynamically selects between the two retrieval paths. This design eliminates SPARQL dependency while synergistically integrating structural and semantic retrieval strengths. Experiments on a BMW automotive KG show a 27% absolute improvement in accuracy for complex multi-turn QA over SPARQL-based baselines and substantial gains over unimodal RAG variants. Human evaluation further confirms superior response relevance and interpretability.

Technology Category

Data Mining & Knowledge Management: Conversational Systems for Recommendation & RetrievalNatural Language Processing: Question AnsweringKnowledge Representation and Reasoning: Knowledge Representation Languages

Application Category

Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAGSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsGraph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Conversational question answering (ConvQA) is a convenient means of searching over RDF knowledge graphs (KGs), where a prevalent approach is to translate natural language questions to SPARQL queries. However, SPARQL has certain shortcomings: (i) it is brittle for complex intents and conversational questions, and (ii) it is not suitable for more abstract needs. Instead, we propose a novel two-pronged system where we fuse: (i) SQL-query results over a database automatically derived from the KG, and (ii) text-search results over verbalizations of KG facts. Our pipeline supports iterative retrieval: when the results of any branch are found to be unsatisfactory, the system can automatically opt for further rounds. We put everything together in a retrieval augmented generation (RAG) setup, where an LLM generates a coherent response from accumulated search results. We demonstrate the superiority of our proposed system over several baselines on a knowledge graph of BMW automobiles.

Problem

• Complex Query Processing
• Dialog-based Question Answering
• Abstract Demand Handling

Innovation

• RAGONITE
• RAG Technology
• Dialog-based Question Answering