Retrieval-Augmented Semantic Parsing: Using Large Language Models to Improve Generalization

2024-12-13

arXiv.org

AI Summary

Open-domain semantic parsing suffers from poor out-of-distribution generalization, particularly for unseen semantic units; existing neural models rely on heuristic rules and fail to effectively model novel lexical or compositional concepts. To address this, we propose RASP—a retrieval-augmented semantic parsing framework that enables deep synergy between large language models (LLMs) and external lexical knowledge bases. RASP dynamically retrieves relevant lexical knowledge during inference and injects it directly into the LLM’s decoding process, enabling real-time grounding of unfamiliar concepts. Evaluated across multiple open-domain semantic parsing benchmarks, RASP achieves near-doubling in zero-shot accuracy on unseen concepts, substantially outperforming conventional encoder-decoder baselines. These results demonstrate that tight integration of retrieval augmentation with LLMs significantly enhances both effectiveness and robustness in semantic generalization.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Open-domain semantic parsing remains a challenging task, as models often rely on heuristics and struggle to handle unseen concepts. In this paper, we investigate the potential of large language models (LLMs) for this task and introduce Retrieval-Augmented Semantic Parsing (RASP), a simple yet effective approach that integrates external lexical knowledge into the parsing process. Our experiments not only show that LLMs outperform previous encoder-decoder baselines for semantic parsing, but that RASP further enhances their ability to predict unseen concepts, nearly doubling the performance of previous models on out-of-distribution concepts. These findings highlight the promise of leveraging large language models and retrieval mechanisms for robust and open-domain semantic parsing.

Problem

• Improving generalization in semantic parsing with lexical knowledge
• Enhancing handling of unseen concepts in open-domain parsing
• Integrating external symbolic knowledge into parsing processes

Innovation

• Integrates external symbolic knowledge parsing
• Uses large language models semantic parsing
• Enhances unseen concepts prediction performance