Query Understanding in LLM-based Conversational Information Seeking

2025-04-08

AI Summary

Conversational information retrieval (CIS) faces significant challenges due to high user intent volatility, strong query ambiguity, and deep contextual dependencies. Method: This paper proposes an LLM-driven multi-turn query understanding framework featuring a context-aware intent parsing model that integrates instruction tuning with interactive reasoning. It introduces, for the first time, proactive query management and adaptive query reconstruction mechanisms—overcoming limitations of conventional static query modeling. Technically, the framework incorporates LLM-based contextual encoding, dynamically constructed evaluation metrics, and an interpretability verification module. Contribution/Results: Experiments on mainstream CIS benchmarks demonstrate a 12.3% improvement in intent identification F1-score and substantial gains in query rewriting quality. Furthermore, we release the first open-source evaluation protocol specifically designed for LLM-enhanced CIS, establishing foundational standards for systematic, reproducible benchmarking in this emerging domain.

Technology Category

• Data Mining & Knowledge Management: Intelligent Query Processing
• Natural Language Processing: Question Answering
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Query understanding in Conversational Information Seeking (CIS) involves accurately interpreting user intent through context-aware interactions. This includes resolving ambiguities, refining queries, and adapting to evolving information needs. Large Language Models (LLMs) enhance this process by interpreting nuanced language and adapting dynamically, improving the relevance and precision of search results in real-time. In this tutorial, we explore advanced techniques to enhance query understanding in LLM-based CIS systems. We delve into LLM-driven methods for developing robust evaluation metrics to assess query understanding quality in multi-turn interactions, strategies for building more interactive systems, and applications like proactive query management and query reformulation. We also discuss key challenges in integrating LLMs for query understanding in conversational search systems and outline future research directions. Our goal is to deepen the audience's understanding of LLM-based conversational query understanding and inspire discussions to drive ongoing advancements in this field.

Problem

• Enhancing query understanding in LLM-based conversational search systems
• Developing robust evaluation metrics for multi-turn interactions
• Addressing challenges in LLM integration for conversational search

Innovation

• LLMs enhance query understanding dynamically
• Develop robust metrics for multi-turn interactions
• Proactive query management and reformulation strategies