Know Your Users! Estimating User Domain Knowledge in Conversational Recommenders

2025-12-15

AI Summary

Existing conversational recommendation systems (CRS) commonly assume users possess domain expertise, overlooking inter-user differences in domain knowledge—leading to inefficient interactions and degraded user experience, especially for novices. Method: We introduce the novel task of *user domain knowledge level estimation* to enable CRS to dynamically adapt linguistic expression and recommendation strategies. To this end, we design a gamified data collection protocol and construct UKDial—the first dialogue benchmark dataset covering users across diverse knowledge levels. We further propose a dialogue behavior modeling and knowledge annotation analysis framework. Contribution/Results: Empirical analysis reveals significant differences between novices and experts in feature exploration preferences, semantic granularity of expressions, and interaction pacing. This work establishes the formal task definition, provides the first dedicated dataset, and delivers empirical foundations for knowledge-aware, adaptive CRS.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Application Domains: Humanities & Computational Social Science
• Knowledge Representation and Reasoning: Preferences

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

The ideal conversational recommender system (CRS) acts like a savvy salesperson, adapting its language and suggestions to each user's level of expertise. However, most current systems treat all users as experts, leading to frustrating and inefficient interactions when users are unfamiliar with a domain. Systems that can adapt their conversational strategies to a user's knowledge level stand to offer a much more natural and effective experience. To make a step toward such adaptive systems, we introduce a new task: estimating user domain knowledge from conversations, enabling a CRS to better understand user needs and personalize interactions. A key obstacle to developing such adaptive systems is the lack of suitable data; to our knowledge, no existing dataset captures the conversational behaviors of users with varying levels of domain knowledge. Furthermore, in most dialogue collection protocols, users are free to express their own preferences, which tends to concentrate on popular items and well-known features, offering little insight into how novices explore or learn about unfamiliar features. To address this, we design a game-based data collection protocol that elicits varied expressions of knowledge, release the resulting dataset, and provide an initial analysis to highlight its potential for future work on user-knowledge-aware CRS.

Problem

• Estimating user domain knowledge from conversational data
• Addressing lack of datasets capturing varied user expertise levels
• Enabling personalized interactions in conversational recommender systems

Innovation

• Estimating user domain knowledge from conversations
• Designing game-based data collection protocol
• Releasing dataset for user-knowledge-aware CRS