Technology Category
Natural Language Processing: (Large) Language ModelsMachine Learning: Large Multimodal Models (LMMs)Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
Show or Tell? Modeling the evolution of request-making in Human-LLM conversations
🤖 AI Summary
Prior work lacks structured modeling and longitudinal analysis of user requests in human–large language model (LLM) interactions. Method: We propose the novel task of “dialogue query segmentation,” decomposing user inputs into four semantically distinct components—request, role, context, and auxiliary expression—and perform sequence labeling and behavioral pattern tracking on large-scale chat logs. We introduce a diachronic analytical framework to examine temporal evolution in user behavior. Contribution/Results: Our study is the first to empirically demonstrate a dynamic shift from early individual exploration toward collective convergence in user querying behavior; further, LLM capability upgrades significantly reshape prompting patterns, with effects stably observable at the community level. We release the first annotated dataset explicitly designed for behavioral evolution analysis, establishing both a methodological foundation and empirical evidence for understanding human–model co-evolution.
Technology Category
Application Category
📝 Abstract
Chat logs provide a rich source of information about LLM users, but patterns of user behavior are often masked by the variability of queries. We present a new task, segmenting chat queries into contents of requests, roles, query-specific context, and additional expressions. We find that, despite the familiarity of chat-based interaction, request-making in LLM queries remains significantly different from comparable human-human interactions. With the data resource, we introduce an important perspective of diachronic analyses with user expressions. We find that query patterns vary between early ones emphasizing requests, and individual users explore patterns but tend to converge with experience. Finally, we show that model capabilities affect user behavior, particularly with the introduction of new models, which are traceable at the community level.
Problem
Research questions and friction points this paper is trying to address.
Segmenting chat queries into components like requests and roles
Analyzing differences between human-LLM and human-human request-making
Tracking how user query patterns evolve with experience and model updates
Innovation
Methods, ideas, or system contributions that make the work stand out.
Segmenting chat queries into multiple components
Diachronic analysis of user expression patterns
Tracking model impact on user behavior evolution
🔎 Similar Papers
No similar papers found.
