🤖 AI Summary
This study addresses the latent biases inherent in diverse human-computer interaction datasets, which severely compromise the training and evaluation of user models. We introduce the concept of a "dataset signature" and employ neural network classifiers to perform source attribution across seven conversational datasets, conducting a systematic investigation integrating user modeling and preference analysis. Our results demonstrate that even under a unified taxonomy, individual datasets remain highly distinguishable, and variations in data sources can fundamentally alter conclusions regarding model quality. To address this, we propose a signature-based data filtering strategy. This approach offers a novel paradigm for mitigating dataset bias and enhancing the reliability of downstream tasks.
📝 Abstract
Human--LLM interaction datasets shape our understanding of AI use and provide a foundation for downstream research, including training and evaluation of user models. In recent years, a growing number of datasets have sought to capture a representative picture of human--LLM interactions. But how different are the pictures these datasets provide, and what do those differences mean for research built on them? We study these questions across seven conversation datasets, spanning in-the-wild chat logs and human preference data. We begin by revisiting the dataset classification experiment of Torralba&Efros and find that neural network classifiers identify the source of a conversation from user messages alone well above chance, indicating distinctive dataset signatures. This separability persists after matching datasets on the dimensions of human-designed taxonomies, implying subtle differences that these taxonomies do not capture. We then examine the implications for user modeling: how dataset signatures propagate to the outputs of user models trained on these datasets; how dataset choice influences evaluations of user model quality and subsequent evaluations of LLM assistants paired with these user models; and how dataset classifiers can guide data selection for training user models. While each dataset is meant to capture a slice of'real-world'interactions, our findings reveal the extent to which these slices diverge, and the consequences of those differences for research built on these foundations.