Rushes: A Human Preference Dataset for Pluralistic Alignment

2026-07-22

AI Summary

This study addresses the limitation of current large language models (LLMs) in aligning with human preferences, as they predominantly rely on aggregate population-level signals and fail to capture individualized, context-dependent engagement behaviors. To overcome this, the authors introduce a gamified interface to collect explicit user choice trajectories within AI-generated branching narratives, constructing the first dataset comprising 44,226 timestamped decision events annotated with user identifiers. Empirical analysis reveals that user choices exhibit low-entropy structures. On an event-level prediction task, a singular value decomposition (SVD)-based model achieves 37.7% accuracy, outperforming GPT-5 (34.23%) and even surpassing a popularity-based baseline (36.4%), thereby exposing a significant “engagement gap” in contemporary LLMs’ capacity to model personalized user interactions.

Abstract

We introduce Rushes, a dataset and benchmark for studying revealed human engagement preferences in interactive narrative environments. Rushes is collected through a game interface where users interact with AI-generated branching narratives and select one choice from a small, explicit candidate set at each decision point. Each interaction logs the full candidate set, the user's choice, and the evolving narrative context, yielding time-ordered trajectories with persistent user-level identifiers. Rushes contains 44,226 decision events from 8,167 unique users across six games, capturing sequential, personalized engagement behavior rather than static judgments. We show that user choices exhibit structured, non-random patterns, quantified by a low choice entropy relative to a uniform baseline. We position Rushes as a diagnostic benchmark for pluralistic alignment and demonstrate a robust Engagement Gap: state-of-the-art LLMs, including GPT-5, fail to outperform simple baselines. While classical Matrix Factorization (SVD) captures measurable personalized signal (37.7%), frontier LLMs (34.23%) struggle to even match the Popularity Baseline (36.4%) on event-level choice prediction. This gap suggests that single, population-level objectives, like those used in modern RLHF, appear insufficient to capture heterogeneous, context-dependent engagement signals. As a result, even highly capable models default to majority preferences rather than adapting to individual trajectories. We release Rushes to support research into pluralistic alignment and sequential decision-making in generative systems. The full code for the platform and dataset will be available here: https://github.com/microsoft/rushes

Problem

• pluralistic alignment
• human preference
• interactive narrative
• personalized engagement
• sequential decision-making

Innovation

• pluralistic alignment
• human preference dataset
• interactive narrative
• engagement gap
• sequential decision-making