PUB: An LLM-Enhanced Personality-Driven User Behaviour Simulator for Recommender System Evaluation

2025-06-05

AI Summary

Traditional recommender system offline evaluation suffers from the sparsity, noise, and coarse-grained user modeling inherent in real-world interaction logs. Existing simulation frameworks struggle to reproduce behavioral diversity and lack psychological interpretability. To address these limitations, we propose the first personalized behavioral simulator integrating the Five-Factor Model (FFM) of personality psychology with large language models (LLMs). Our method dynamically infers user personality traits from real interaction logs and item metadata, then generates high-fidelity, behaviorally diverse synthetic interaction sequences. Crucially, it embeds personality theory deeply into the simulation architecture, enabling personality-driven interaction generation and interpretable association analysis between traits and behavioral metrics. Empirical evaluation on the Amazon dataset demonstrates that synthetic logs closely match real-world statistical distributions. Moreover, we quantitatively uncover—for the first time—significant correlations between openness and conscientiousness traits and key recommendation metrics, including click-through rate and recommendation diversity.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

Traditional offline evaluation methods for recommender systems struggle to capture the complexity of modern platforms due to sparse behavioural signals, noisy data, and limited modelling of user personality traits. While simulation frameworks can generate synthetic data to address these gaps, existing methods fail to replicate behavioural diversity, limiting their effectiveness. To overcome these challenges, we propose the Personality-driven User Behaviour Simulator (PUB), an LLM-based simulation framework that integrates the Big Five personality traits to model personalised user behaviour. PUB dynamically infers user personality from behavioural logs (e.g., ratings, reviews) and item metadata, then generates synthetic interactions that preserve statistical fidelity to real-world data. Experiments on the Amazon review datasets show that logs generated by PUB closely align with real user behaviour and reveal meaningful associations between personality traits and recommendation outcomes. These results highlight the potential of the personality-driven simulator to advance recommender system evaluation, offering scalable, controllable, high-fidelity alternatives to resource-intensive real-world experiments.

Problem

• Captures user personality traits in recommender systems
• Generates synthetic data with behavioral diversity
• Improves evaluation fidelity using LLM-based simulation

Innovation

• LLM-based simulation with Big Five traits
• Dynamic personality inference from logs
• Generates high-fidelity synthetic interactions