A Behavioral Trait Leaks into Preferences: Diagnosing Trait Interference in LLM User Simulators

2026-09-21

AI Summary

论文解决了LLM用户模拟器中特质干扰导致的偏好偏差问题，提出了一种基于页面质量锚定的方法PQA来指导模拟器，从而提高评估可靠性。

Abstract

LLM-based user simulators aim to bridge the offline-online gap in recommender evaluation by emulating users through injected traits, where preference attributes determine what a user engages with and a behavioral activity trait governs how long they browse. However, we show this intended trait independence collapses during simulation, causing two failures: (i) Trait Interference, where amplified activity distorts preference boundaries and forces interactions with mismatched items to sustain browsing, and (ii) Evaluation Invalidity, where satisfaction scores inflate with activity-driven page counts despite taste mismatches, biasing evaluation toward trait distributions rather than recommender performance. To resolve this, we propose PQA, a page-level quality anchoring method that guides simulators using a personalized anchor reflecting each user's intrinsic preference standard. By assessing whether a page meets this standard before further browsing, PQA enables proactive exits from low-quality pages, letting the activity trait retain its intended role of modulating browsing depth within preference-conforming pages. Experiments show PQA mitigates trait interference and improves the reliability of LLM-based simulator evaluation under activity shifts. Our code is available at https://github.com/chaehyun1/PQA

Problem

• Trait Interference
• Evaluation Invalidity
• LLM-based user simulators

Innovation

• PQA
• Trait Interference
• Page-level Quality Anchoring