MUSE: Multi-Domain Chinese User Simulation via Self-Evolving Profiles and Rubric-Guided Alignment

2026-04-15

AI Summary

This work addresses critical limitations in existing Chinese multi-domain user simulators, which often suffer from shallow user profiling, inconsistent role portrayal over long conversations, and weak behavioral controllability. To overcome these challenges, the paper proposes a novel user simulation framework that integrates an Iterative Profile Self-Evolution (IPSE) mechanism, role-reversal supervised fine-tuning, and a multi-turn reinforcement learning strategy guided by scoring rules. This approach enables fine-grained behavioral alignment and sustained role consistency across extended dialogues. Experimental results demonstrate that the proposed method significantly outperforms strong baselines at both utterance and conversation levels, generating user behaviors that are more realistic, coherent, and highly controllable.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Multiagent Systems: Agent-Based Simulation and Emergent Behavior
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Web Mining and Content Analysis: Web data generation and simulation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

User simulators are essential for the scalable training and evaluation of interactive AI systems. However, existing approaches often rely on shallow user profiling, struggle to maintain persona consistency over long interactions, and are largely limited to English or single-domain settings. We present MUSE, a multi-domain Chinese user simulation framework designed to generate human-like, controllable, and behaviorally consistent responses. First, we propose Iterative Profile Self-Evolution (IPSE), which gradually optimizes user profiles by comparing and reasoning discrepancies between simulated trajectories and real dialogue behaviors. We then apply Role-Reversal Supervised Fine-Tuning to improve local response realism and human-like expression. To enable fine-grained behavioral alignment, we further train a specialized rubric-based reward model and incorporate it into rubric-guided multi-turn reinforcement learning, which optimizes the simulator at the dialogue level and enhances long-horizon behavioral consistency. Experiments show that MUSE consistently outperforms strong baselines in both utterance-level and session-level evaluations, generating responses that are more realistic, coherent, and persona-consistent over extended interactions.

Problem

• user simulation
• persona consistency
• multi-domain
• Chinese language
• long-horizon interaction

Innovation

• user simulation
• profile self-evolution
• rubric-guided alignment
• multi-domain Chinese dialogue
• reinforcement learning