BEYOND DIALOGUE: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model

2024-08-20

arXiv.org

AI Summary

Current role-playing models suffer from two key limitations: (1) global inconsistency between predefined character profiles and dialogue content, and (2) absence of sentence-level alignment between profile specifications and utterances. To address these, we propose the Profile-Dialogue Alignment Framework, introducing the novel “Beyond Dialogue” scenario-based task paradigm—extending role grounding beyond turn-level interactions to holistic contextual reasoning. Our approach leverages Reasoning-Augmented Prompting to achieve fine-grained semantic alignment between character profiles and individual utterances. We further design an automated contrastive alignment mechanism and an end-to-end, low-overhead training pipeline. Extensive evaluation across multi-dimensional role consistency benchmarks demonstrates significant improvements over state-of-the-art general-purpose and specialized role-playing models. The codebase and benchmark dataset are publicly released.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Argumentation

Application Category

• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

The rapid advancement of large language models (LLMs) has revolutionized role-playing, enabling the development of general role-playing models. However, current role-playing training has two significant issues: (I) Using a predefined role profile to prompt dialogue training for specific scenarios usually leads to inconsistencies and even conflicts between the dialogue and the profile, resulting in training biases. (II) The model learns to imitate the role based solely on the profile, neglecting profile-dialogue alignment at the sentence level. In this work, we propose a simple yet effective framework called BEYOND DIALOGUE, designed to overcome these hurdles. This framework innovatively introduces"beyond dialogue"tasks to align dialogue with profile traits based on each specific scenario, thereby eliminating biases during training. Furthermore, by adopting an innovative prompting mechanism that generates reasoning outcomes for training, the framework allows the model to achieve fine-grained alignment between profile and dialogue at the sentence level. The aforementioned methods are fully automated and low-cost. Additionally, the integration of automated dialogue and objective evaluation methods forms a comprehensive framework, paving the way for general role-playing. Experimental results demonstrate that our model excels in adhering to and reflecting various dimensions of role profiles, outperforming most proprietary general and specialized role-playing baselines. All code and datasets are available at https://github.com/yuyouyu32/BeyondDialogue.

Problem

• Aligns dialogue with role profiles to reduce training biases
• Ensures sentence-level profile-dialogue alignment for accurate role imitation
• Automates alignment tasks for low-cost general role-playing

Innovation

• Aligns dialogue with profile traits automatically
• Uses beyond dialogue tasks for bias elimination
• Achieves fine-grained alignment via prompting mechanism