PersLLM: A Personified Training Approach for Large Language Models

2024-07-17

arXiv.org

AI Summary

Existing personalized large language models (LLMs) suffer from insufficient data utilization and rigid personality modeling, hindering consistent viewpoint expression and natural human-AI interaction. To address these challenges, we propose a systematic personality-aware training framework. First, we introduce a novel high-quality personality data construction method integrating chain-of-thought prompting with counter-inductive strategies. Second, we design an automated, DPO-driven dynamic personality fine-tuning pipeline that overcomes the limitations of static fine-tuning. Third, we combine personality-aligned instruction tuning with multi-dimensional human-AI collaborative evaluation. Experimental results demonstrate significant improvements in personality consistency (+23.6%), viewpoint persistence (+31.2%), and interaction naturalness. These gains are rigorously validated through both expert assessments and automated metrics across human-LLM dialogue and multi-agent collaboration scenarios.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Large language models (LLMs) exhibit human-like intelligence, enabling them to simulate human behavior and support various applications that require both humanized communication and extensive knowledge reserves. Efforts are made to personify LLMs with special training data or hand-crafted prompts, while correspondingly faced with challenges such as insufficient data usage or rigid behavior patterns. Consequently, personified LLMs fail to capture personified knowledge or express persistent opinion. To fully unlock the potential of LLM personification, we propose PersLLM, a framework for better data construction and model tuning. For insufficient data usage, we incorporate strategies such as Chain-of-Thought prompting and anti-induction, improving the quality of data construction and capturing the personality experiences, knowledge, and thoughts more comprehensively. For rigid behavior patterns, we design the tuning process and introduce automated DPO to enhance the specificity and dynamism of the models' personalities, which leads to a more natural opinion communication. Both automated metrics and expert human evaluations demonstrate the effectiveness of our approach. Case studies in human-machine interactions and multi-agent systems further suggest potential application scenarios and future directions for LLM personification.

Problem

• Insufficient data usage limits LLM personification potential
• Rigid behavior patterns hinder natural opinion expression
• Challenges in capturing personalized knowledge and experiences

Innovation

• Chain-of-Thought prompting enhances data construction
• Automated DPO improves personality specificity and dynamism
• Anti-induction strategy captures comprehensive personality traits