Problem
Research questions and friction points this paper is trying to address.
Personality-aware fine-tuning
Social simulations
Consistency and controllability
Dialogue generation
Large language models
Do Personality-Tuned LLMs Make Better Social Agents?
🤖 AI Summary
研究通过个性调整微调语言模型，试图提高社交模拟中对话生成的一致性和可控性，但结果显示微调模型在角色扮演上并未优于基线模型。
📝 Abstract
LLMs are increasingly used in social simulations for socially interactive agents and robots, offering more flexibility than rule-based systems. However, even though they mimic human behaviour very well, there is a persistent alienness to them. This work investigates whether personality-aware fine-tuning can reduce this gap by improving the consistency and controllability of personality-conditioned dialogue generation compared with instruction prompting alone. We fine-tune two small open-weight LLMs, Qwen2.5-7B-Instruct and Ministral-8B-Instruct, using a corpus that combines personality-labelled social media posts and dialogues to create a personality-based dialogue engine for social simulation. The resulting models are evaluated across multiple social interaction scenarios using three independent LLM judges, which assess personality fidelity and provide evidence-based behavioral interpretations. We additionally quantify inter-rater agreement and lexical characteristics of the generated dialogue. Results indicate that fine-tuned models are not better at role-playing different personalities than their respective baseline models. However, low inter-rater agreement limits the confidence with which these results can be interpreted. Concerning the quality of generated texts, fine-tuned models are mostly comparable to the baselines, with fine-tuning improving the linguistic diversity of the Qwen models. While the results appear generally usable and the baseline models offer the best overall performance, future studies should place greater emphasis on the quality and domain alignment of training data for accurate personality role-playing.
Innovation
Methods, ideas, or system contributions that make the work stand out.
personality-aware fine-tuning
social simulation
dialogue generation
linguistic diversity
🔎 Similar Papers
No similar papers found.
