Problem
Research questions and friction points this paper is trying to address.
Role-Playing Agents
Reinforcement Learning
Distributional Bottleneck
Curriculum Learning
Adversarial Closed-Loop Curriculum for Evolving Role-Playing Agents
🤖 AI Summary
This study addresses the distribution bottleneck in reinforcement learning (RL) for role-playing agents, where a fixed scenario pool prevents training data from dynamically focusing on areas of weakness as model capabilities improve. To overcome this limitation, we propose AdvRole, a framework that introduces an adversarial context rewriting mechanism driven by performance-gap rewards. By alternately optimizing the policy model and the scenario generator, AdvRole reformulates role-playing RL into closed-loop curriculum learning, enabling the co-evolution of scenario distributions and model proficiency. Extensive experiments conducted on Chinese, English, and newly released multilingual benchmarks demonstrate that AdvRole consistently outperforms existing baselines, yielding substantial improvements in the role-playing capabilities of agents.
📝 Abstract
Role-playing agents based on large language models have been widely applied in areas such as personalized assistance and social simulation. Recent RL methods typically train on a fixed scenario pool collected before learning begins. This creates a distributional bottleneck: as the agent improves, the scenarios where it performs poorly also change, while the training distribution remains static. Therefore, we propose AdvRole, an adversarial context rewriting framework that turns role-playing RL into a closed-loop curriculum. AdvRole alternates between an Actor that learns to role-play and a Rewriter that edits character profiles and dialogue contexts into actor-specific hard scenarios. The Rewriter is trained with a performance-gap reward, which favors rewrites that reduce the current Actor's score relative to the original scenario. As a result, the scenario pool evolves with the Actor and continuously targets under-mastered regions of the character-context space. Experiments on three role-playing benchmarks covering English and Chinese, as well as a new multilingual benchmark we release, show that AdvRole consistently outperforms baselines.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Adversarial Curriculum Learning
Closed-loop Reinforcement Learning
Context Rewriting
Role-playing Agents
Performance-gap Reward
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
No related jobs found.
