🤖 AI Summary
This study investigates the effectiveness and boundary conditions of role-playing prompting for reasoning in large language models. To address the limitations of conventional approaches, we propose a metacognition-based "cognitive alignment" hypothesis and introduce a training-free Mixed Language Concatenation Prediction (MLCP) strategy. The proposed method is systematically validated through multi-model cross-domain experiments, entropy divergence analysis, and observations within the latent thought space. Experimental results demonstrate that MLCP consistently outperforms traditional role-playing methods across all evaluated models. Furthermore, this work elucidates the underlying mechanisms and dependency conditions governing role-playing prompts, offering a novel paradigm for prompt optimization in large language models.
📝 Abstract
Role-playing prompting has become a popular yet simple technique for improving LLM reasoning and output quality. However, whether it consistently boosts performance across diverse domains remains unclear, as systematic validation is lacking. To fill this gap, we run multi-model, cross-domain, and multilingual experiments on MMLU and MMLU-Redux. We find that gains from role-play prompting depend heavily on model capacity, knowledge domain, and prompt language. Drawing on metacognition theory, we propose the persona-related cognitive alignment hypothesis: role-play works only when the LLM correctly grasps the designated persona and its associated knowledge domain. We test this hypothesis through persona information richness ablation, layer-wise entropy divergence analysis, and latent thought-space deflection observation. To reduce persona cognitive bias and stabilize role-play performance, we propose \textbf{M}ixed-\textbf{L}anguage \textbf{C}oncatenate \textbf{P}rediction \textbf{(MLCP}), a simple, training-free, and efficient multilingual prompt concatenation strategy. It aggregates semantically equivalent role prompts to enrich complementary representational cues. Extensive experiments show that MLCP consistently outperforms vanilla role-play prompting across all tested LLMs.