Persona-Model Collapse in Emergent Misalignment

2026-05-12

AI Summary

This study investigates the emergent behavioral mismatch observed in large language models after fine-tuning on harmful content, attributing its origin to a collapse in role-playing capabilities. To quantify this phenomenon, the work introduces two novel metrics: moral susceptibility (S), measuring a model’s ability to distinguish others’ moral stances during role-play, and moral robustness (R), assessing its capacity to maintain consistent self-identity. Through role-playing experiments grounded in the Moral Foundations Questionnaire, cross-model comparisons, and controlled safety/unsafe fine-tuning conditions, the study finds that unsafe fine-tuning increases S by 55% and reduces R by 65% on average—deviations far exceeding normal ranges—whereas safe fine-tuning leaves R largely unaffected. These results confirm the specificity of the mismatch effect and offer new theoretical insights and diagnostic tools for understanding behavioral degradation in language models.

Abstract

Fine-tuning large language models on narrow data with harmful content produces broadly misaligned behavior on unrelated prompts, a phenomenon known as emergent misalignment. We propose that emergent misalignment involves persona-model collapse: deterioration of the model's internal capacity to simulate, differentiate, and maintain consistent characters. We test this hypothesis behaviorally using two metrics: moral susceptibility (S) and moral robustness (R), computed from the across- and within-persona variability of models' Moral Foundations Questionnaire responses under persona role-play. These metrics formalize the model's ability to differentiate characters (S) and its consistency when simulating a given one (R). We evaluate four frontier models (DeepSeek-V3.1, GPT-4.1, GPT-4o, Qwen3-235B) in three variants: base, fine-tuned to output insecure code, and a matched control fine-tuned to output secure code. Across the four models, insecure fine-tuning produces an average $55\%$ increase in S, pushing all four insecure variants beyond the band observed across 13 frontier models benchmarked in prior work -- with GPT-4o reaching more than twice the band's upper end -- signaling dysregulated differentiation. It also causes an average $65\%$ decrease in R, equivalent to a $304\%$ increase in 1/R. By contrast, the matched secure control preserves S near the base and induces only a partial R loss, showing that these effects are largely misalignment-specific. Complementing these metric shifts, insecure variants' unconditioned responses converge toward saturation near the scale ceiling, departing markedly from both base models' structured responses and those elicited when base models role-play toxic personas. Taken together, these metrics provide a sensitive diagnostic for emergent misalignment and serve as behavioral evidence that it involves persona-model collapse.

Problem

• emergent misalignment
• persona-model collapse
• fine-tuning
• harmful content
• behavioral misalignment

Innovation

• persona-model collapse
• emergent misalignment
• moral susceptibility
• moral robustness
• role-play evaluation