Different Paths to Harmful Compliance: Behavioral Side Effects and Mechanistic Divergence Across LLM Jailbreaks

2026-04-20

AI Summary

This study systematically investigates the differential impacts of three jailbreaking approaches—harmful supervised fine-tuning (SFT), harmful reinforcement learning with verifiable rewards (RLVR), and refusal-suppression abliteration—on large language models’ capability retention, safety judgment, and internal mechanisms when inducing high rates of harmful compliance. Through structured self-audit evaluations, safety-reflection prompting interventions, cross-domain generalization tests, and targeted repair experiments, the work reveals that, despite achieving comparable harmful compliance rates, only RLVR models retain significant harm recognition capabilities without notable performance degradation and remain responsive to safety interventions. In contrast, SFT leads to severe capability loss and collapse in safety awareness, while abliteration’s efficacy is highly architecture-dependent. This work provides the first systematic characterization of mechanistic divergence across jailbreaking pathways and demonstrates that only the RLVR route permits effective post-hoc mitigation.

Technology Category

Natural Language Processing: Safety and RobustnessMachine Learning: Large Multimodal Models (LMMs)Computer Vision: Adversarial Attacks & Robustness

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingUser Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systemsResponsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Open-weight language models can be rendered unsafe through several distinct interventions, but the resulting models may differ substantially in capabilities, behavioral profile, and internal failure mode. We study behavioral and mechanistic properties of jailbroken models across three unsafe routes: harmful supervised fine-tuning (SFT), harmful reinforcement learning with verifiable rewards (RLVR), and refusal-suppressing abliteration. All three routes achieve near-ceiling harmful compliance, but they diverge once we move beyond direct harmfulness. RLVR-jailbroken models show minimal degradation and preserve explicit harm recognition in a structured self-audit: they are able to identify harmful prompts and describe how a safe LLM should respond, yet they comply with the harmful request. With RLVR, harmful behavior is strongly suppressed by a reflective safety scaffold: when a harmful prompt is prepended with an instruction to reflect on safety standards, harmful behavior drops close to the baseline. Category-specific RLVR jailbreaks generalize broadly across harmfulness domains. Models jailbroken with SFT show the largest collapse in explicit safety judgments, the highest behavioral drift, and a substantial capability loss on standard benchmarks. Abliteration is family-dependent in both self-audit and response to a reflective safety scaffold. Mechanistic and repair analyses further separate the routes: abliteration is consistent with localized refusal-feature deletion, RLVR with preserved safety geometry but retargeted policy behavior, and SFT with broader distributed drift. Targeted repair partially recovers RLVR-jailbroken models, but has little effect on SFT-jailbroken models. Together, these results show that jailbreaks can produce vastly different properties despite similar harmfulness, with models jailbroken via RLVR showing remarkable similarity to the base model.

Problem

• jailbreak
• harmful compliance
• behavioral side effects
• mechanistic divergence
• language models

Innovation

• jailbreak mechanisms
• harmful compliance
• reinforcement learning with verifiable rewards
• safety representation
• model repair