Asymmetric Collapse in Model Merging: When Refusal Over- writes Recognition

2026-07-26

AI Summary

This study addresses the challenge of integrating multidimensional safety capabilities—specifically harm identification and adversarial refusal—through parameter merging. The authors conduct controlled experiments by merging Gemma-3-1B-IT models fine-tuned separately on harm classification (CARES) and adversarial jailbreaking resistance (WildJailbreak) using training-free fusion methods, including Linear, SLERP, TIES, and DARE-TIES. Evaluation across identification accuracy, attack robustness, and instruction following reveals an asymmetric collapse of safety capabilities: despite near-orthogonal task vectors, the stronger magnitude of refusal-related weights dominates the merged model, causing CARES accuracy to plummet to at most 12.9%, while refusal rates remain high at 81–85%. This finding highlights a critical limitation of current merging techniques in preserving fine-grained safety attributes.

Abstract

Model merging is often used to combine capabilities from separately fine-tuned models without additional training, but it is unclear whether standard merging methods preserve multiple safety-relevant behaviors simultaneously. We study this question through a controlled case study using two Gemma-3-1B-IT finetunes on two complementary safety objectives: CARES harm-level classification and WildJailbreak adversarial refusal. We merge the two fine-tunes using Linear, SLERP, TIES, and DARE-TIES, and evaluate the merged models on classification accuracy, attack resistance, and benign compliance. Across all four methods, attack resistance transfers significantly more than classification accuracy: merged models retain 81-85% jailbreak refusal rates while CARES accuracy falls to at most 12.9%. Weight-space measurements suggest that this asymmetry is not caused by strongly opposing task-vector directions: the two task vectors are nearly orthogonal (cosine similarity 0.011). Instead, the refusal fine-tune induces consistently larger per-layer task-vector magnitudes, causing magnitude-sensitive methods to favor refusal updates. These results show that standard model merging can collapse safety recognition into broad refusal when safety-relevant task vectors differ substantially in scale.

Problem

• model merging
• safety alignment
• asymmetric collapse
• refusal behavior
• harm recognition

Innovation

• asymmetric collapse
• model merging
• task vector magnitude
• safety alignment
• refusal behavior