Cross-Branch Conflict as a Shield: Safeguarding Facial Identities in Unified Multimodal Image Editing

2026-07-18

AI Summary

This work addresses the vulnerability of unified multimodal image editing models to malicious identity manipulation, a risk inadequately mitigated by existing adversarial defenses. The study reveals that perturbations applied to a single model branch are insufficient to disrupt the unified editing process effectively. To overcome this limitation, the authors propose a novel adversarial protection mechanism that jointly perturbs both the Vision Transformer (ViT) understanding branch and the VAE generation branch. By leveraging linear Centered Kernel Alignment (CKA), the method explicitly disrupts cross-branch representational compatibility and simultaneously corrupts latent representations to eliminate identity-preserving information while introducing visual-contextual inconsistencies. Evaluated across multiple state-of-the-art unified editing models, the approach substantially outperforms current defenses, effectively suppressing identity-preserving edits and significantly enhancing the robustness of facial privacy protection.

Abstract

Unified multimodal models (UMMs) have recently demonstrated powerful instruction-based image editing capabilities, but they also raise serious concerns about unauthorized manipulation of personal portraits. Existing adversarial protection methods are mainly designed for either visual understanding or image generation models and often become ineffective when transferred to UMMs, which process an image through multiple complementary visual pathways. In this work, we first conduct a feature-level analysis of unified image editing. We observe that the ViT-based understanding branch and the VAE-based generation branch exhibit non-trivial structural agreement for the same input image. Although perturbing an individual branch can reduce this agreement and induce intermediate hidden-state deviations, such effects are asymmetric and gradually attenuated during multimodal fusion and generation. These observations reveal that single-branch feature distortion is insufficient for consistently disrupting unified image editing. Motivated by this finding, we propose CCS, a unified adversarial protection framework that jointly drives the ViT and VAE representations away from their clean counterparts while explicitly disrupting their cross-branch compatibility through linear CKA. By simultaneously removing stable information from both visual pathways and creating incompatible visual contexts, CCS prevents the UMM from recovering reliable identity information during editing. Extensive experiments demonstrate that CCS consistently outperforms existing protection methods in suppressing identity-preserving edits.

Problem

• unified multimodal models
• adversarial protection
• facial identity
• image editing
• cross-branch conflict

Innovation

• cross-branch conflict
• unified multimodal models
• adversarial protection
• feature compatibility
• identity preservation