A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combination

2026-08-02

AI Summary

This work addresses the vulnerability of multimodal large language models to jailbreak attacks, noting that existing red-teaming approaches lack a systematic atomic strategy space and an efficient automation framework. To bridge this gap, the study introduces the first cross-modal atomic jailbreak strategy space, decomposing text-and-image attack strategies into structural, semantic, and syntactic levels. It further proposes a Hierarchical Atomic Compositional Attack (HACA) framework that enables automated selection, composition, and instruction generation of textual and visual strategies through six-dimensional strategy modeling, a cross-modal joint planner, and a unified generator. Evaluated on five mainstream models, HACA achieves an average attack success rate of 95.48%, substantially outperforming current manual or semi-automated methods and enabling comprehensive, systematic, and automated red-teaming evaluation.

Abstract

Multimodal Large Language Models (MLLMs) have achieved impressive progress in image-text comprehension and generation, yet they remain susceptible to jailbreak attacks that can trigger harmful outputs and pose serious safety concerns. Existing multimodal jailbreak attacks have shown the feasibility of such attacks, but they still face two fundamental challenges: the lack of a atomic multi-modal strategy space, the absence of a concise and efficient executable framework beyond human-craft experience. To address these challenges, we first decompose the text-image jailbreak strategy space into three levels: structural, semantic, and syntactic, constructing a jailbreak strategy set encompassing both text and image modalities to systematically achieve combined coverage of different attack types. Then we propose a multimodal automated red team jailbreak method named Hierarchical Atomic Combination Attack (HACA). Specifically, based on a six-dimensional strategy space, a cross-modal joint planner is used to select and combine the different atomic jailbreak strategy for subsequent jailbreak command generation. Finally, at the implementation level, we explore to apply a unified generate executor to directly generate jailbreak instructions based on the selected multi-modal strategies. A series of experiments show that our automated red team method can achieve an attack success rate of average 95.48\% against five mainstream MLLMs.

Problem

• Multimodal Large Language Models
• jailbreak attacks
• redteaming evaluation
• atomic strategy
• multimodal safety

Innovation

• multimodal jailbreak
• atomic strategy decoupling
• automated red teaming
• cross-modal planning
• Hierarchical Atomic Combination Attack