Innocent Panels, Hateful Stories: Evaluating and Detecting Hateful Intent in Multi-Turn Visual Story Generation

2026-08-05

AI Summary

This study addresses the critical challenge that existing content moderation systems struggle to detect group-level hateful intent embedded in multi-turn generated visual narratives, primarily due to their neglect of inter-image narrative coherence. To tackle this gap, we present the first systematic investigation of the problem, introducing a multilingual, multi-style dataset of multi-turn visual hate stories and proposing a dual-stage defense framework. Our approach integrates an interaction-aware monitor during the prompting phase and a vision-language joint model for semantic reasoning over image sequences post-generation. Experiments reveal that state-of-the-art text-to-image models can produce hate narratives with up to 99.0% success, while current moderation systems achieve less than 34.9% recall. In contrast, our method attains recalls of 97.3% and 80.2% at the prompting and post-generation stages, respectively, marking a significant advance from single-image moderation toward stateful, narrative-aware content safety.

Abstract

Picture books and comics have long been used to disseminate hateful narratives because they are easily understood even by children, as exemplified by the notorious Nazi propaganda picture book \emph{Der Giftpilz}. Recently, frontier text-to-image (T2I) systems such as Gemini and GPT-Image have enabled conversational generation with consistent characters and scenes across turns, making hateful visual stories, namely ordered image groups that collectively convey hateful narratives, cheap and scalable to produce. Although prior work has studied hateful content generation by T2I systems, it focuses on individual images, leaving group-level hateful meaning largely unexplored. We aim to address the gap. Concretely, we introduce \texttt{HatefulStoryPrompts}, comprising 330 multi-turn configurations from 55 hateful stories across two languages and three visual styles, and evaluate five frontier models over 4,950 attempts. Every model completes over 80\% of the stories, with the strongest reaching 99.0\%. We further evaluate existing moderation systems on \texttt{HatefulVisualStory}, a human-labeled dataset of 969 hateful image sets and 990 benign controls, and find that they frequently miss group-level hateful meaning: dedicated safety models achieve at most 34.9\% recall, while a strong vision-language model reaches 67.5\%. Finally, we propose complementary proactive and post-generation defenses. An interaction-aware monitor achieves 97.3\% recall for prompt-only sessions and 92.6\% when the user supplies the first image, while post-generation methods jointly analyzing completed image groups reach 80.2\%. Our work shows that, as image generation evolves from isolated outputs to coherent visual narratives, safety must evolve accordingly, from per-image moderation to stateful reasoning over interactions and image relationships.

Problem

• hateful intent
• visual story generation
• multi-turn
• group-level meaning
• content moderation

Innovation

• hateful visual storytelling
• multi-turn image generation
• group-level content moderation
• stateful safety reasoning
• vision-language safety