MLLM-VADStory: Domain Knowledge-Driven Multimodal LLMs for Video Ad Storyline Insights

2026-01-08

AI Summary

This study addresses the lack of systematic understanding and scalable analytical methods for narrative structures in video advertising, which hinders the extraction of effective creative strategies from large-scale data. To bridge this gap, we propose the first multimodal large language model framework that integrates domain-specific advertising knowledge. Our approach segments videos into functional units, introduces a novel taxonomy of ad-specific functional roles, and leverages domain-guided sequential modeling to automatically infer communicative intent and reconstruct narrative storylines. Evaluation on a dataset of 50,000 social media video ads demonstrates that narrative-driven creatives significantly improve viewer retention. Moreover, the high-performing story arcs identified by our method offer actionable insights and practical guidance for designing compelling advertising content.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Search and Retrieval-Augmented AI: Ad search and search for Web retail
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We propose MLLM-VADStory, a novel domain knowledge-guided multimodal large language models (MLLM) framework to systematically quantify and generate insights for video ad storyline understanding at scale. The framework is centered on the core idea that ad narratives are structured by functional intent, with each scene unit performing a distinct communicative function, delivering product and brand-oriented information within seconds. MLLM-VADStory segments ads into functional units, classifies each unit's functionality using a novel advertising-specific functional role taxonomy, and then aggregates functional sequences across ads to recover data-driven storyline structures. Applying the framework to 50k social media video ads across four industry subverticals, we find that story-based creatives improve video retention, and we recommend top-performing story arcs to guide advertisers in creative design. Our framework demonstrates the value of using domain knowledge to guide MLLMs in generating scalable insights for video ad storylines, making it a versatile tool for understanding video creatives in general.

Problem

• video ad storyline
• multimodal LLMs
• functional intent
• domain knowledge
• narrative structure

Innovation

• Multimodal Large Language Models
• Domain Knowledge Integration
• Video Ad Storyline Analysis
• Functional Role Taxonomy
• Data-driven Creative Insights