Thinking with Comics: Enhancing Multimodal Reasoning through Structured Visual Storytelling

2026-02-02

AI Summary

Static images lack temporal structure, while videos suffer from high redundancy and computational overhead that hinder multimodal reasoning efficiency. This work proposes a “comic thinking” paradigm, introducing comics—as a structured visual narrative medium—into multimodal reasoning for the first time to construct an intermediate representation that balances temporal expressiveness with computational efficiency. By designing a comic-based dual-path reasoning mechanism and systematically evaluating it on multi-step causal reasoning, temporal understanding, and long-context tasks, the study demonstrates that the proposed approach significantly outperforms image-based reasoning. It achieves comparable or superior performance while substantially reducing computational costs, thereby validating the effectiveness and potential of comics as an efficient multimodal intermediate representation.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Chain-of-Thought reasoning has driven large language models to extend from thinking with text to thinking with images and videos. However, different modalities still have clear limitations: static images struggle to represent temporal structure, while videos introduce substantial redundancy and computational cost. In this work, we propose Thinking with Comics, a visual reasoning paradigm that uses comics as a high information-density medium positioned between images and videos. Comics preserve temporal structure, embedded text, and narrative coherence while requiring significantly lower reasoning cost. We systematically study two reasoning paths based on comics and evaluate them on a range of reasoning tasks and long-context understanding tasks. Experimental results show that Thinking with Comics outperforms Thinking with Images on multi-step temporal and causal reasoning tasks, while remaining substantially more efficient than Thinking with Video. Further analysis indicates that different comic narrative structures and styles consistently affect performance across tasks, suggesting that comics serve as an effective intermediate visual representation for improving multimodal reasoning.

Problem

• multimodal reasoning
• temporal structure
• visual storytelling
• computational cost
• information density

Innovation

• comics
• multimodal reasoning
• visual storytelling
• chain-of-thought
• temporal reasoning