MIRAGE: A Micro-Interaction Relational Architecture for Grounded Exploration in Multi-Figure Artworks

2026-04-26

AI Summary

This work addresses the frequent misinterpretation of subtle interpersonal cues—such as gaze, posture, and spatial arrangement—in multi-character illustrations by current vision-language models, which often lack traceable visual grounding. To mitigate this, the authors propose an evidence-centered, structured framework that decouples spatial localization from narrative generation, explicitly modeling hypotheses about identity, pose, and gaze to construct a verifiable micro-interaction graph. By integrating visual perception, identity tracking, pose estimation, and gaze reasoning, the approach prevents interaction hypotheses from collapsing into weakly grounded, monolithic narratives. This leads to significantly improved identity consistency, reduced relational hallucination, and more comprehensive coverage of nuanced interactions in blind evaluations. The findings suggest that structured grounding can serve as a reliable control layer for human-aligned visual narrative understanding.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Language and Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Appreciating multi-figure paintings requires understanding how characters relate through subtle cues like gaze alignment, gesture, and spatial arrangement. We present MIRAGE, an evidence-centric framework designed to scaffold the exploration of these "micro-interactions" in multi-figure artworks. While such cues are essential for deep narrative appreciation, they are often distributed across complex scenes and difficult for viewers to systematically identify. Existing vision-language models (VLMs) frequently fail to provide reliable assistance, offering ungrounded interpretations that lack traceable visual evidence. MIRAGE addresses this by constructing a structured intermediate representation capturing identities, pose cues, and gaze hypotheses. However, the challenge extends beyond extracting these cues to coordinating them during interpretation. Without an explicit mechanism to organize and reconcile relational evidence, models often collapse multiple interaction hypotheses into a single unstable or weakly grounded narrative, even when low-level signals are available. This representation allows users to verify how high-level interpretations are anchored in low-level visual facts. By separating spatial grounding from narrative generation, MIRAGE enables users to inspect and reason about figure-to-figure relationships through a verifiable evidence layer. We evaluate MIRAGE against painting-only VLM baselines using a blind assessment protocol. Results show that MIRAGE significantly improves identity consistency, reduces relational hallucinations, and increases the coverage of subtle interactions. These findings suggest that structured grounding can serve as a critical interaction control layer, providing the necessary scaffolding for a more reliable, transparent, and human-led understanding of complex visual narratives.

Problem

• micro-interactions
• multi-figure artworks
• visual grounding
• relational reasoning
• narrative hallucination

Innovation

• micro-interactions
• structured grounding
• evidence-centric framework
• relational hallucinations
• visual narrative understanding