Facilitating Visual Media Exploration for Blind and Low Vision Users through AI-Powered Interactive Storytelling

2025-08-05

AI Summary

Existing visual media assistive tools fragment exploration and narrative, causing narrative disruption, high cognitive load, and low user engagement—particularly for visually impaired users. Method: We propose an AI-driven interactive narrative paradigm integrating hierarchical, parallel, and branching narrative structures with multimodal perception techniques—including image understanding, temporal synchronization analysis, and 360° video navigation—to enable accessible, coherent, and controllable exploration of visual content. Contribution/Results: This work is the first to jointly embed these three narrative mechanisms into the AI generation pipeline, preserving narrative continuity while granting visually impaired users multi-level, real-time, and autonomous interaction capabilities. Empirical evaluation across diverse visual media demonstrates statistically significant reductions in cognitive load (p < 0.01), alongside improved exploratory freedom and immersion. The framework establishes a scalable theoretical foundation and technical pathway for accessible human–computer interaction.

Technology Category

• Humans and AI: AI for Accessibility
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Empowering blind and low vision (BLV) users to explore visual media improves content comprehension, strengthens user agency, and fulfills diverse information needs. However, most existing tools separate exploration from the main narration, which disrupts the narrative flow, increases cognitive load, and limits deep engagement with visual media. To address these challenges, my PhD research introduces the paradigm of AI-powered interactive storytelling, which leverages AI to generate interactive narratives, enabling BLV users to explore visual media within a coherent storytelling experience. I have operationalized this paradigm through three techniques: (1) Hierarchical Narrative, which supports photo-collection exploration at different levels of detail; (2) Parallel Narrative, which provides seamless access to time-synced video comments; and (3) Branching Narrative, which enables immersive navigation of 360° videos. Together, these techniques demonstrate that AI-powered interactive storytelling can effectively balance user agency with narrative coherence across diverse media formats. My future work will advance this paradigm by enabling more personalized and expressive storytelling experiences for BLV audiences.

Problem

• Enhancing visual media exploration for blind and low vision users
• Reducing cognitive load by integrating exploration with narration
• Enabling immersive engagement with diverse media formats through AI

Innovation

• AI-powered interactive storytelling for BLV users
• Hierarchical Narrative for photo exploration
• Branching Narrative for 360° video navigation