Co-Designing Multimodal Systems for Accessible Remote Dance Instruction

2025-11-12

AI Summary

Blind and low-vision learners face significant barriers in accessing visually dominated remote video-based dance instruction, particularly in perceiving movement articulation and rhythmic timing. Method: We propose a multimodal accessible remote dance teaching framework, co-designed through three iterative participatory workshops (N=28), integrating audio description, spatialized audio, haptic feedback, and adaptive multimodal interaction. The framework supports staged learning, movement vocabulary construction, and dynamic modality selection to jointly convey dance structure, spatial relationships, and temporal rhythm. Contribution/Results: It is the first systematic approach enabling accessible instruction for high-complexity dance movements. We developed eight functional prototypes and distilled six generalizable design principles. Empirical evaluation demonstrates significant improvements in movement comprehension accuracy and temporal sequencing proficiency. This work advances theoretical foundations and practical paradigms for disability-inclusive arts education.

Technology Category

• Humans and AI: AI for Accessibility
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Security and Privacy: Data transparency and provenance

Abstract

Videos make exercise instruction widely available, but they rely on visual demonstrations that blind and low vision (BLV) learners cannot see. While audio descriptions (AD) can make videos accessible, describing movements remains challenging as the AD must convey extit{what} to do (mechanics, location, orientation) and extit{how} to do it (speed, fluidity, timing). Prior work thus used extit{multimodal instruction} to support BLV learners with individual simple movements. However, it is unclear how these approaches scale to dance instruction with unique, complex movements and precise timing constraints. To inform accessible remote dance instruction systems, we conducted three co-design workshops (N=28) with BLV dancers, instructors, and experts in sound, haptics, and AD. Participants designed 8 systems revealing common themes: staged learning to dissect routines, crafting vocabularies for movements, and selectively using modalities (narration for movement structure, sound for expression, and haptics for spatial cues). We conclude with design recommendations to make learning dance accessible.

Problem

• Making remote dance instruction accessible for blind learners
• Describing complex dance movements with precise timing constraints
• Scaling multimodal systems beyond simple movements to dance

Innovation

• Co-designed multimodal systems with BLV dancers
• Staged learning dissects complex dance routines
• Narration sound haptics convey structure expression cues