Mixed Reality Scenic Live Streaming for Cultural Heritage: Visual Interactions in a Historic Landscape

2025-11-21

AI Summary

This study addresses three key limitations of scenic live streaming (SLS): low interactivity, shallow cultural immersion, and insufficient collective participation. To overcome these, we propose the Mixed-Reality-enhanced Scenic Live Streaming system (MRSLS)—a novel framework that integrates geospatial structures with indigenous cultural semantics. Operating on unmoderated, fixed-camera real-world streams, MRSLS overlays dynamic, context-aware cultural information via mixed reality (MR) and enables multi-user real-time shared interaction. We evaluated the system through user interviews, iterative prototyping, and a mixed-methods experiment (N=43). Results demonstrate that MRSLS significantly enhances landscape immersion (p<0.01), cultural memory retention (+32.7%), and social co-presence (p<0.05). Critically, it achieves a balanced triad of cultural authenticity, participatory engagement, and perceptual realism. This work advances human–machine collaborative design paradigms for MR-mediated heritage activation and dissemination.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Scenic Live Streams (SLS), capturing real-world scenic sites from fixed cameras without streamers, have gained increasing popularity recently. They afford unique real-time lenses into remote sites for viewers' synchronous and collective engagement. Foregrounding its lack of dynamism and interactivity, we aim to maximize the potential of SLS by making it interactive. Namely MRSLS, we overlaid plain SLS with interactive Mixed Reality content that matches the site's geographical structures and local cultural backgrounds. We further highlight the substantial benefit of MRSLS to cultural heritage site interactions, and we demonstrate this design proposal with an MRSLS prototype at a UNESCO-listed heritage site in China. The design process includes an interview (N=6) to pinpoint local scenery and culture, as well as two iterative design studies (N=15, 14). A mixed-methods, between-subjects study (N=43, 37) shows that MRSLS affords immersive scenery appreciation, effective cultural imprints, and vivid shared experience. With its balance between cultural, participatory, and authentic attributes, we appeal for more HCI attention to (MR)SLS as an under-explored design space.

Problem

• Enhancing scenic live streams with interactive Mixed Reality content
• Improving cultural heritage site engagement through immersive technology
• Addressing lack of dynamism in fixed-camera live streaming systems

Innovation

• Overlaid scenic live streams with interactive Mixed Reality content
• Matched MR content to geographical and cultural site features
• Designed prototype for UNESCO heritage site with user studies